A. Biometric Systems With Verification/Identification

Depending on the context, the two categories of biometric systems are: 1) verification systems and 2) identification systems, summarised in Figure 1 [6]. Verification refers to validating a person’s identity based on their individual characteristics, which are stored/registered on a server. In technical terms, this type of a biometric system performs a one-to-one matching between the ‘claimed’ and ‘registered’ data, in order to determine whether the claim is true. In other words, the question asked for this application is ‘Is this person A?’ as illustrated in Figure 1 (top panel). In contrast, an identification system confirms the identify of an individual from cross-pattern matching of the all available information, that is, based on one-to-many template matching. The underlining question for this application is ‘Who is this person?’ as illustrated in Figure 1 (bottom panel).

Fig. 1.

Person recognition systems. Top: Verification system. Bottom: Identification system.

Show All

B. Feasible EEG Biometrics Design

Traditionally, EEG-based biometrics research has been undertaken based on both publicly available datasets [11] and custom recordings as part of research efforts [13]. However, most of existing studies failed to rigorously address the key criterion, collectability, which is also related to repeatability. A large number of studies, especially those conducted at the dawn of EEG biometrics research, employed classification of the clients based on supervised learning with the training and validation data coming from the same recording trial. However, this experimental setup cannot truly evaluate the performance in identifying individual features, since such classification does not take into account the varying characteristic among multiple recording trials and recording days. In addition, EEG is prone to contamination by artefacts from subjects’ movements (e.g. eye blinks, chewing), while the sources of external noise include electrode noise, power line noise, and electromagnetic interference from the surroundings. This opens the possibility to additionally incorrectly associate ‘EEG patterns’ with either trial-dependent features or so-called noise-related features – in other words, this setup is biased in favour of high classification rate. Therefore, given the notorious variability of EEG patterns across days, biometrics studies based on a single recording day (even for a single subject) can only validate very limited scenarios, without any notion of repeatability and long-term feasibility [13].

Figure 2 shows the concept of a rigorous EEG biometrics verification/identification system in the real-world. Individuals participate in EEG recordings and their EEG signals are registered and stored on a server or in a database (left panel). The client is granted access to their account by providing new EEG data in verification scenarios, whereby the algorithm discriminates the identify of an individual is in identification scenarios through new EEG recordings. Recall that the registered EEG must be recorded beforehand.

Fig. 2.

Feasible EEG biometrics verification framework. Left: EEG recording registration. The registered EEG signal must have been recorded beforehand. Right: Verification and identification system.

Show All

In order to fulfil the feasibility requirement, several studies performed successful EEG-based biometrics from multiple recording trials conducted on multiple distinct days, thus satisfying the collectability requirement. However, the majority of these studies were still conducted in an unrealistic scenario, whereby the training and validation data in the classification process are split into segments, with all the segments coming from multiple trials but on the same recording day being randomly assigned to the training and validation datasets. Therefore, this biased setup, despite being based on the classification from multiple recording trials, mixes the training and test recordings from the same recording day and thus cannot truly evaluate the performance in the identification of individual features.

In order to truly validate the robustness of a EEG biometrics application, within a rigorous setup, it is therefore necessary to both: i) conduct multiple recordings over multiple days, and ii) to assign recordings on one day as the training data and use the recordings from the other days as the validation data. In other words, the training and validation datasets should be created so as not to share the same recording days (as illustrated later in Figure 6, Setup-R). As emphasised by Rocca et al., the issue of the repeatability of EEG biometrics in different recording sessions is still a critical open issue [14].

Fig. 6.

Two validation scenarios (Setup-R and Setup-B), where $X_{R}^{(i,j,k)} \in \mathbb {R} ^ {N \times D}$ and $X_{N}^{(i,-,k)} \in \mathbb {R} ^ {N \times D}$ denote a feature matrix from a single trial of the subject $i$ , recording day $j$ , and trial $k$ , from the $S_{R}$ recordings and the $S_{N}$ recordings, respectively. The number of segments per recording trial $N$ depends on the chosen segment lengths $L_{seg}$ . The dimension $D$ is the number of features per EEG segment.

Show All

C. Previous Protocols

Table I summarises the state-of-the-art of the existing EEG biometrics applications based on multiple data acquisition days.

TABLE I Recording and Classification Set-Up, and Performance Comparison Among Existing EEG Biometrics

D. Biometrics Based on Collectable EEG Systems

With a perspective of collectability, a biometrics application with dry EEG electrodes was recently introduced [23]. While conventional wet EEG headsets require the application of a conductive gel which is generally time-consuming, the dry headset with 16 scalp channels took on the average 2 minutes to be operational. The brain-computer interface based biometrics application with rapid serial visual presentation paradigm achieved CRR = 100% with 27 s window size over all 29 subjects. Although the recordings were performed over a single recording day per subject, the application with a dry headset was a step forward towards establishing collectable EEG biometrics in real-world.

In a recent effort to enable collectable EEG, the in-ear sensing technology [12] was introduced into the research community. The ear-EEG has been proven to provide on-par signal quality, compared to conventional scalp-EEG, in terms of steady state responses [12], [24], monitoring sleep stages [25], [26], and also for monitoring cardiac activity [27], [28]. The advantages of the in-ear EEG sensing for a potential biometrics application in the real-world are:

• Unobtrusiveness: The latest ‘off-the-shelf’ generic viscoelastic EEG sensor is made from affordable/consumable standard earplugs [29],

• Robustness: The viscoelastic substrate expands after the insertion, so the electrodes fit firmly inside the ear canal [27], where the position of electrodes remains the same in different recording sessions,

• User-friendliness: The sensor can be applied straightforwardly by the user, without the need for a trained person.

E. Problem Formulation

We investigate the possibility of biometrics verification with a wearable in-ear sensor, which is capable of fulfilling the collectability requirement. The data were recorded over temporally distinct recording days, in order to additionally highlight the uniqueness and permanence aspects. Although the changes in EEG rhythms may well depend on the time period of years rather than days, the alpha band features during the resting state with eyes closed were reported as the most stable EEG feature over two years [31]. Since EEG alpha rhythms predominantly originate during wakeful relaxation with eyes closed, we chose our recording task to be the resting state with eyes closed. This task was used in multiple previous studies [15]–​[17], [19], [20], [22]. In order to design a feasible biometrics application in the real-world, we considered imposters in two different ways: i) registered subjects in a database, and ii) subjects not belonging to a database. Previously, Riera et al. [16] also used a single trial of EEG recording from multiple subjects as ‘intruders’, while the ‘imposters’ data were EEG recordings available from multiple other experiments. For rigour, we collected two types of data: 1) based on multiple recordings from fifteen subjects over two days, and 2) multiple recordings from five subjects, which were only used for imposters’ data. The classification was performed by both a non-parametric and parametric approach. The non-parametric classifier, minimum cosine distance, is a simplest way for evaluating the similarity between the training and validation matrix, whereas the parametric approach, the support vector machine (SVM), was tuned within the training matrix in order to find optimal hyper-parameters and weights for validation. The same hyper-parameters and weights were used for classifying the validation matrix. Besides, the linear discriminant analysis (LDA) was also employed as a classifier. Through the binary client-imposter classification, we then evaluated the feasibility of our in-ear EEG biometrics.

A. Data Acquisition

The recordings were conducted at Imperial College London, for two different groups of subjects and under the ethics approval, Joint Research Office at Imperial College London ICREC12_1_1. One set of data were the recordings used as both clients and imposters data, denoted by $S_{R}$ , and the other subset were the recordings for only imposters’ data, denoted by $S_{N}$ . Table II summarises the two recording configurations of $S_{R}$ and $S_{N}$ .

TABLE II Two EEG Recordings and Corresponding Subset

For the $S_{R}$ subset of recordings, fifteen healthy male subjects (aged 22–38 years) participated in two temporally separate sessions, with the interval between two recording sessions between 5 and 15 days, depending on the subject. The participants were seated in a comfortable chair during the experiment, and were asked to rest with eyes closed. The length of each recording was 190 s, and the recording was undertaken three times (trials) per one day. The interval between each recording trial was approximately 5 to 10 minutes. In total, six trials were recorded per subject. The in-ear sensor was inserted in the subject’s left ear canal after earwax was removed; it then expanded to conform to the shape of the ear canal. The reference gold-cup standard electrodes were attached behind the ipsilateral earlobe and the ground electrodes were placed on the ipsilateral helix. For simplicity, the upper electrode is denoted by Ch1, while Ch2 refers to the bottom electrode, as shown in Figure 3 (left panel). The two EEG signals from flexible electrodes were recorded using the g.tec g.USBamp amplifier with a 24-bit resolution, at a sampling frequency $fs =$ 1200 Hz.

Fig. 3.

The in-ear sensor used in our study. Left: Wearable in-ear sensor with two flexible electrodes. Right: Placement of the generic viscoelastic earpiece.

Show All

For the $S_{N}$ subset of recordings, five healthy subjects (aged 22–29 years) participated in three recording trials. Similar to the $S_{R}$ subset of recordings, the participants were seated in a comfortable chair, and were resting with eyes closed. The duration of recording was also 190 s. A generic earpiece with two flexible electrodes [29] was inserted in the subject’s left ear canal and the same reference and ground configuration was utilised for the $S_{R}$ subset of recordings.

Similar to the setup in [22], there was no restriction on the activities that the subjects performed, and no health test such as their diet and sleep, was carried out neither before or between an EEG acquisition and the following one, nor during the days of the recordings. This lack of restrictions allowed us to acquire data in conditions close to real life.

B. Ear-EEG Sensor

The in-ear EEG sensor is made of a memory-foam substrate and two conductive flexible electrodes, as shown in Figure 3. The substrate material is a viscoelastic foam, therefore the ‘one-fits-all’ generic earpiece fits any ear regardless of the shape. The size of earpiece was the same for over twenty subjects (both the $S_{R}$ and $S_{N}$ subjects). Further details of the construction of such a viscoelastic earpiece and its detailed recordings of various brain functions can be found in [27] and [29].

C. Pre-Processing

The two channels of the so-obtained ear-EEG were analysed based on the framework illustrated in Figure 4. In each recording, for both the $S_{R}$ and $S_{N}$ recordings, the first 5 s of recording data were removed from the analyses, in order to omit noisy recordings arising at the beginning of the acquisition. The two recorded channels of EEG were bandpass filtered with the fourth-order Butterworth filter with the pass-band 0.5–30 Hz. The bandpass filtered signals were split into segments. The symbol $N$ denotes the number of segments per recording trial from both the $S_{R}$ and $S_{N}$ subsets. The lengths of segments were chosen as $L_{seg} = 10, 20, 30, 60, 90$ s. Therefore, when the segment length was $L_{seg} =$ 60 s, $N=3$ (190 – 5 = 185 s, $\lfloor{185/60}\rfloor = 3$ ) segments were extracted from every recording trial of the $S_{R}$ and $S_{N}$ . Within each segment, the data was split into epochs of 2 s length. The epochs with the amplitudes of greater than 50 $\mu \text{V}$ for either Ch1 or Ch2 were considered corrupted by artefacts and removed from the analyses. This method resulted in a loss of 4.3 % of the data, namely approximately 7.7 s out of 190 s per recording trial.

Fig. 4.

Flowchart for the biometrics analysis framework in this study.

Show All

D. Feature Extraction

After the pre-processing, two types of features were extracted from each segment of the ear-EEG. For a fair comparison with the state-of-the-art, these features were selected to be the same or similar to those used in the recent studies based on the resting state with eyes closed [19], [20], and included: 1) a frequency domain feature – power spectral density (PSD), and 2) coefficients of an autoregressive (AR) model.

1) The PSD Features:

Figure 5 shows power spectral density for the in-ear EEG Ch1 (left) and for the in-ear EEG Ch2 (right) of two subjects. For this analysis, the recorded signals were conditioned with the fourth-order Butterworth filter with the pass-band 0.5–30 Hz. The PSD were obtained using Welch’s averaged periodogram method [32], the window length was 20 s with 50% of overlap. The PSDs are overlaid between different recording days (red: Day1, blue: Day2), as well as among different recording trials with the same recording days, especially visible from 3 to 20 Hz. Previously, Maiorana et al. utilised PSD features for EEG biometrics based on the resting state eyes closed and achieved the best performance between the PSD features from theta to beta band, which was classified by the minimum cosine approach [22]; the inclusion of the delta band decreased their identification performance. In our in-ear EEG biometrics approach, the obtained PSDs were visually examined and we found that the ratio between the the total $\alpha$ band (8 – 13 Hz) power and the total $\theta -\alpha _{high}$ band (4 – 16 Hz) power is a relatively more significant individual factor for biometrics, rather than the total $\alpha$ band (8 – 13 Hz) power, which is proposed in [19]. Therefore, in each segment of length $L_{seg}$ , univariate PSD was calculated by Welch’s method with 2 s of the window length and no overlap. Three features were obtained for each PSD: 1) The ratio between the total $\alpha$ band (8 – 13 Hz) power and the total $\theta -\alpha _{high}$ band (4 – 16 Hz) power, 2) the maximum power in $\alpha$ band, and 3) the frequency corresponding to the maximum of $\alpha$ band power. In total, $D = 6$ (three features $\times$ two channels) frequency domain features were extracted from each segment.

Fig. 5.

Power spectral density for the in-ear EEG Ch1 (left) and the in-ear EEG Ch2 (right) of Subject 1 (top panels) and Subject 2 (bottom panels). The thick lines correspond to the averaged periodogram obtained by the all recordings from the 1st day (red) and the 2nd day (blue), whereas the thin lines are the averaged periodogram obtained by a single trial.

Show All

E. Validation Scenarios

With the extraction of the both univariate AR and PSD features from two channels, the dimension $D$ of features per EEG segment was twenty six. Recall that the first 5 s of recording data were removed from the analyses. For each trial of the $S_{R}$ and $S_{N}$ recordings, the data with the duration of 190 s was split into segments of length $L_{seg} = 10, 20, 30, 60, 90$ s. Therefore, $N = 18, 9, 6, 3, 2$ segments were respectively obtained. Each recording trial was represented by the feature matrix $X_{R}$ for the $S_{R}$ recordings and $X_{N}$ for the $S_{N}$ recordings, such matrices have $N \times D$ elements. In this way, a set of six feature matrices was obtained from one subject for the $S_{R}$ recordings (three recording trials per one day, over two different days), whilst a set of three feature matrices was obtained from one subject for the $S_{N}$ recordings (three recording trials per one day, one recording day). We next discuss the use of feature matrices in two different validation scenarios.

As emphasised in Introduction, we introduce a feasible EEG biometrics which satisfies the collectability requirements, which are also related to repeatability. Therefore, for rigour, we used all feature matrices $X_{R}$ from the 1st day of recordings as the training data, and feature matrices from the 2nd day of recordings as the validation data, and vice versa (Setup-R).¹ Our goal was to examine the robustness of the proposed approach over the two different time periods in Setup-R. For the second setup, Setup-B,¹ training feature matrices were also selected from the trials which were recorded at the same recording day as the validation matrices. Namely, the training and validation data are split by mixing the data from the same recording days. Notice that, although used in most available EEG biometrics studies [15]–​[17], Setup-B could not evaluate the repeatability/reproducibility of the application, because the training and validation data were both from the same recording days. In other words, such an approach benefits from the recording-day-dependent EEG characteristic in the classification. However, as the number of feasible biometric modalities with in-ear EEG sensor is limited and for comparison with other studies, for convenience we also provide the results for Setup-B.

Figure 6 summarises the two validation scenarios, Setup-R and Setup-B. For clarity, we denote by $VC$ the validation feature matrix for the client, by $VI$ the validation feature matrix for the imposters, while $TC$ is the training feature matrix for the client, and $TI$ as the training feature matrix for the imposters. The feature matrix from a single trial of the subject $i$ , recording day $j$ , and trial $k$ , from the $S_{R}$ recordings is denoted by $X_{R}^{(i,j,k)}$ . Then, the training feature matrix $Y_{T}$ and the validation matrix $Y_{V}$ are given as

View Source\begin{align*} Y_{T}=&[Y_{TC}^{T}, Y_{TI}^{T}] ^ {T}, \\ Y_{V}=&Y_{V_{R}} = [Y_{VC}^{T}, Y_{VI}^{T}] ^ {T}. \end{align*}

$$\begin{equation*} Y_{V} = Y_{V_{R}} + Y_{VI_{N}} = [Y_{VC}^{T}, Y_{VI}^{T}, Y^{T}_{VI_{N}} ] ^ {T}. \end{equation*}$$ View Source\begin{equation*} Y_{V} = Y_{V_{R}} + Y_{VI_{N}} = [Y_{VC}^{T}, Y_{VI}^{T}, Y^{T}_{VI_{N}} ] ^ {T}. \end{equation*}

TABLE III Dimensions of the Training and Validation Matrix in Setup-R and Setup-B

F. Classification

For both the Setup-R and Setup-B, we selected every trial from every subject for the validation of client data, so as to have validated ninety times (three trials $\times$ two days $\times$ fifteen subjects). For each validation, both the largest and smallest values were found for each feature (column-wise) from the training matrix, then the validation matrix was normalised to the range [0, 1] based on these largest and smallest values. Three classification algorithms were employed: 1) a non-parametric approach – minimum cosine distance [22], 2, 3) parametric approaches – linear discriminant analysis (LDA) [33] and support vector machine (SVM) [34].

1) Cosine Distance:

The cosine distance is the simplest way for evaluating the similarity between the rows of the validation matrix, $Y_{V_{(l,:)}}$ , where $l = 1, \ldots , 15N$ for $Y_{V} = Y_{V_{R}}$ and $l = 1, \ldots , 20N$ for $Y_{V} = Y_{V_{R}} + Y_{VI_{N}}$ , and the training matrix, $Y_{T}$ , and is given by

$$\begin{equation*} d\left ({Y_{V_{(l,:)}}, Y_{T}}\right ) = \min _{n} \frac {\sum _{m=1}^{D} Y_{V_{(l,m)}} Y_{T_{(n,m)}} }{\sqrt {\sum _{m=1}^{D} (Y_{V_{(l,m)}})^{2}}\sqrt {\sum _{m=1}^{D} (Y_{T_{(n,m)}})^{2}}}. \end{equation*}$$ View Source\begin{equation*} d\left ({Y_{V_{(l,:)}}, Y_{T}}\right ) = \min _{n} \frac {\sum _{m=1}^{D} Y_{V_{(l,m)}} Y_{T_{(n,m)}} }{\sqrt {\sum _{m=1}^{D} (Y_{V_{(l,m)}})^{2}}\sqrt {\sum _{m=1}^{D} (Y_{T_{(n,m)}})^{2}}}. \end{equation*} In other words, the cosine distance is used for evaluating the similarity between a given test sample (e.g $l$ th row of the validation matrix, $Y_{V_{(l,:)}}$ ) and a template (training) feature matrix, $Y_{T}$ . The distances between the $l$ th row of the validation matrix, $Y_{V_{(l,:)}}$ , and the each row of training matrix $Y_{T}$ were first computed, then the minimum among the computed distances was selected.

3) SVM:

The binary-class SVM was employed as a parametric classifier [34]. For both Setup-R and Setup-B, four hyper-parameters: type of kernel, regularisation constant for loss function $C$ , inverse of bandwidth $\gamma$ of kernel function, and order of polynomial $d$ , were tuned by 5-fold cross-validation within the training matrix. Then, the same hyper-parameters were used in order to obtain the optimal weight parameters within the training matrix. The same hyper-parameters and weight parameters as in the training were used for validation. Table IV summarises the hyper-parameters for SVM.

TABLE IV Hyper-Parameters for SVM

G. Performance Evaluation

Feature extraction and classification with minimum cosine distance and with LDA was performed using Matlab 2016b, and the classification with SVM was conducted in Python 2.7.12 Anaconda 4.2.0 (x86_64) operated on an iMac with 2.8GHz Intel Core i5, 16GB of RAM.

For the verification setup (the number of classes $M=2$ , client-imposter classification), the performance was evaluated through the false accept rate (FAR), false reject rate (FRR), half total error rate (HTER), accuracy (AC), and true positive rate (TPR), defined as:

View Source\begin{align*} FAR=&FP/(FP+TN), \quad FRR = FN/(TP+FN),\, \\ HTER=&\frac {FAR + FRR}{2},\quad AC = \frac {TP+TN}{TP+FN+FP+TN}, \\ TPR=&TP/(TP+FN). \end{align*}

For the identification setup ($M=15$ ), the performance was evaluated by subject-wise sensitivity (SE), identification rate (IR) and Cohen’s kappa ($\kappa$ ) coefficient as:

View Source\begin{align*} SE_{i}=&TP_{i}/(TP_{i}+FN_{i}), \quad IR = \frac {\sum _{i = 1}^{15} TP_{i}}{N_{segment}}\\ \pi _{e}=&\frac {\sum _{i = 1}^{15} \left \{{(TP_{i} + FP_{i}) (TP_{i} + FN_{i})}\right \}}{{N_{segment}}^{2}}, \quad \kappa = \frac {IR - \pi _{e}}{1 - \pi _{e}}, \end{align*}

A. Client-Imposter Verification With Different Segment Sizes

Table VI summarises validation results for both Setup-R and Setup-B, over different segment sizes $L_{seg} = 10, 20, 30, 60, 90$ s. Both the PSD and AR features were used. The elements in TP, FN, FP and TN columns denote the number of segments classified by a validation stage. The chance level in this scenario is 14/15 = 93.3%; this is because every subject is once used for client data, $V_{VC} \in \mathbb {R}^{N \times D}$ in Table III, and imposter data are selected from the other ‘non-client’ subjects, $V_{VI} \in \mathbb {R}^{14N \times D}$ in Table III. The ratio between $V_{VC}$ and $V_{VI}$ is therefore 1:14, and thus the chance level is 14/15. In Setup-R, the results with $L_{seg} = 60$ s achieved both the best HTER score, 17.2%, and the best accuracy (AC), 95.7%. Notice that, the number of TP (=183) is larger than FN + FP (=174) with $L_{seg} = 60$ s, therefore, the likelihood of making a true positive verification is higher than making a false verification. In Setup-B, the results with $L_{seg} = 90$ s obtained both the best HTER score, 6.9%, and the best accuracy (AC), 98.3%.

TABLE VI Client-Impostor Verification Over Different Segment Sizes

B. Client-Imposter Verification With Different Features

Table VII shows the validation results in Setup-R, and over a range of different selections of features, such as AR coefficients, frequency band power, and the combination of AR and band power features for the segment length of $L_{seg} = 60$ s. The classification results using both AR features and PSD features were the highest in terms of both HTER and AC, which corresponds to Table VI (upper-panel), for $L_{seg} = 60$ s.

TABLE VII Rigorous Setup: Client-Imposter Verification Over Different Features in Setup-R

C. Client-Imposter Verification With Different Classifiers

Table VIII shows the imposter-client verification accuracy based on the minimum cosine distance, LDA, and SVM, for both Setup-R and Setup-B, with a segment size of $L_{seg} = 60$ s. Both the PSD and AR features were used. In Setup-R, the results with cosine distance were the best in terms of both HTER score, 17.2% and AC, 95.7%. In Setup-B, the results of both HTER and AC were the best based on the SVM classifier, 5.5% and 99.0%, respectively.

TABLE VIII Client-Impostor Verification With Different Classifiers

D. Validation Including Non-Registered Imposters

Table IX summarises the confusion matrices of both Setup-R and Setup-B with segment sizes $L_{seg} = 60$ s, classified by the minimum cosine distance, LDA, and SVM; these correspond to Table VIII, panels Setup-R and Setup-B. The confusion matrices were categorised into:

• Client matrix $Y_{VC}$ from dataset $S_{R}$ ,

• Imposter matrix $Y_{VI}$ from dataset $S_{R}$

• Imposter matrix $Y_{VI_{N}}$ from dataset $S_{N}$ .

TABLE IX Confusion Matrix of the Client-Imposter Verification Scenario from Different Datasets in Both Setup-R and Setup-B

In Setup-R, the TPR of client $Y_{VC}$ , achieved by the minimum cosine distance was the highest, with respective value of 67.8%. However, the TPRs obtained by SVM for imposters $Y_{VI}$ and $Y_{VI_{N}}$ were 98.6% and 96.2%, respectively, which was higher than those achieved by LDA. In Setup-B, both the TPR of client $Y_{VC}$ and that of imposters $Y_{VI}$ and $Y_{VI_{N}}$ by SVM were the highest, with respective values of 89.3%, 99.7% and 96.3%.

E. Client-Imposter Verification Results per Subject

Table X (middle columns) summarises the subject- and day-wise validation results with PSD and AR features from $L_{seg} =$ 60 s segments in Setup-R, which corresponds to Table VI (upper-panel) for $L_{seg} =$ 60 s. The first and second columns in the Verification part show respectively subject-wise HTER and AC, with the training matrix $Y_{T}$ selected from all the first day recordings and classified based on the second day recordings. The third and fourth columns in the Verification part show classification results obtained based on the training matrix $Y_{T}$ , which was selected from all the second day recordings in order to classify the first day recordings.

TABLE X Recording Details, Accuracy and HTER in the Verification Problem, and Sensitivity in the Identification Problem for Each Subject With Different Training and Validation Data in Setup-R

F. Biometrics Identification Scenarios

Table X (right column) summarises the subject-wise identification rate obtained by the minimum cosine distance classifier with the PSD and AR features from $L_{seg} = 60$ s segments in Setup-R. Previously, we considered a binary client-imposter classification problem (e.g. $M=2$ ) for each subject, each day, and each trial, however, the classification algorithm used in this study was the simple minimum cosine distance between the validation matrix and training matrix. For the prediction of $l$ th row of the validation matrix, $Y_{V_{(l,:)}}$ , the minimum distance between the training matrix, $Y_{T}$ , was found, e.g. the $n$ th row of the training matrix, and the same label was assigned to the $n$ th row of the training matrix as the prediction label for $l$ th row of the validation matrix, $Y_{V_{(l,:)}}$ . Notice that the minimum distance approach is applicable for biometrics identification problems, which is a one-to-many classification. The number of classes was $M = 15$ , which corresponds to the the number of subjects in the $S_{R}$ recordings, therefore the chance level was 1/15= 6.7%. The achieved identification rate was 67.8% with $L_{seg} = 60$ s segments in Setup-R, while the achieved Cohen’s kappa coefficient was $\kappa = 0.65$ (Substantial agreement) [35].

Figure 7 shows identification rate of both Setup-R and Setup-B, with different segment sizes $L_{seg} = 10, 20, 30, 60, 90$ s. In Setup-B, the identification rate with $L_{seg} = 10$ s was 67.7%, which was almost the same as the result with $L_{seg} = 60$ s in Setup-R. The highest identification rate, 87.2%, was achieved with $L_{seg} = 90$ s, where corresponding Kappa was $\kappa = 0.86$ (Almost Perfect agreement) in Setup-B.

Fig. 7.

The identification rate with different segment size in Setup-R and Setup-B. The error bars indicate the standard error.

Show All

A. Verification With Different Segment Sizes

Firstly, the classification results were compared for different segment lengths $L_{seg}$ , shown in Table VI. Within the same setup, i.e. Setup-R or Setup-B, the performance of HTER and AC increased with the segment length, although the results with $L_{seg} = 60$ s and $L_{seg} = 90$ s are almost the same. Longer segments allowed for more data epochs to be averaged over, hence the EEG noise inference for classification diminished and the inherent EEG characteristic were able to be captured by averaging. However, a longer segment length also implies a longer recording time, which is not ideal for feasible EEG biometrics. Compared to the results in Setup-R and Setup-B for the same segment length, both the HTERs and accuracy (ACs) of Setup-B were clearly better than those of Setup-R. In terms of client discrimination, the decrease in FRR was significant from Setup-R to Setup-B. In Setup-B, a larger number of client segments was correctly classified (see TP) than in Setup-R. In setup-R, only the result with $L_{seg} = 60$ s achieved TP > (FN + FP), which indicates that the likelihood of making a true positive verification is higher than making a false verification; in Setup-B, the shortest segment size, $L_{seg} = 10$ s, achieved TP > (FN + FP).

The difference between Setup-R and Setup-B was that the training matrices $Y_{T}$ in Setup-B included the trials which were recorded on the same day as the validation trial. In other words, the assigned validation data (trial) and the part of assigned training data (trials) were recorded within 5 – 10 minutes in the same environment. Therefore, the training matrix contains significantly similar EEG recordings to the validation matrix in Setup-B, and this leads to a higher classification performance than the classification in Setup-B.

With an increase in the segment size $L_{seg}$ , the number of segments per recording trial, $N$ , became smaller, especially for $N = 2$ with the segment size $L_{seg} = 90$ s; therefore, the training matrix only contains six ($2\times 3$ trials) examples of client data in Setup-R (c.f. ten examples in Setup-B), which might not be enough for training client data. Hence, the performance with $L_{seg} = 90$ s was slightly lower than that with $L_{seg} = 60$ s in Setup-R.

B. Verification With Different Classifiers

Table VIII shows the classification comparison among the minimum cosine distance methods, LDA and SVM. The SVM was used as a parametric classifier; firstly, the optimal hyper-parameters (see details in Table IV) were selected from 5-fold cross-validation within the training matrix, and then weight parameters based on these chosen hyper-parameters were obtained. Notice that we could tune the classifier in different ways, e.g. in order to minimise false acceptance or minimise false rejection. The optimal tuning in this study was performed so as to maximise class sensitivities, i.e. maximise the number of TP and TN elements, which resulted in minimum HTERs. In both Setup-R and Setup-B, the FARs by SVM were smaller than those achieved by both the minimum cosine distance and the LDA, because the tuning was performed for maximising TN elements. Since the number of imposter elements was fourteen times bigger than the number of clients in both Setup-R and Setup-B (i.e. chance level was 14/15), the SVM parameters were tuned for higher sensitivity to imposters. As a result, the FRR by SVM, which were related to client sensitivity given in Table IX, were higher than those achieved by both minimum cosine distance and LDA in Setup-R.

In Setup-B, as mentioned above, the training matrix contains the data from the same recording day, which are more similar EEG patterns than the data obtained from a different recording day. Therefore, the SVM model chose hyper-parameters and weight parameters from the training matrix, so as to better the validation data in Setup-B, which led to higher performance than by both the minimum cosine distance and LDA.

Notice that, as described before, tuning of the hyper-parameters was performed within the training matrix, then the so-obtained hyper-parameters were used for finding the optimal weight parameters within the training matrix. The same hyper-parameters and weight parameters were used for classifying the validation matrix. This setup is applicable for feasible EEG biometrics scenarios in the real-world.

C. Validation Including Non-Registered Imposters

In Table IX, the confusion matrices for the client matrix $Y_{VC}$ and imposter matrix $Y_{VI}$ from dataset $S_{R}$ and imposter matrix $Y_{VI_{N}}$ , are given, which were then used for a comparison between Setup-R and Setup-B. Compared to the results obtained within the same classifier (minimum cosine distance, LDA, SVM) in Setup-R and Setup-B, the true positive rate (TPR) of clients $Y_{VC}$ and the sensitivity of imposter $Y_{VI}$ from dataset $S_{R}$ in Setup-B were higher than those in Setup-R. As described before, in Setup-B, the two client trials from the same recording day as the validation trial were included into the training matrix, and therefore more segments were correctly classified. In contrast, the TPRs of imposter $Y_{VI_{N}}$ from dataset $S_{N}$ by both the LDA and SVM in Setup-B were almost the same to those in Setup-R; 89.9% and 88.7% for LDA, 96.2% and 96.3% for SVM. Compared to the TPR between two imposter data ($Y_{VI}$ and $Y_{VI_{N}}$ ), regardless of the classifiers, the TPR of $Y_{VI}$ were higher than those of $Y_{VI_{N}}$ . Since the imposter data from $S_{N}$ were not included in the training matrix, the more $S_{N}$ data were misclassified as ‘client’ than $S_{R}$ data misclassified as ‘client’.

However, in the real-world scenarios for biometrics, imposters are not always ‘registered’. The lower TPR for $Y_{VI_{N}}$ means that the application is inadequate for attack from non-registered subjects. One potential way to overcome the vulnerability of the minimum cosine distance classifier is by introducing threshold for classification. If the nearest distance is larger than the given distance parameter, the segment is excluded from the classification or is classified as imposter.

D. Client-Imposter Verification Results per Subject

For subject-wise classification, Table X summarises classification results obtained by the minimum cosine distance in Setup-R, for different training-validation scenarios. The results varied across subjects and for training-validation configurations between 91.1% to 100% of AC and between 0.0% to 35.8% of HTER.

The size of viscoelastic earpiece was the same for twenty subjects (both $S_{R}$ and $S_{N}$ subjects), therefore all the subjects were able to wear it comfortably. The upper bounds of the electrode impedance over three recordings per day of each participant are given in Table X (Impedance part). The highest performance was achieved by Subject 1, with maximum impedances of 9 $\text{k}\Omega$ and 10 $\text{k}\Omega$ for the 1st and 2nd recording, respectively. Even though the impedances for Subject 2 and Subject 5 were smaller than those for Subject 1, the corresponding performance was below average over fifteen subjects. Besides, the lowest performance was exhibited by Subject 8, for whom the impedances were smaller than 11 $\text{k}\Omega$ for Day1 and 11 $\text{k}\Omega$ for Day2. Figure 8 shows average PSDs for Subject 8 – observe that the PSDs for the EEG recorded on Day2 (blue) is slightly larger than those on Day1 (red).

Fig. 8.

Power spectral density for the in-ear EEG Ch1 (left) and the in-ear EEG Ch2 (right) of Subject 8. The thick lines correspond to the averaged periodogram obtained by the all recordings from the 1st day (red) and the 2nd day (blue), whereas the thin lines are the averaged periodogram obtained by a single trial.

Show All

E. Biometrics Identification

In terms of biometrics identification results, a one-to-many subject-to-subject classification problem, the average sensitivity over fifteen subjects, i.e. the identification rate, was 67.8% in Setup-R with $L_{seg} = 60$ s, as shown in Table X (right column). Figure 7 illustrates the identification rates of both Setup-R and Setup-B, with different segment sizes $L_{seg} = 10, 20, 30, 60, 90$ s. The identification rate increased with segment length, although the results with $L_{seg} = 60$ s and $L_{seg} = 90$ s are almost the same.

Notice that the performances with $L_{seg} = 10$ s in Setup-B and $L_{seg} = 60$ s in Setup-R were almost the same, 67.7% and 67.8%, respectively. The highest identification rate in Setup-B was 87.2% with $L_{seg} = 90$ s. Indeed, the training matrix for Setup-B included the trials which were recorded at the same day as the validation trial; therefore the performance was better than in Setup-R.

In a previous biometrics identification study, Maiorana et al. [22] analysed 19 channels of EEG during EC tasks in three different recording days, and achieved the rank-1 identification rate (R1IR) of 90.8% for a segment length 45 s. Notice that it is hard to compare the performance with our approach, because the number of channels was very different, as 19 scalp EEG channels covered the entire head vs our 2 in-ear EEG channels embedded on an earplug. Therefore, although our results were lower, the proof-of-concept in-ear biometrics emphasised the collectability aspect in fully wearable scenarios.

F. Alpha Attenuation in the Real-World Scenarios

One limitation of using the alpha band, is the sensitivity to drowsiness, a state where the alpha band power is naturally elevated. For illustration, Figure 9 shows the PSD obtained from a subject, calculated by Welch’s averaging periodogram method. The subject slept during one recording, then the subject was woken up and another recording started less than 10 minutes after the first recording. The PSD graphs in Figure 9 are overlapped except for the alpha band; the alpha power observed during the ‘sleepy’ recording trial was smaller than that at the ‘normal’ recording, thus demonstrating the alpha attenuation due to fatigue, sleepiness, and drowsiness. The alpha attenuation is well known in the research in sleep medicine [26], [36], where it is particularly used to monitor sleep onset.

Fig. 9.

Power spectral density for the in-ear EEG Ch1 (left), and the in-ear EEG Ch2 (right) of one subject. The thick lines correspond to the averaged periodogram obtained by the recordings from ‘sleepy’ trial (red) and ‘normal’ trial (blue). Observe that the alpha power attenuated during the ‘sleepy’ trial.

Show All