KR102594818B1 - Eigenface Electro-Encephalographic signal classification method using covariance de-centering and EEG signal classification device using the same - Google Patents

Abstract

The Eigenface EEG signal classification method using covariance decentering extracts each EEG signal corresponding to a predetermined frequency band among the EEG signals of each channel, then applies data whitening to each channel direction to separate each EEG channel data into independent Converting to data, generating a plurality of EEG image data using each EEG channel data to which data whitening is applied for each channel direction, and extracting the Eigen phase from the plurality of EEG image data, a covariance matrix It is characterized in that it includes the step of extracting the Eigen face through a decentering process.

Description

Eigenface Electro-Encephalographic signal classification method using covariance de-centering and EEG signal classification device using the same}

The present invention relates to a method for classifying brain wave signals in a BCI (Brain Computer Interface), and more specifically, to an Eigenface brain wave signal classification method using covariance decentering and an brain wave signal classification device using the same.

Figure 1 is a conceptual diagram of a Brain-Computer Interface (BCI) system.

Referring to Figure 1, the brain is an organ that controls the human central nervous system and controls not only thoughts and memories, but also human body movements. In general, signals about body movement can be obtained through the muscles of end organs responsible for movement, such as the hand. At this time, the brain is also active to command movement, and the recording of the electrical signals that occur when the neurons in the brain are active is called an electroencephalogram (EEG) or electroencephalogram (EEG). The technology that uses these brain waves to use external devices, such as mechanical manipulation, without going through the peripheral nervous system, which is responsible for movement, is called Brain-Computer Interface (BCI).

Motor Imagery Classification is one of the major research areas for implementing BCI systems. It is the process of analyzing brain waves to find the body part imagined to move. Accordingly, various methods are being studied for feature extraction methods for classifying motion imagination EEG signals. Common Spatial Pattern (CSP) is a representative method for EEG signal classification, and is a method of applying a spatial filter that maximizes the variance difference between two different classes. Lotte et al. showed that it was possible to classify brainwave motion imagination using CSP, but there were cases where excellent performance was not shown for certain subjects. In addition, Yeonmo Yang et al. showed that the EEG analysis method using Eigenface can classify EEG by interpreting EEG as a single image. This method showed high accuracy centered on the channel, but showed low accuracy when analyzed centered on each attempt (trial).

Yeon-Mo Yang, Wansu Lim, ByeongMan Kim, “Eigenface analysis for brain signal classification: a novel algorithm”, International Journal of Telemedicine and Clinical Practices, Vol. 2, No. 2, 2017.

The present invention was proposed to solve the above technical problems, and provides an Eigenface EEG signal classification method and EEG signal classification device that improve classification performance using data whitening and covariance decentering.

According to an embodiment of the present invention to solve the above problem, after extracting the EEG signals corresponding to a predetermined frequency band among the EEG signals of each channel, data whitening is applied to each channel direction to obtain data from each EEG channel. Converting into independent data, generating a plurality of EEG images using each EEG channel data to which data whitening is applied for each channel direction, and extracting the Eigen phase from the plurality of EEG images, covariance A method of classifying Eigen face EEG signals using covariance decentering is provided, which includes extracting the Eigen face through a decentering process using a matrix.

In addition, the step of extracting the Eigen face in the present invention involves extracting the Eigen face after going through the Gram-Schmidt orthonormalization process and extracting some or all of the extracted Eigen face depending on the complexity of the data to be classified or the number of classes to be classified. It is characterized by being able to be used selectively.

In addition, in the present invention, the features of the EEG data are calculated using the Eigen pace, and the EEG signals are classified according to the results of performing machine learning with a convolutional neural network (CNN) using the features of the EEG data. It is characterized in that it further includes the step of: At this time, the feature of the EEG data calculated using Eigenface can be any value or image, and the step of classifying the EEG signal according to the type of feature selected by the user is not only CNN, but also linear discriminant analysis (Linear Discriminant Analysis). You may perform machine learning methods such as Linear Discriminant Analysis (LDA) or Support Vector Machine (SVM).

In addition, according to another embodiment of the present invention, an EEG signal receiver that receives EEG signals measured through a plurality of electrode channels in contact with the user's scalp, and a EEG signal corresponding to a predetermined frequency band among the EEG signals of each channel A preprocessor that filters each EEG signal, applies data whitening to each channel direction on the EEG signal output from the preprocessor, converts each EEG channel data into independent data, and data whitening is applied to each channel direction. EEG signal classification comprising a signal processing unit that generates a plurality of EEG images using each EEG channel data, and extracts the Eigen phase through a decentering process using a covariance matrix when extracting the Eigen phase from the plurality of EEG images. A device is provided.

In addition, the signal processing unit included in the present invention extracts the Eigen faces after going through the Gram-Schmidt orthonormalization process and selectively selects some or all of the extracted Eigen faces according to the complexity of the data to be classified or the number of classes to be classified. It is characterized by being usable.

In addition, the signal processing unit included in the present invention calculates features in the form of values and images using some or all of the Eigen face and a plurality of EEG image data and uses the features of the EEG data to create a convolutional neural network. It is characterized by classifying EEG signals according to the results of machine learning using methods such as (Convolutional Neural Network, CNN), Linear Disciminant Analysis (LDA), and Support Vector Machine (SVM). .

The EEG signal classification method and EEG signal classification device using the same according to an embodiment of the present invention can improve the classification performance of EEG signals by using data whitening and covariance decentering. In other words, the proposed method applies data whitening to the EEG signal and then extracts the features after going through the process of decentering the covariance matrix in the Eigen face feature extraction process. As a result, the Eigenface EEG analysis method using covariance decentering shows high accuracy for each trial and a stable recognition rate.

Figure 1 is a conceptual diagram of a BCI (Brain-Computer Interface) system.
Figure 2 is a configuration diagram of an EEG signal classification device according to an embodiment of the present invention.
Figure 3 is a diagram showing the process of the Eigenface EEG signal classification method using covariance decentering according to an embodiment of the present invention.
Figure 3a is another diagram showing the process of the Eigenface EEG signal classification method using covariance decentering according to an embodiment of the present invention.
Figure 4 is a diagram showing the whitening effect
Figure 4a is a diagram showing the whitening effect on left hand data
Figure 4b is a diagram showing the whitening effect on right hand data
Figure 4c is a covariance comparison diagram according to whitening for left-hand and right-hand data.
Figure 5 is a first diagram showing the change in the covariance matrix of the improved method.
Figure 5a is a second diagram showing the change in the covariance matrix of the improved method.
Figure 5b is a diagram showing the results showing the change in the covariance matrix when channel data whitening is not performed;
Figure 5c is a diagram showing the change in covariance matrix of the proposed method performing channel data whitening.
Figure 6 is an enlarged view of the change in the covariance matrix of Figure 5b
Figure 7 is a diagram showing the results of conventional Eigenface analysis for brain signal (EFA) and the proposed method.
Figure 8 shows the results showing the reconstructed image of the conventional Eigenface analysis for brain signal (EFA) and the image reconstructed by the proposed method.

Hereinafter, in order to explain the present invention in detail so that a person skilled in the art can easily implement the technical idea of the present invention, embodiments of the present invention will be described with reference to the accompanying drawings.

FIG. 2 is a configuration diagram of an EEG signal classification device according to an embodiment of the present invention, FIG. 3 is a diagram showing the process of the Eigenface EEG signal classification method using covariance decentering according to an embodiment of the present invention, and FIG. 3A is This is another diagram showing the process of the Eigen-Pace EEG signal classification method using covariance decentering according to an embodiment of the present invention.

The brain wave signal classification device according to this embodiment includes only a brief configuration to clearly explain the technical idea to be proposed.

Referring to FIGS. 2 to 3A, the EEG signal classification device includes an EEG signal receiving unit 110, a preprocessing unit 120, and a signal processing unit 130.

The main operations of the brain wave signal classification device configured as above are as follows.

The brain wave signal receiver 110 receives brain wave signals measured through a plurality of electrode channels in contact with the user's scalp.

The brain wave signal receiver 110 acquires the user's brain wave signal using brain wave signal measuring equipment. That is, the EEG signal measurement equipment measures EEG signals from a plurality of channels and provides them to the EEG signal receiver 110, and the EEG signal receiver 110 acquires EEG signals based on the EEG measured from the EEG signal measurement equipment. You can. It is desirable that the EEG signal measurement equipment be used to extract at least 128 EEG signals per second.

The preprocessor 120 filters the EEG signals corresponding to a predetermined frequency band among the EEG signals of each channel.

When the EEG signal is transmitted from the EEG signal receiver 110, the preprocessor 120 basically filters the EEG signal corresponding to a predetermined frequency band among the EEG signals.

In this embodiment, the preprocessor 120 samples the electroencephalographic signal (EEG) at 250Hz and applies a 50Hz notch filter and a bandpass filtered between 0.5Hz and 100Hz, which may vary depending on the embodiment. At this time, the preprocessor 120 may filter the data set using a 5th order Butterworth filter between 7Hz and 30Hz. This is because this frequency band contains Mu waves and beta waves, which are associated with motor imagination.

The signal processing unit 130 applies data whitening to each channel direction of the EEG signal output from the preprocessor to convert each EEG channel data into independent data, and converts each EEG channel data to which data whitening has been applied for each channel direction. When generating multiple brain wave images and extracting the Eigen face from the multiple brain wave images, the Eigen face is extracted through a decentering process using a covariance matrix.

The signal processing unit 130 constructs an EEG image using at least one of the EEG signal for each trial in which the EEG signal was measured, the EEG signal for each electrode channel used when measuring the EEG signal, and the EEG measurement time. At this time, trial refers to the number of times the brain wave signal is measured - the number of attempts to think of a specific movement. For example, if the number of trials is 6, the number of times the brain wave signal was measured is 6.

In this way, when the signal processing unit 130 constructs an EEG image using the EEG signal for each trial in which the EEG signal was measured, the EEG signal for each electrode channel used when measuring the EEG signal, and the EEG measurement time, the EEG image is generated as a three-dimensional image. It can be.

If the raw data of the 3D EEG signal is used as is or the 3D image is used as the Eigen face and the process of extracting features from the Eigen face is complicated, the electrode channel and EEG measurement used in the trial and EEG signal measurement A two-dimensional brain wave image can be created by fixing one element of time and constructing the brain wave image using the remaining two elements.

In the embodiment, the signal processing unit 130 fixes the trial during the trial, the electrode channel used when measuring the EEG signal, and the EEG measurement time, and uses the EEG signal for each electrode channel used when measuring the EEG signal according to the EEG measurement time to generate a two-dimensional EEG. Create an image. At this time, two-dimensional brain wave images are generated as many as the number of trials.

For example, when measuring brain wave signals in number O, trial #1, trial #2, … , receives trial #O, trial #1, trial #2, … , Trial #O is electrode channel #1, electrode channel #2, … , if it contains the EEG signal corresponding to electrode channel #L, Trial #1, Trial #2, etc. used when measuring the EEG signal. , fix trial #O, electrode channel #1, electrode channel #2, … according to the EEG measurement time. , A 2D EEG image is generated using the EEG signal for each electrode channel #L, and 2D EEG images will be generated as many as the number of trials.

In another embodiment, the signal processing unit 130 fixes the electrode channel used in the trial, the electrode channel used in measuring the EEG signal, and the electrode channel used in measuring the EEG signal during the time, and creates a two-dimensional EEG image using the EEG signal for each trial according to the EEG measurement time. creates . At this time, two-dimensional EEG images are generated as many as the number of electrode channels used when measuring EEG signals.

For example, when measuring brain wave signals in number O, trial #1, trial #2, … , receives trial #O, trial #1, trial #2, … , Trial #O is electrode channel #1, electrode channel #2, … according to the change in time. , If it contains EEG signals corresponding to electrode channel #L, electrode channels #1, #2, … used when measuring EEG signals. , #L is fixed, and trials according to the EEG measurement time: Trial #1, Trial #2, … , A 2D EEG image is generated using the EEG signal for each trial #O, and 2D EEG images will be generated as many as L, which is the number of electrode channels used.

In this way, the signal processing unit 130 fixes one element of the trial, the electrode channel used when measuring the EEG signal, and the EEG measurement time, and constructs the EEG image using the remaining two elements to generate a two-dimensional EEG image, thereby generating a two-dimensional EEG image. By using a dimensional EEG image as an Eigen face, features can be extracted from the Eigen face.

At this time, the signal processor 130 extracts a specific Eigen face from among the Eigen faces using Eigen face technology from a plurality of brain wave images. The signal processor 130 may extract a specific Eigen face whose Eigen value is greater than or equal to a threshold value.

Figure 3 is a diagram showing the process of the Eigenface EEG signal classification method using covariance decentering according to an embodiment of the present invention, and Figure 3a is a diagram showing the process of the Eigenface EEG signal classification method using covariance decentering according to an embodiment of the present invention. This is another drawing showing the process.

Referring to FIGS. 3 and 3A, the Eigenface EEG signal classification method using covariance decentering processed in the EEG signal classification device will be described as follows.

The brain wave signal classification device analyzes brain wave signals to proceed with the process of finding the body part imagined to move.

First, a step is performed to extract the EEG signals corresponding to a predetermined frequency band among the EEG signals of each channel.

Next, data whitening is applied to each channel direction to convert each EEG channel data into independent data. By performing data whitening, each EEG channel data has smaller correlation and unit variance.

Figure 4 is a diagram showing the whitening effect, Figure 4A is a diagram showing the whitening effect on left hand data, Figure 4B is a diagram showing the whitening effect on right hand data, and Figure 4C is a diagram showing the whitening effect on left hand and right hand data. This is a comparison of covariance according to whitening.

Referring to FIGS. 4 to 4C, whitening transformation is a preprocessing method that applies principal component analysis (PCA). Because whitened data is rotated, whitening is sometimes referred to as “Sphering.” Whitening allows data to have smaller correlations and unit variances with each other, and basically, the whitening calculation of data X is defined as follows.

Here, Λ is the eigenvalue and P is the eigenvector.

In this embodiment, data whitening can be calculated using the following equation for M time, N channel, and L trial.

Here, I _O is the brain wave signal.

As shown in FIGS. 4A and 4B, although the data is in the same direction (left-hand data or right-hand data), the parts where the correlation between channels is strong are different. In other words, if whitening between channel data is not applied, the covariance matrix between data appears very diverse. Therefore, the correlation between channel data can be erased through whitening, and the process of making the covariance to 0 and the variance to 1 is performed. In other words, through whitening, it is possible to achieve the effect of reducing dependency independently of each other so that each channel has no correlation.

Next, the step of generating a plurality of EEG images is performed using each EEG channel data to which data whitening has been applied for each channel direction.

Next, in extracting the Eigen face from a plurality of EEG images, the step of extracting the Eigen face through a decentering process using a covariance matrix is performed.

In other words, the Eigenface is constructed after covariance decentering, and centering is a method of reducing the correlation between variables.

The centering of the random variable X can be calculated as follows.

The proposed decentering centers backwards to magnify the correlation.

In the present invention, a covariance de-centering (CDC) technique was proposed to improve the performance of the EEG classification method using Eigen pace. The covariance calculated to obtain the Eigen face is a value that represents the correlation between data. The proposed covariance decentering reconstructs the covariance matrix by decentering it by adding the average of each column to the covariance matrix. The decentering method of the covariance matrix is as follows.

Here, C is input and C’ is output.

By decentering by adding the average of the covariance matrix, weight can be given to data with high correlation with other data, and the Eigenface method based on Principal Component Analysis (PCA) uses the covariance matrix obtained from image vectors. Because the obtained basis vectors are used in increasing order, features can be further emphasized.

At this time, when decentering occurs, the symmetry of the covariance matrix disappears and the eigenvectors become non-orthogonal to each other, so the Eigen phase is extracted after going through the Gram-Schmidt orthogonalization process.

Lastly, we proceed with the step of calculating the features of the EEG data using the Eigen pace and classifying the EEG signals according to the results of performing machine learning with a convolutional neural network (CNN) using the features of the EEG data. .

At this time, the Eigenface used to calculate features can selectively use the entire Eigenface or some Eigenfaces that well represent the characteristics of the data, depending on the complexity of the EEG data or the number of classes to be classified. Additionally, the calculated feature can be in the form of a value or an image, and in the case of a numeric form, it can be a specific value or a coordinate in n-dimensional Euclidean Space. . Depending on the type of calculated features, in addition to convolutional neural networks, machine learning methods such as Linear Disciminant Analysis (LDA) and Support Vector Machine (SVM) can be used to classify EEG signals.

Referring again to FIG. 3A, the Eigenface EEG signal classification method using covariance decentering is,

Channel whitening of two-dimensional data with the trial fixed, obtaining the first Eigen face through a covariance process, and processing the Eigen face into three-dimensional data through covariance de-centering (CDC). a step of obtaining a second Eigen face, a step of obtaining a training coefficient by projecting the training EEG data onto the Eigen face, a step of obtaining a test coefficient by projecting the test EEG data onto the Eigen face, and obtaining an image with the training coefficient. It can be processed through a CNN training step through a CNN training step, a CNN testing step through images with training coefficients, and a step of classifying the EEG data evaluation results.

Figure 5 is a first diagram showing the change in the covariance matrix of the improved method.

Referring to Figure 5, the change in the covariance matrix during the process with simulated data of two left trials and two right trials under the conditions of time: 7, channel: 3, trial: 4 is shown.

The top is a covariance matrix obtained according to the existing method, the break is a covariance matrix changed using data that has gone through the whitening step, and the bottom is a covariance matrix of the break changed through covariance de-centering (CDC). represents.

Figure 5a is a second diagram showing the change in the covariance matrix of the improved method.

Referring to Figure 5a, the left side represents a feature generated through the existing method, and the right side represents a feature created through the proposed method. Red dots represent left data, blue dots represent right data. The blue dots are also two dots, but since the two data on the right are the same, they were created in duplicate in the same place. This data is randomly generated simulated data, and changes in the covariance matrix can be confirmed.

Figure 5b is a diagram showing the results of the change in the covariance matrix when channel data whitening is not performed, Figure 5c is a diagram showing the change in the covariance matrix of the proposed method in which channel data whitening is performed, and Figure 6 is a This is an enlarged view of the change in the covariance matrix in 5b.

Referring to FIGS. 5B to 6 , results showing changes in the covariance matrix are shown when channel data whitening is not performed and when channel data whitening is performed.

In other words, the result on the right represents the result of performing covariance de-centering (CDC) on the covariance matrix on the left.

Figure 6 is an enlarged view of the covariance matrix changed through performing the covariance de-centering (CDC) step. The x-axis and y-axis represent the number of trials, and the z-axis represents the calculated covariance value.

Changes due to covariance de-centering (CDC) occur throughout the covariance matrix, and visible changes can be confirmed in the covariance values within the red circle.

Figure 7 is a diagram showing the results of the conventional Eigenface analysis for brain signal (EFA) and the proposed method.

Referring to Figure 7, the distribution of training data and test data for machine learning is shown, and the proposed method can generate good features that can distinguish between “left hand and right hand thinking” EEG signals. You can check that.

In other words, when performing data whitening, which is the proposed method, and applying the Eigen-Pace EEG signal classification method using covariance decentering, it can be confirmed that better features than before can be generated in generating features for classifying EEG signals. there is.

Figure 8 shows the results showing the reconstructed image of the conventional Eigenface analysis for brain signal (EFA) and the image reconstructed by the proposed method.

Referring to Figure 8, the reconstructed image of the proposed CDC-EFA is clearly distinguished from the image of the conventional EFA.

In the image reconstructed using the proposed method, the bright and dark parts corresponding to the left and right classes are clear, whereas in the case of the conventional EFA, the unclear image is blurred or has less contrast.

The brain wave signal classification method and the brain wave signal classification device using the same according to an embodiment of the present invention can improve classification performance by using data whitening and covariance decentering. In other words, the proposed method applies data whitening to the EEG signal and then extracts the features after going through the process of decentering the covariance matrix in the Eigen face feature extraction process. As a result, the Eigenface EEG analysis method using covariance decentering shows high accuracy for each trial and a stable recognition rate.

As such, a person skilled in the art to which the present invention pertains will understand that the present invention can be implemented in other specific forms without changing its technical idea or essential features. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive. The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

110: EEG signal receiving unit
120: preprocessing unit
130: signal processing unit

Claims (6)

After the pre-processing unit extracts the EEG signals corresponding to a predetermined frequency band among the EEG signals of each channel, the signal processing unit applies data whitening through computational processing using the Eigen value and Eigen vector for each channel direction to each EEG channel. converting data into independent data;
The signal processing unit generating a plurality of EEG images using each EEG channel data to which data whitening has been applied through computational processing using Eigen values and Eigen vectors for each channel direction;
When the signal processor extracts the Eigen face from the plurality of EEG images, extracting the Eigen face through a decentering process using a covariance matrix; and
The signal processing unit calculates features in the form of numbers, images, or coordinates of the EEG data with coordinates in n-dimensional Euclidean space based on the Eigenface, and calculates features in the form of numbers, images, or coordinates of the EEG data using a convolutional neural network (CNN). ) and classifying the EEG signal according to the results of performing machine learning as an input;
The step of extracting the Eigen face is an Eigen face EEG signal classification method using covariance decentering, characterized in that the Eigen face is extracted after going through a Gram-Schmidt orthogonalization process.
delete delete An EEG signal receiving unit that receives EEG signals measured through a plurality of electrode channels in contact with the user's scalp;
a preprocessor that filters brain wave signals corresponding to a predetermined frequency band among the brain wave signals of each channel; and
Data whitening is applied to the EEG signal output from the preprocessor through computational processing using Eigen values and Eigen vectors for each channel direction to convert each EEG channel data into independent data, and Eigen values and Eigen values for each channel direction. A plurality of EEG images are generated using data from each EEG channel to which data whitening has been applied through computational processing using Eigen vectors, and when extracting the Eigen phase from the plurality of EEG images, the Eigen phase is decentered through a covariance matrix. Extract the face, calculate the features in the form of numbers, images, or coordinates of the EEG data with coordinates in n-dimensional Euclidean space based on the Eigen face, and calculate the features in the form of numbers, images, or coordinates of the EEG data through a convolutional neural network ( It includes a signal processing unit that classifies the brain wave signal according to the results of machine learning as an input to (CNN),
The signal processing unit is a brain wave signal classification device characterized in that the Eigen phase is extracted after going through the Gram-Schmidt orthonormalization process. delete delete