CN112836593B - An emotion recognition method and system integrating prior and automatic EEG features - Google Patents

Abstract

The invention discloses an emotion recognition method and system that integrates prior and automatic EEG features. The method first extracts the differential entropy features of different frequencies related to emotions from the original EEG signal, and then maps it into a differential entropy matrix. And concatenated into a space-frequency feature matrix as a priori emotion feature driven by mathematics; at the same time, the time-frequency domain information in the original EEG signal is automatically extracted through the scaling convolution layer as a data-driven automatic EEG emotion feature; and then the prior emotion The features are transformed and spliced with the automatic EEG emotional features, and then fused to extract their high-order semantic features, and finally sent to the soft maximum activation function to recognize and classify emotions. The invention provides an emotion recognition method that can combine prior knowledge and data drive, can simultaneously model time-space-frequency domain information, and improve the intelligence and universality of emotion recognition.

Description

An emotion recognition method and system integrating prior and automatic EEG features

technical field

The invention belongs to the technical field of biological information recognition, and in particular relates to an emotion recognition method and system that integrates prior and automatic EEG features.

Background technique

How to discover biologically meaningful knowledge and laws from brain activity data and make use of them is becoming a difficult and hot spot in the theoretical and practical research in the field of biological information recognition. EEG signal-based emotion recognition and its association research has become a hot topic in the fields of neural engineering and artificial intelligence. In recent years, the advancement of machine learning technology has provided available technical methods for the research of emotion recognition based on EEG signals. In various applications of machine learning, hand-designed features are still dominant, and it mainly relies on the prior knowledge of the designer. Traditional hand-crafted features make good use of prior knowledge, but also lose the original EEG signal. It is difficult to make full use of the various information provided by the data, and the design of manual features relies on strong assumptions, resulting in poor portability. Deep learning can automatically learn the representation of features from data, but the type of features extracted by this method is single and lacks prior knowledge.

Contents of the invention

Aiming at the defects and deficiencies in the prior art, the present invention provides an emotion recognition method and system that integrates prior and automatic EEG features, which can improve the lack of prior EEG emotion knowledge and strong Hypothetical and preserved task-related features are not robust to the problem.

To achieve the above object, the present invention takes the following technical solutions:

An emotion recognition method that integrates prior and automatic EEG features. This method first extracts the differential entropy features of different frequencies related to emotions from the original EEG signal, maps them to a differential entropy matrix, and then converts the differential entropy features of different frequency bands The entropy matrix is spliced into a space-frequency feature matrix as a priori emotional feature driven by mathematics; at the same time, the time-frequency domain information in the original EEG signal is automatically extracted through the scaling convolution layer as the automatic EEG emotional feature driven by data; The experimental emotional features are transformed and spliced with the data-driven automatic EEG emotional features, and then fused to extract their high-order semantic features, and finally sent to the soft maximum activation function to identify and classify emotions.

The present invention also includes following technical characteristics:

Specifically, the method includes the following steps:

Step 1, construct prior emotion features:

Extract the differential entropy features of different frequencies related to emotions from the original EEG signal, map it into a differential entropy matrix according to the electrode spatial position, and then stitch the differential entropy matrix of different frequency bands into a space-frequency feature matrix, which is used as a mathematical drive prior emotional characteristics;

Step 2, construct automatic EEG emotion features:

The original EEG signal is sent to an independent scaling convolution layer channel by channel, and the output results of all channels are spliced into a three-dimensional tensor as a data-driven automatic EEG emotional feature;

Step 3, feature fusion:

Perform feature transformation on the prior emotional features and automatic EEG emotional features obtained in step 1 and step 2 to obtain the mathematically driven prior emotional feature vector and the data-driven automatic EEG emotional feature vector respectively; the prior emotional feature vector In-depth fusion with EEG emotion feature vectors to extract high-order semantic features;

Step 4, classification recognition:

Send the high-order semantic features extracted in step 3 into the soft maximum activation function to identify and classify emotions.

Specifically, said step one includes the following steps:

Step 1.1, using Fourier transform to divide the original EEG signal into four frequency bands (4-7Hz), (8-15Hz), (16-32Hz) and (33-45Hz) channel by channel;

Step 1.2, calculate the power spectral density on the four frequency bands in step 1.1 channel by channel;

Step 1.3, performing logarithmic function nonlinear transformation to the power spectral density on the four frequency bands in step 1.2, to obtain its differential entropy feature;

In step 1.4, according to the definition information of the electrode spatial position of the EEG acquisition equipment, the differential entropy feature of each EEG channel on the same frequency band in step 1.3 is corresponding to the element with the closest Euclidean distance in a two-dimensional feature matrix, and there are no other EEG channels The corresponding two-dimensional feature matrix elements are uniformly set to zero; the differential entropy feature map of each EEG channel on the four frequency bands obtains the differential entropy feature matrix of the four frequency bands;

In step 1.5, the differential entropy feature matrices of the four frequency bands mapped in step 1.4 are spliced sequentially from left to right and from top to bottom to obtain prior emotion features.

Specifically, the second step includes the following steps:

In step 2.1, the original EEG signal is sent to an independent scaling convolution layer channel by channel, and each channel obtains a two-dimensional time-frequency map with time-frequency information;

In step 2.2, all the time-frequency graphs are stacked and stitched in the channel dimension, and a three-dimensional tensor is obtained as a data-driven automatic EEG emotional feature.

Specifically, the third step includes the following steps:

Step 3.1, through the independent three-layer convolutional neural network layer and one layer of maximum pooling layer, respectively perform feature transformation on the prior emotion feature and the data-driven automatic EEG emotion feature, and obtain the prior emotion feature vector and EEG emotion Feature vector;

Step 3.2, splicing the prior emotion feature vector and the EEG emotion feature vector together along the column direction to obtain the combined emotion feature vector with EEG space-time-frequency domain information;

In step 3.3, the emotional feature vector finally obtained in step 3.2 is input into a fully connected neural network layer to extract high-order semantic features that integrate prior and automatic EEG emotional features.

Specifically, the fourth step uses the linear neural network layer and the soft maximum transfer function to classify and identify the extracted high-order semantic features, and then identify the emotional state of the person.

An emotion recognition system that integrates prior and automatic EEG features, including:

A priori emotional feature building block, which is used to extract the differential entropy features of different frequencies related to emotion from the original EEG signal, map it into a differential entropy matrix according to the spatial position of the electrode, and then stitch the differential entropy matrix of different frequency bands into empty frequency feature matrix as a mathematically driven prior emotion feature;

The automatic EEG emotional feature building module is used to send the original EEG signal into an independent scaling convolution layer channel by channel, and the output results of all channels are spliced into a three-dimensional tensor as a data-driven automatic EEG emotional feature;

The feature fusion module is used to perform feature transformation on the prior emotional features and the automatic EEG emotional features to obtain the mathematically-driven prior emotional feature vectors and the data-driven automatic EEG emotional feature vectors; and combine the prior emotional feature vectors and EEG emotional feature vectors are deeply fused to extract high-order semantic features;

The classification identification module is used to send the obtained high-order semantic features into the soft maximum activation function to identify and classify emotions.

Specifically, in the priori emotion feature building block, the original EEG signal is firstly divided into (4-7Hz), (8-15Hz), (16-32Hz) and (33-45Hz) channel by channel using Fourier transform ) four frequency bands; then calculate the power spectral density on the four frequency bands channel by channel; carry out the logarithmic function nonlinear transformation to the power spectral density on the four frequency bands to obtain its differential entropy feature; according to the definition of the electrode space position of the EEG acquisition equipment Information, the differential entropy feature of each EEG channel on the same frequency band corresponds to the element with the closest Euclidean distance in a two-dimensional feature matrix, and the other two-dimensional feature matrix elements that have no EEG channel corresponding to it are uniformly set to zero; four. The differential entropy feature mapping of each EEG channel on a frequency band obtains the differential entropy feature matrix of four frequency bands; the mapped differential entropy feature matrix of the four frequency bands is sequentially spliced from left to right and from top to bottom Get the prior emotion features.

Specifically, in the automatic EEG emotional feature building module, the original EEG signal is sent to an independent scaling convolution layer channel by channel, and each channel obtains a two-dimensional time-frequency map with time-frequency information; In the channel dimension, all time-frequency diagrams are stacked and spliced to obtain a three-dimensional tensor as a data-driven automatic EEG emotional feature.

Specifically, in the feature fusion module, through an independent three-layer convolutional neural network layer and a layer of maximum pooling layer, the prior emotional features and the data-driven automatic EEG emotional features are respectively subjected to feature transformation to obtain a priori The emotional feature vector and the EEG emotional feature vector; the prior emotional feature vector and the EEG emotional feature vector are spliced together along the column direction to obtain the combined emotional feature vector with EEG space-time-frequency domain information; will finally get The emotional feature vector of the input to a fully connected neural network layer to extract high-order semantic features that fuse prior and automatic EEG emotional features;

The classification recognition module classifies and recognizes the extracted high-order semantic features through the linear neural network layer and the soft maximum transfer function, and then recognizes the emotional state of the person.

Compared with the prior art, the present invention has beneficial technical effects as follows:

(1) The EEG emotional space-frequency feature matrix constructed by fusing the spatial position information and frequency information of the EEG channel in the present invention, compared with the existing traditional manual feature extraction method, not only includes the space-frequency feature matrix in the original EEG signal Frequency domain information, as well as prior EEG emotion-related knowledge.

(II) The present invention provides an EEG emotion recognition method that can integrate mathematics and data-driven features by effectively integrating prior and data-driven EEG emotion recognition features, which can improve the EEG emotion recognition process. Problems with single feature types, lack of prior EEG emotion knowledge, strong assumptions, and retained task-related features are not robust.

(III) The EEG emotion recognition method includes information in the time-space-frequency domain in the original EEG signal, and realizes the complementarity of information.

(IV) The present invention solves the loss of characteristic information in the EEG emotion recognition process through the multi-input neural network, lack of prior EEG emotion knowledge and other regrets, and proposes a kind of emotion recognition that does not depend on strong assumptions and has strong portability Methods.

Description of drawings

Figure 1 is a schematic diagram of the construction process of the prior characteristic space-frequency matrix.

Fig. 2 is a schematic diagram of an EEG emotion recognition network that integrates prior and automatic EEG features according to the present invention.

FIG. 3 is a flow chart of the detailed steps of the EEG emotion recognition method of the present invention.

Detailed ways

Specific embodiments of the present invention will be described in detail below. It should be understood that the specific embodiments described here are only used to illustrate and explain the present invention, and are not intended to limit the present invention.

As shown in Figure 1, Figure 2 and Figure 3, the present invention provides an emotion recognition method and system that integrates prior and automatic EEG features. The method first extracts the difference of different frequencies related to emotion from the original EEG signal The entropy feature is mapped to a differential entropy matrix, and then the differential entropy matrix of different frequency bands is spliced into a space-frequency feature matrix as a priori emotional feature driven by mathematics; at the same time, the time-frequency in the original EEG signal is automatically extracted by scaling the convolutional layer The domain information is used as the data-driven automatic EEG emotional features; then, the mathematically-driven prior emotional features and the data-driven automatic EEG emotional features are transformed and spliced, and then fused to extract their high-order semantic features, and finally sent to the The softmax activation function recognizes and classifies emotions.

In the present invention, a channel refers to each conductive electrode for collecting EEG signals. Generally, multiple conductive electrodes (ie, channels) are used to collect EEG signals for collecting EEG signals.

The channel dimension refers to the automatic EEG emotional feature tensor, which has three dimensions in total, and all the channel numbers constitute a dimension in the three-dimensional tensor.

The softmax activation function is the softmax activation function or the normalized exponential function. The softmax activation function can convert the prediction results from negative infinity to positive infinity into non-negative numbers, and the overall sum is 1.

The linear neural network layer is a neural network layer composed of a series of linear neurons, and the output can take any value.

The method comprises the steps of:

Step 1, construct prior emotion features:

First, the differential entropy features of different frequencies related to emotions are extracted from the original EEG signal, and they are mapped into a differential entropy matrix according to the electrode spatial position, and then the differential entropy matrix of different frequency bands is spliced into a space-frequency feature matrix, which is used as a mathematical Driven a priori emotional characteristics;

Step 2, construct automatic EEG emotion features:

The original EEG signal is sent to an independent scaling convolution layer channel by channel, and the output results of all channels are spliced into a three-dimensional tensor as a data-driven automatic EEG emotional feature;

Step 3, feature fusion:

Perform feature transformation on the prior emotional features and automatic EEG emotional features obtained in step 1 and step 2 to obtain the mathematically driven prior emotional feature vector and the data-driven EEG emotional feature vector respectively; the prior emotional feature vector and EEG emotional feature vectors are deeply fused to extract high-order semantic features;

Step 4, classification recognition:

Send the high-order semantic features extracted in step 3 into the soft maximum activation function to identify and classify emotions.

In this embodiment, step 1 constructs prior emotion features, as shown in Figure 1, and its specific steps are as follows:

Step 1.1, using Fourier transform to divide the original EEG signal into four frequency bands (4-7Hz), (8-15Hz), (16-32Hz) and (33-45Hz) channel by channel;

Step 1.2, calculate the power spectral density on the four frequency bands in step 1.1 channel by channel;

Step 1.3, performing logarithmic function nonlinear transformation to the power spectral density on the four frequency bands in step 1.2, to obtain its differential entropy feature;

In step 1.4, according to the definition information of the electrode spatial position of the EEG acquisition equipment, the differential entropy feature of each EEG channel on the same frequency band in step 1.3 is corresponding to the element with the closest Euclidean distance in a two-dimensional feature matrix, and there are no other EEG channels The corresponding two-dimensional feature matrix elements are uniformly set to zero; the differential entropy feature map of each EEG channel on the four frequency bands obtains the differential entropy feature matrix of the four frequency bands;

Step 1.5, the differential entropy feature matrix of the four frequency bands mapped in step 1.4 is spliced successively from left to right and from top to bottom to obtain the prior emotion feature (i.e. the space-frequency feature matrix in Figure 1);

Step 2 Construct automatic EEG emotion features, the specific steps are as follows:

In step 2.1, the original EEG signal is sent to an independent scaling convolution layer channel by channel, and each channel obtains a two-dimensional time-frequency map with time-frequency information;

In step 2.2, all the time-frequency graphs are stacked and spliced in the channel dimension to obtain a three-dimensional tensor as a data-driven automatic EEG emotional feature (automatic EEG time-frequency feature tensor).

Step 3 feature fusion, as shown in Figure 2, the specific steps of the fusion are:

Step 3.1, through the independent three-layer convolutional neural network layer and one layer of maximum pooling layer, respectively perform feature transformation on the prior emotion feature and the data-driven automatic EEG emotion feature, and obtain the prior emotion feature vector and EEG emotion Feature vector;

Step 3.2, splicing the prior emotion feature vector and the EEG emotion feature vector together along the column direction to obtain the combined emotion feature vector with EEG space-time-frequency domain information;

In step 3.3, the emotional feature vector finally obtained in step 3.2 is input into a fully connected neural network layer to extract high-order semantic features that integrate prior and automatic EEG emotional features.

In the fourth step, the emotions are classified and recognized, and the extracted high-order semantic features are classified and recognized through the linear neural network layer and the soft maximum transfer function, and then the emotional state of the person is recognized. More specifically, the extracted high-order semantic features are sent to the linear neural network layer (there are many neurons in this layer, and the activation function used by each neuron is a softmax function), and the prediction probability is high through the softmax activation function. The emotion category is the result of the final emotion recognition output (that is, if the positive probability is greater than the negative probability, the output is positive, and if the negative probability is greater than the positive probability, the output is negative).

An emotion recognition system that integrates prior and automatic EEG features, including:

A priori emotional feature building block, which is used to extract the differential entropy features of different frequencies related to emotion from the original EEG signal, map it into a differential entropy matrix according to the spatial position of the electrode, and then stitch the differential entropy matrix of different frequency bands into empty frequency feature matrix as a mathematically driven prior emotion feature;

The automatic EEG emotional feature building module is used to send the original EEG signal into an independent scaling convolution layer channel by channel, and the output results of all channels are spliced into a three-dimensional tensor as a data-driven automatic EEG emotional feature;

The feature fusion module is used to perform feature transformation on the prior emotional features and the automatic EEG emotional features to obtain the mathematically-driven prior emotional feature vectors and the data-driven automatic EEG emotional feature vectors; and combine the prior emotional feature vectors and EEG emotional feature vectors are deeply fused to extract high-order semantic features;

The classification identification module is used to send the obtained high-order semantic features into the soft maximum activation function to identify and classify emotions.

Specifically, in the priori emotion feature building block, the original EEG signal is firstly divided into (4-7Hz), (8-15Hz), (16-32Hz) and (33-45Hz) channel by channel using Fourier transform ) four frequency bands; then calculate the power spectral density on the four frequency bands channel by channel; carry out the logarithmic function nonlinear transformation to the power spectral density on the four frequency bands to obtain its differential entropy feature; according to the definition of the electrode space position of the EEG acquisition equipment Information, the differential entropy feature of each EEG channel on the same frequency band corresponds to the element with the closest Euclidean distance in a two-dimensional feature matrix, and the other two-dimensional feature matrix elements that have no EEG channel corresponding to it are uniformly set to zero; four. The differential entropy feature mapping of each EEG channel on a frequency band obtains the differential entropy feature matrix of four frequency bands; the mapped differential entropy feature matrix of the four frequency bands is sequentially spliced from left to right and from top to bottom Get the prior emotion features.

Specifically, in the automatic EEG emotional feature building module, the original EEG signal is sent to an independent scaling convolution layer channel by channel, and each channel obtains a two-dimensional time-frequency map with time-frequency information; In the channel dimension, all time-frequency diagrams are stacked and spliced to obtain a three-dimensional tensor as a data-driven automatic EEG emotional feature.

Specifically, in the feature fusion module, through an independent three-layer convolutional neural network layer and a layer of maximum pooling layer, the prior emotional features and the data-driven automatic EEG emotional features are respectively subjected to feature transformation to obtain a priori The emotional feature vector and the EEG emotional feature vector; the prior emotional feature vector and the EEG emotional feature vector are spliced together along the column direction to obtain the combined emotional feature vector with EEG space-time-frequency domain information; will finally get The emotional feature vector of the input to a fully connected neural network layer to extract high-order semantic features that fuse prior and automatic EEG emotional features;

The classification recognition module classifies and recognizes the extracted high-order semantic features through the linear neural network layer and the soft maximum transfer function, and then recognizes the emotional state of the person.

Example 1

In this embodiment, the DEAP EEG signal emotion data set is taken as an example for verification, and a section of 32-channel original EEG signal is input, which specifically includes the following steps:

Step 1, construct prior emotion features:

First, extract the differential entropy features of different frequencies related to emotions from the original EEG signals of 32 channels, map them into a 9×9 differential entropy matrix according to the spatial position of the electrodes, and then stitch the differential entropy matrices of different frequency bands into 18 ×18 space-frequency characteristic matrix;

Step 2, construct automatic EEG emotion features:

The 32-channel original EEG signal is sent to 32 independent scaling convolution layers channel by channel, and then the output results of all channels are spliced into a three-dimensional tensor as a data-driven automatic EEG emotional feature;

Step 3, feature fusion:

Perform feature transformation on the prior emotional features and automatic EEG emotional features obtained in step 1 and step 2 to obtain the mathematically driven prior emotional feature vector and the data-driven EEG emotional feature vector respectively; the prior emotional feature vector and EEG emotional feature vectors are deeply fused to extract high-order semantic features;

Step 4, classification recognition:

Send the high-order semantic features extracted in step 3 into the soft maximum activation function to identify and classify emotions.

The final experimental results on the DEAP EEG signal emotion data set are shown in Table 1. The emotion recognition results of the emotion recognition method in the three dimensions of arousal, valence and dominance by simultaneously integrating prior and automatic EEG features are as follows: 71.33%, 71.25% and 71.10%, compared with only using prior emotion features or automatic EEG emotion features, the effect is better.

Table 1 Emotion recognition results of fusion of prior and automatic EEG features

Claims (5)

1. an emotion recognition method of fusion prior and automatic EEG feature, it is characterized in that, the method first extracts the differential entropy feature of the different frequencies relevant to emotion from the original EEG signal, and maps it to a differential entropy matrix, Then, the differential entropy matrix of different frequency bands is spliced into a space-frequency feature matrix as a priori emotional feature driven by mathematics; at the same time, the time-frequency domain information in the original EEG signal is automatically extracted through the scaling convolution layer as a data-driven automatic EEG emotional feature ;Mathematically-driven prior emotional features and data-driven automatic EEG emotional features are transformed and spliced, and then fused to extract their high-order semantic features, and finally sent to the soft maximum activation function to identify and classify emotions; The method includes the following steps: Step 1, construct prior emotion features: Extract the differential entropy features of different frequencies related to emotions from the original EEG signal, map it into a differential entropy matrix according to the electrode spatial position, and then stitch the differential entropy matrix of different frequency bands into a space-frequency feature matrix, which is used as a mathematical drive prior emotional characteristics; Step 2, construct automatic EEG emotion features: The original EEG signal is sent to an independent scaling convolution layer channel by channel, and the output results of all channels are spliced into a three-dimensional tensor as a data-driven automatic EEG emotional feature; Step 3, feature fusion: Perform feature transformation on the prior emotional features and automatic EEG emotional features obtained in step 1 and step 2 to obtain the mathematically driven prior emotional feature vector and the data-driven automatic EEG emotional feature vector respectively; the prior emotional feature vector In-depth fusion with EEG emotion feature vectors to extract high-order semantic features; Step 4, classification recognition: Send the high-order semantic features extracted in step 3 into the soft maximum activation function to identify and classify emotions; Described step one comprises the steps: Step 1.1, using Fourier transform to divide the original EEG signal into four frequency bands (4-7Hz), (8-15Hz), (16-32Hz) and (33-45Hz) channel by channel; Step 1.2, calculate the power spectral density on the four frequency bands in step 1.1 channel by channel; Step 1.3, performing logarithmic function nonlinear transformation to the power spectral density on the four frequency bands in step 1.2, to obtain its differential entropy feature; In step 1.4, according to the definition information of the electrode spatial position of the EEG acquisition equipment, the differential entropy feature of each EEG channel on the same frequency band in step 1.3 is corresponding to the element with the closest Euclidean distance in a two-dimensional feature matrix, and there are no other EEG channels The corresponding two-dimensional feature matrix elements are uniformly set to zero; the differential entropy feature map of each EEG channel on the four frequency bands obtains the differential entropy feature matrix of the four frequency bands; In step 1.5, the differential entropy feature matrix of the four frequency bands mapped in step 1.4 is spliced sequentially from left to right and from top to bottom to obtain prior emotional features; Described step 2 comprises the following steps: In step 2.1, the original EEG signal is sent to an independent scaling convolution layer channel by channel, and each channel obtains a two-dimensional time-frequency map with time-frequency information; In step 2.2, all time-frequency graphs are stacked and spliced in the channel dimension to obtain a three-dimensional tensor as a data-driven automatic EEG emotion feature; Described step three comprises the following steps: Step 3.1, through the independent three-layer convolutional neural network layer and one layer of maximum pooling layer, respectively perform feature transformation on the prior emotion feature and the data-driven automatic EEG emotion feature, and obtain the prior emotion feature vector and EEG emotion Feature vector; Step 3.2, splicing the prior emotion feature vector and the EEG emotion feature vector along the column direction to obtain the combined emotion feature vector with EEG space-time-frequency domain information; Step 3.3, input the emotional feature vector finally obtained in step 3.2 into a fully connected neural network layer, and extract high-order semantic features that fuse prior and automatic EEG emotional features; The fourth step classifies and recognizes the extracted high-order semantic features through the linear neural network layer and the soft maximum transfer function, and then recognizes the emotional state of the person. 2. An emotion recognition system integrating prior and automatic EEG features, characterized in that it comprises: A priori emotional feature building block, which is used to extract the differential entropy features of different frequencies related to emotion from the original EEG signal, map it into a differential entropy matrix according to the spatial position of the electrode, and then stitch the differential entropy matrix of different frequency bands into empty frequency feature matrix as a mathematically driven prior emotion feature; The automatic EEG emotional feature building module is used to send the original EEG signal into an independent scaling convolution layer channel by channel, and the output results of all channels are spliced into a three-dimensional tensor as a data-driven automatic EEG emotional feature; The feature fusion module is used to perform feature transformation on the prior emotional features and the automatic EEG emotional features to obtain the mathematically-driven prior emotional feature vectors and the data-driven automatic EEG emotional feature vectors; and combine the prior emotional feature vectors and EEG emotional feature vectors are deeply fused to extract high-order semantic features; The classification identification module is used to send the obtained high-order semantic features into the soft maximum activation function to identify and classify emotions. 3. the emotion recognition system of fusion priori as claimed in claim 2 and automatic EEG feature, it is characterized in that, in described priori emotion feature building block, at first utilize Fourier transform to divide original EEG signal channel by channel For (4-7Hz), (8-15Hz), (16-32Hz) and (33-45Hz) four frequency bands; then calculate the power spectral density of the four frequency bands channel by channel; for the power spectral density of the four frequency bands Perform the nonlinear transformation of the logarithmic function to obtain its differential entropy feature; according to the definition information of the electrode spatial position of the EEG acquisition equipment, the differential entropy feature of each EEG channel on the same frequency band and the element with the closest Euclidean distance in a two-dimensional feature matrix Correspondingly, the remaining elements of the two-dimensional feature matrix that do not correspond to the EEG channel are uniformly set to zero; the differential entropy feature mapping of each EEG channel on the four frequency bands obtains the differential entropy feature matrix of the four frequency bands; the mapped The differential entropy feature matrix of the four frequency bands is spliced sequentially from left to right and top to bottom to obtain the prior emotion features. 4. the emotion recognition system of fusion prior art and automatic EEG feature as claimed in claim 3, is characterized in that, in described automatic EEG emotion feature construction module, original EEG signal is sent into independent scaling by channel In the convolutional layer, each channel obtains a two-dimensional time-frequency graph with time-frequency information; in the channel dimension, all time-frequency graphs are stacked and spliced to obtain a three-dimensional tensor as a data-driven automatic EEG emotional traits. 5. the emotion recognition system of fusion prior and automatic EEG feature as claimed in claim 4, is characterized in that, in described feature fusion module, through independent three-layer convolutional neural network layer and one deck maximum pooling layer , perform feature transformation on the prior emotional features and data-driven automatic EEG emotional features, respectively, to obtain the prior emotional feature vector and the EEG emotional feature vector; the prior emotional feature vector and the EEG emotional feature vector along the direction of the column Spliced together to obtain the combined emotional feature vector with EEG space-time-frequency domain information; input the final emotional feature vector into a fully connected neural network layer to extract high-level semantics that fuse prior and automatic EEG emotional features feature; The classification recognition module classifies and recognizes the extracted high-order semantic features through the linear neural network layer and the soft maximum transfer function, and then recognizes the emotional state of the person.

Images