CN108881985A - Program points-scoring system based on brain electricity Emotion identification - Google Patents

Abstract

The invention discloses a program scoring system based on EEG emotion recognition, which comprises an EEG signal acquisition module, an EEG signal preprocessing module, an EEG signal analysis module, an emotion recognition analysis module, a program effect analysis module and a program The scoring module first uses the EEG signal acquisition module to collect the audience to watch different segments and watch the EEG signals for a period of time, and then performs filtering operations through the EEG signal preprocessing module, and then realizes the feature extraction of the EEG signals through wavelet packet decomposition; and then The emotion recognition and analysis module conducts emotion recognition on the extracted features through the backpropagation neural network; and the program effect analysis module is used to monitor the duration of the emotional impact of the audience; finally, the program scoring module performs scoring operations on the corresponding programs; The invention analyzes and calculates the audience's actual influence of the program from the audience's EEG signal, and then scores, which can eliminate external interference and reflect the most authentic effect of the program.

Description

Program Scoring System Based on EEG Emotion Recognition

technical field

The invention belongs to the technical field of emotion recognition, and in particular relates to a program scoring system based on EEG emotion recognition.

Background technique

At present, there are not many scoring systems that can be used for scoring film and television programs, and they are relatively single. The star rating code of JS film and television websites is one of the most popular scoring methods for major film and television websites at present. You can score movies or videos by sliding the mouse over the stars. But in essence, it is still just based on the audience's feelings to give points casually, and other scoring systems are the same, but it is unscientific to rate a work only based on the subjective consciousness of the audience, because people's subjective consciousness will be different. Affected by internal prejudices, external comments and other factors, they may even deliberately belittle or inflate the works regardless of the content of the program. Therefore, the ratings do not seem to be a true reflection of the wishes of netizens. Emotion recognition technology has been widely used in various fields such as medical care, smart home appliances, and security systems. Emotion recognition originally refers to the individual's recognition of other people's emotions. Now it mostly refers to AI's automatic identification of the individual's emotional state by obtaining the individual's physiological or non-physiological signals, which is an important part of emotional computing. The content of emotion recognition research includes facial expression, voice, heart rate, behavior, text and physiological signal recognition, etc., and the user's emotional state can be judged through the above content, but the above-mentioned judgment results of facial expression, voice, heart rate, behavior, etc. may be different. It is changed by the influence of the environment, that is, the accuracy of judgment is insufficient.

Contents of the invention

The main purpose of the present invention is to provide a program scoring system based on EEG emotion recognition, by analyzing the effect degree of the program on the emotional impact of the audience, the program is scored on the basis of this; many disturbing factors and unsatisfactory scoring can be avoided Fairness, maximizing the evaluation of the real value of the program, so as to provide more meaningful reference for subsequent viewing choices of viewers. The specific technical solutions are as follows:

A program scoring system based on EEG emotion recognition, which scores programs by analyzing the degree of effect of the program on the emotional impact of the audience. The program scoring system includes a sequentially connected EEG signal acquisition module, EEG signal preprocessing module, EEG signal analysis module, emotion recognition analysis module, program effect analysis module and program scoring module, in which:

The EEG signal collection module is used to collect the EEG signals of viewers watching different clips and watching for a period of time;

An EEG signal preprocessing module, configured to filter the collected EEG signals;

EEG signal analysis module, used for performing feature extraction operation on the filtered EEG signal;

Emotion recognition and analysis module, used for performing emotion recognition on the extracted features through the backpropagation neural network, to judge whether the audience produces the expected emotional response of the program;

The program effect analysis module is used to monitor the audience's EEG signals corresponding to the segment after watching the corresponding segment and for a period of time after viewing, and analyze the duration of the emotional impact received by the audience that is positively or negatively related to the corresponding segment;

The program scoring module performs scoring operations on corresponding programs based on the analysis results of the program effect analysis module.

As preferably, the EEG signal acquisition module adopts Emotiv EPOC+ equipment to collect the EEG signals, and the Emotiv EPOC+ equipment is provided with 14 EEG signal sensors; and the Emotiv EPOC+ equipment is connected to a computer, and the computer The software on the computer displays the EEG signal data collected by the Emotiv EPOC+ device in real time, and the 14 EEG signal transmission channels AF3, F7, F3, FC5, T7, P7, O1 are set on the Emotiv EPOC+ device and the connected computer , O2, P8, T8, FC6, F4, F8, AF42 for data interaction.

As preferably, the EEG signal preprocessing module realizes the preprocessing function by the EEGLAB tool;

The program scoring module realizes the scoring operation by establishing a program scoring model.

Preferably, the feature extraction operation is realized by wavelet packet decomposition.

Preferably, the EEG signals include Delta (0-4Hz), Theta (4-8Hz), Alpha (8-15Hz), Beta (15-30Hz), and Gamma (30-60Hz) five main frequency bands; The wavelet packet decomposition is provided with four layers of wavelet packet tree nodes (1, 1), (2, 1), (3, 1), (4, 0) and (4, 1) corresponding to the five main frequency bands.

Preferably, the backpropagation neural network is a multi-objective feedforward backpropagation neural network, the emotion recognition analysis module builds a classifier based on the multi-objective feedforward backpropagation neural network, and the activation function of the classifier Using a unipolar Sigmoid function

Preferably, the program effect analysis module uses the formula for the duration of the program's expected emotional effect generated after the audience watches the corresponding segment calculate, Indicates the duration.

Preferably, the types of emotions generated by the audience for different programs include four categories: "joy, anger, sadness and fear".

Preferably, the program scoring module is provided with a program rating standard, and the corresponding program is scored through the standard; and the program effect analysis module is provided with an emotional impact index corresponding to the standard, through the The emotional impact index judges the duration of the emotional impact of a given program on the audience.

The present invention can reduce the interference factors and unfairness of program ratings, maximize the true value of programs, and provide more meaningful references for later viewing choices of viewers; compared with the prior art, the present invention has the advantages and disadvantages of The effect is:

(1) Use Emotiv EPOC+ equipment to collect the EEG signals of the audience, which is portable and easy to operate;

(2) Using the wavelet packet transform algorithm to analyze the results is based on the consideration that the EEG signal is a typical non-stationary signal;

(3) The scoring system not only takes into account the impact on the audience when they watch the program, but also considers the emotional impact of the program on the audience after watching the program for a period of time, making the system's scoring more realistic.

Description of drawings

Fig. 1 is the structural composition block diagram representation of the program scoring system based on EEG emotion recognition described in the embodiment of the present invention;

Fig. 2 is a schematic diagram of the scoring flow chart of the program scoring system based on EEG emotion recognition described in the embodiment of the present invention;

Fig. 3 is the EEG signal diagram schematic diagram of Emotiv EPOC+ equipment described in the present invention;

Fig. 4 is a schematic diagram of the filtered EEG signal of the present invention;

FIG. 5 is a schematic structural diagram of the backpropagation neural network of the present invention.

Detailed ways

In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention.

Referring to Fig. 1, in the embodiment of the present invention, a kind of program rating system based on EEG emotion recognition is provided, by analyzing the degree of effect of the program on audience's emotional impact, the program is scored, and the program rating system includes sequentially connected EEG Signal collection module, EEG signal preprocessing module, EEG signal analysis module, emotion recognition analysis module, program effect analysis module and program scoring module, among which, the EEG signal collection module is used to collect audiences watching different clips and watching for a period of time The EEG signal; the EEG signal preprocessing module is used to filter the collected EEG signal; the EEG signal analysis module is used to perform feature extraction on the filtered EEG signal; the emotion recognition analysis module is used to extract The characteristics of the backpropagation neural network are used for emotional recognition to determine whether the audience has the expected emotional response to the program; the program effect analysis module is used to monitor the audience's EEG signals when watching the corresponding segment and a period of time after watching it, and analyze the audience's reception The duration of the emotional impact that is positively or negatively related to the corresponding segment is obtained; the program scoring module performs scoring operations on the corresponding program based on the analysis results of the program effect analysis module; in conjunction with Fig. 2, in the embodiment of the present invention, the program scoring module process In addition, by adding the test data of many classic works, the program can be scored more accurately and the score will be more scientific.

The scoring process of the program scoring system based on EEG emotion recognition of the present invention will be specifically described below:

First, the EEG signal data is collected by the EEG signal acquisition module, specifically, the EEG signal acquisition module of the present invention adopts the Emotiv EPOC+ equipment to collect the EEG signals, and the Emotiv EPOC+ equipment is provided with 14 EEG signal sensors ; and the Emotiv EPOC+ device is connected to the computer, and the EEG signal data collected by the Emotiv EPOC+ device is displayed in real time through the software on the computer. The 14 EEG signal transmission channels AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, AF42 for data interaction; preferably, the Emotiv EPOC+ device is connected to the computer through the EMOTIV USB receiver, and the Emotiv can be viewed through the Pure·EEG software on the computer For the EEG signal collected by the EPOC+ device, refer to Figure 3 for details; of course, the present invention does not limit and fix the connection mode between the Emotiv EPOC+ device and the computer and the viewing software of the EEG signal, and can be selected according to the actual situation .

Then, the EEG signal is preprocessed by the EEG signal preprocessing module. Preferably, the present invention uses the EEGLAB tool to filter the EEG signal and decompose the wavelet packet to realize the feature extraction operation; wherein the filtering mainly includes the main steps of data Importing, electrode position importing, filtering, removing artifacts, superimposed average process, Figure 4 shows a signal diagram of an EEG signal after filtering operation; then the filtered EEG signal is decomposed by wavelet packets to perform feature Extract, the present invention preferably gets four-layer wavelet packet; Concrete, EEG signal contains Delta (0-4Hz), Theta (4-8Hz), Alpha (8-15Hz), Beta (15-30Hz) and Gamma (30 -60Hz) five main frequency bands, correspondingly, the four-layer wavelet packet decomposition sets the wavelet packet tree nodes (1, 1), (2, 1), (3, 1), (4, 0) and (4, 1) , while referring to Table 1, we can see that the nodes (1, 1) (2, 1) (3, 1) (4, 0) and (4, 1) respectively cover the five main frequency bands of EEG signals: Delta (0-4Hz ), Theta(4-8Hz), Alpha(8-15Hz), Beta(15-30Hz), and Gamma(30-60Hz); wavelet packet decomposition selects the optimal base time call cost function After that, the feature extracted from the wavelet packet coefficient of each selected node is (μ _x , σ _x , E _x ), and according to the formula and the formula Solve for μ _x , σ _x and E _x respectively, where _xi is the wavelet packet coefficient, μ _x is the expected value of the wavelet packet coefficient, σ _x is the standard deviation of the wavelet packet system, E _x is the Shannon entropy of the wavelet packet coefficient, and then form a Eigenvector: Since each electrode signal of the EEG has 5 nodes, each node has 3 features, and there are 14 electrodes in total, so each eigenvector has 14×5×3=210 features.

Table I

Then, carry out emotional recognition to the extracted features by the backpropagation neural network, confirm whether the viewer produces the expected emotional response to the program watched, wherein the structural composition diagram of the backpropagation neural network can refer to Fig. 5, in the present invention, The backpropagation neural network is preferably a multi-objective feedforward and backpropagation neural network, and the emotion recognition analysis module builds a classifier based on the multi-objective feedforward and backpropagation neural network, and the activation function of the classifier adopts a unipolar Sigmoid function It can be seen that the process of identifying emotions through the classifier is as follows: firstly, a multi-objective forward propagation network is established. After establishment, the output of the kth neuron in the network output layer can be expressed as At the same time, in the process of backpropagation, error calculation is also required to correct the network, in which the error of the last layer of neural network is passed through the formula Calculation, where, ⊙ is the Hamilton product. From back to front, the error generated by each layer of neural network is calculated by the formula δ ^L =((ω ^l+1 ) ^T δ ^L+1 )⊙δ′(Z ^L ), and is calculated by the formula Calculate the weight of the gradient, by the formula Calculate the gradient of the bias, thereby improving the recognition rate of the backpropagation neural network; then the weights of each layer are updated by the gradient descent method, which can be obtained by the formula Indicates that, and for the number of neurons in the hidden layer, according to the empirical formula In the formula, n is the number of neurons in the hidden layer, n _i is the number of input neurons, n _o is the number of output neurons, a is a constant between 1 and 10; n _i is 210, n _o is The number of emotions to be recognized is 4, initially select n as the mean value of the empirical formula 21 to find the optimal value of the number of neurons: first define In the formula, N _c is the number of correctly classified samples, and N _t is the total number of samples. It can be seen from calculations and experiments that when a is set to 4.4 and the number of neurons in the hidden layer is 20, the CCrate is the highest and the effect is the best.

Then the program effect analysis module analyzes the expected emotional effect of the program produced after the audience watches the corresponding segment. The present invention specifically uses the duration of the emotional effect to to analyze, available through the formula Calculation; in the specific embodiment, the present invention divides the audience's emotion types produced by different programs into four categories of "joy, anger, sadness and fear", roughly divides a program into m segments, and finds n audiences to watch program, of which Indicates the duration corresponding to the emotions produced by the k-th viewer watching the i-th segment of the program, taking the average of the emotional duration of all audiences in the same segment as the final emotional impact duration of the segment, then Indicates the duration of the influence of the i-th segment on the audience's emotions.

Finally, the program scoring module scores the program according to the analysis results of the program effect analysis module. The specific process is as follows: first, a measurement standard is set, and the emotional impact of the specified program on the audience is judged by the measurement standard, so as to score the corresponding program; At the same time, 10 classic film and television clips are selected for each of the four different emotional types of "joy, anger, sadness and fear", and the duration of the emotional impact of each of the 10 classic clips of each emotional type on the audience is measured according to the process shown in Figure 2. For example, for For the emotional type of fear, add 10 classic clips of horror movies to the program rating model, and measure the duration of the fear emotion that the audience will have in each of the 10 clips, record it as T ₁ , T ₂ ,..., T ₁₀ , taking the minimum value among T ₁ , T ₂ ,..., T ₁₀ as the standard duration of the impact of the fear segment on the audience, the present invention uses Indicates the standard impact duration corresponding to the i-th fragment, then we can get Then define a variable h _i , h _i ∈ [0,1] represents a certain emotional impact index of the i-th segment of the program to be rated on the audience, and h _i is divided into a positively related emotional impact index and Negatively Correlated Sentiment Impact Index in If the analyzed emotion is positively related to the segment, then the positive impact index corresponding to this emotion is The positive correlation emotional impact value of this segment is the sum of positive correlation emotional impact indices, namely In the same way, it can be obtained that the negative correlation emotional impact value of this segment is For example, for a love segment, joy and sadness are positively related emotions, and the positively related emotional impact value of this segment is the sum of the impact indices of joy and sadness; anger and fear are negatively related emotions, and the negatively correlated emotional impact value of this segment is The emotional impact value is the sum of the impact indices of anger and fear.

To sum up, it can be seen that whether it is a positive correlation or a negative correlation emotional impact index, the greater the impact index value, the greater the impact of such emotions in the segment on the audience. Taking the coefficient of the total negative correlation emotional impact index as 0.2, the full score of the program is 10, and the final score is represented by P, then the final program scoring formula can be obtained as the combined formula And the program rating process in Fig. 2 can scientifically obtain the final rating of a certain program.

The present invention can reduce the interference factors and unfairness of program ratings, maximize the true value of programs, and provide more meaningful references for later viewing choices of viewers; compared with the prior art, the present invention has the advantages and disadvantages of The effect is:

(1) Use Emotiv EPOC+ equipment to collect the EEG signals of the audience, which is portable and easy to operate;

(2) Using the wavelet packet transform algorithm to analyze the results is based on the consideration that the EEG signal is a typical non-stationary signal;

(3) The scoring system not only takes into account the impact on the audience when they watch the program, but also considers the emotional impact of the program on the audience after watching the program for a period of time, making the system's scoring more realistic.

The above are only preferred embodiments of the present invention, but do not limit the scope of patents of the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, it can still understand the foregoing aspects The technical solutions described in the specific embodiments are modified, or some of the technical features are equivalently replaced. All equivalent structures made by utilizing the contents of the specification and drawings of the present invention and directly or indirectly used in other related technical fields are also within the protection scope of the patent of the present invention.

Claims (9)

1. based on the program points-scoring system of brain electricity Emotion identification, spectators' emotion is influenced by analysis program effect degree come pair Program scores, which is characterized in that the program points-scoring system includes sequentially connected electroencephalogramsignal signal acquisition module, EEG signals Preprocessing module, electroencephalogramsignal signal analyzing module, Emotion identification analysis module, program effect analysis module and program grading module, Wherein：

Electroencephalogramsignal signal acquisition module, for acquiring spectators' viewing different fragments and finishing watching the EEG signals of a period of time；

EEG signals preprocessing module, for the EEG signals of acquisition to be filtered；

Electroencephalogramsignal signal analyzing module, for making feature extraction operation to the filtered EEG signals；

Emotion identification analysis module carries out Emotion identification by reverse transmittance nerve network for the feature to extraction, sentences Whether disconnected spectators generate emotional reactions expected from program；

Program effect analysis module, for monitoring spectators' a period of time after viewing homologous segment and viewing corresponding segment brain electricity Signal, and analyze the duration lasting to the positive and negative relevant emotion influence of the homologous segment that spectators receive；

Program grading module, the analysis result based on the program effect analysis module make scoring operation to corresponding program.

2. the program points-scoring system according to claim 1 based on brain electricity Emotion identification, which is characterized in that the brain telecommunications Number acquisition module acquires the EEG signals using Emotiv EPOC+ equipment, is provided with 14 in the Emotiv EPOC+ equipment A EEG signals sensor；And the Emotiv EPOC+ equipment is connect with computer, is shown in real time by the software on computer Show the EEG signals data of Emotiv EPOC+ equipment acquisition, on the computer of the Emotiv EPOC+ equipment and connection Pass through 14 EEG signals transmission channels AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, AF42 of setting Carry out data interaction.

3. the program points-scoring system according to claim 1 based on brain electricity Emotion identification, which is characterized in that the brain telecommunications Number preprocessing module realizes preprocessing function by EEGLAB tool；

The program grading module realizes scoring operation by establishing program Rating Model.

4. the program points-scoring system according to claim 1 based on brain electricity Emotion identification, which is characterized in that the feature Extraction operation is realized by WAVELET PACKET DECOMPOSITION.

5. the program points-scoring system according to claim 4 based on brain electricity Emotion identification, which is characterized in that the brain telecommunications It number include Delta (0-4Hz), Theta (4-8Hz), Alpha (8-15Hz), Beta (15-30Hz) and Gamma (30- 60Hz) five primary bands；Corresponding five primary bands of the WAVELET PACKET DECOMPOSITION be provided with four layers of wavelet packet tree node (1, 1), (2,1), (3,1) (4,0) and (4,1).

6. the program points-scoring system according to claim 1 based on brain electricity Emotion identification, which is characterized in that the reversed biography Broadcasting neural network is multiple target baek-propagetion network, and the Emotion identification analysis module is feedovered based on the multiple target Reverse transmittance nerve network constructs classifier, and the activation primitive of the classifier uses unipolarity Sigmoid function

7. the program points-scoring system according to claim 1 based on brain electricity Emotion identification, which is characterized in that the program effect The time that fruit analysis module watches spectators the expected mood lasts of the program generated after the homologous segment passes through formulaIt calculates,Indicate duration.

8. the program points-scoring system according to claim 1 based on brain electricity Emotion identification, which is characterized in that spectators are to difference Categories of emotions caused by program includes " happy, indignation, sad and fear " four classes.

9. the program points-scoring system according to claim 1 based on brain electricity Emotion identification, which is characterized in that the program is commented Sub-module is provided with the measurement standard of program scoring, does scoring operation to corresponding program by the measurement standard；And it is described Program effect analysis module corresponds to the measurement standard and is provided with emotion influence index, is referred to by emotion influence index judgement Program is determined to the emotion influence duration of spectators.