CN112347258B - Short text aspect level emotion classification method - Google Patents

Abstract

The invention discloses an aspect-level sentiment classification method for short texts. The steps include: 1. After segmenting the short text into words, generating word vectors, marking all aspect vectors and part-of-speech vectors from words in the short text; 2. Using an XLNET model to judge The aspect possessed by the short text, and splicing the word vector, part of speech vector, and aspect vector of each word in sequence; 3. Input the vector after splicing each word in step 2 into the sentiment classification Bilstm model, and each obtained Then input the hidden vector of each word into the Attention mechanism, and return the weight of the hidden vector of each word; 4. Use the hidden vector and weight corresponding to each word to do a weighted average, and the result enters the softmax neural network to get the corresponding emotion , with a larger probability value as the sentiment classification result. The invention can identify different emotions in different aspects of short texts, so as to complete the fine-grained emotion classification.

Description

An Aspect-Level Sentiment Classification Method for Short Texts

technical field

The invention belongs to the field of natural language processing in artificial intelligence, and specifically relates to a sentiment classification method of a Bilstm model incorporating a word vector, an aspect vector and a part-of-speech vector, and adding an Attention mechanism.

Background technique

With the development of e-commerce platforms, short text comments have increasingly become an important way for users to express their emotional opinions. Often, more than one aspect is involved in the short text, and even opposite emotional attitudes may be held for different aspects. At present, most sentiment classifications for short texts belong to coarse-grained classification, that is, only one sentiment classification is given for short texts, and the sentiments corresponding to different aspects cannot be identified in a fine-grained manner.

SUMMARY OF THE INVENTION

In order to solve the above-mentioned shortcomings of the prior art, the present invention proposes an aspect-level sentiment classification method for short texts, in order to recognize the aspects contained in the short texts first, and then identify the emotions corresponding to the aspects, so as to identify the short texts. Different sentiments in different aspects, and complete the fine-grained sentiment classification of short texts.

The present invention adopts the following technical scheme in order to achieve the above-mentioned purpose of the invention:

The feature of a short text aspect-level emotion classification method of the present invention is to carry out the following steps:

Step 1. Obtain all the short texts in the comment data and use them as corpus, and perform the preprocessing operations of classification, cleaning, and word segmentation on any short text in the corpus, and obtain the word vector set of the corresponding short text, which is denoted as t=( t ₁ ,t ₂ ,…,t _i ,…,t _k ), t _i represents the ith word, i∈[1,k], k represents the total number of words in the short text;

Step 2: Perform part-of-speech recognition on the word vector set t=(t ₁ , t ₂ ,...,t _i ,...,t _k ) to obtain a part-of-speech representation vector set p"'=(p"' ₁ ,p"' ₂ ,…,p″′ _i ,…,p″′ _k ), p″′ _i represents the part of speech corresponding to the i-th word t _i ;

Step 3. Preprocess all the short texts in the corpus according to Step 1, and after deleting the repeated words, a dictionary is obtained, and each word in the dictionary is numbered as the index position key of the corresponding word;

Step 4. Perform index processing on the word vector set t=(t ₁ ,t ₂ ,...,t _i ,...,t _k ) using the index position key of each word in the dictionary to obtain an index vector set s"' =(s″′ ₁ ,s″′ ₂ ,…,s″′ _i ,…,s″′ _k ); s″′ _i represents the index position corresponding to the i-th word t _i ;

Step 5. Denote the total number of words corresponding to the short text with the largest total number of words in the corpus as max; according to the total number of words max, the part-of-speech representation vector set p"' and the index vector set s"' are respectively marked with " 0" to complete, so that the total number of words in the part-of-speech representation vector set p"' and the index vector set s"' is equal to max; denote the completed part-of-speech representation vector set as p" and the completed index vector set Denoted as s";

Step 6, use the pre-training XLNET model to carry out the transformation processing of the n-dimensional word vector on the jth part of speech in the completed part of speech representation vector set p", and obtain the jth part of speech vector, denoted as p _j = (p _{j ,1} ,p _j,2 ,…,p _j,d ,…,p _j,n ), p _j,d represents the d-th dimension value in the j-th part-of-speech vector, d∈[1,n];

Step 7, utilize the pre-training XLNET model to carry out the transformation processing of the n-dimensional word vector to the jth word in the index vector set s" after the completion, obtain the jth word vector, denoted as s _j = (s _{j, 1} ,s _j,2 ,…,s _j,d ,…,s _j,n ), s _j,d represents the d-th dimension value in the j-th word vector;

Step 8. Select F aspect words from the corpus, input the f aspect word into the pre-training XLNET model, and obtain the word vector a _f =(a _f,1 ,a _f corresponding to the f aspect word _,2 ,…,a _f,d ,…,a _f,n ), a _f,d represents the d-th dimension value corresponding to the f-th aspect word;

Step 9. Arrange the elements in the part-of-speech representation vector set p"' and the index vector set s"' respectively in reverse order, and fill them with "0" according to the total number of words max; obtain the reversed-order part-of-speech representation vector set after completion. Denote it as p"" and the set of reversed index vectors after the complement is denoted as s"";

Step 10. Use the pre-trained XLNET model to perform the transformation processing of the n-dimensional word vector on the jth part of speech in the complemented reverse order part of speech representation vector set p"" to obtain the jth reverse order part of speech vector, denoted as _p'j =(p′ _j,1 ,p′ _j,2 ,…,p′ _j,d ,…,p′ _j,n ), p′ _j,d represents the d-th dimension value in the j-th reversed part-of-speech vector, d∈[1,n];

Step 11, use the pre-training XLNET model to carry out the transformation processing of the n-dimensional word vector on the jth word in the complemented reverse order index vector set s"" to obtain the jth reverse order word vector, denoted as s' _j = (s′ _j,1 ,s′ _j,2 ,…,s′ _j,d ,…,s′ _j,n ), s′ _j,d represents the d-th dimension value in the j-th reversed word vector;

Step 12. Form the forward input features of the Bilstm model:

The jth word vector s _j , the jth part of speech vector p _j and the word vectors corresponding to the F aspect words are sequentially spliced to obtain a feature vector fw_cell _j and use it as the input of the forward Bilstm model, the feature vector fw_cell _j The dimension of is 2×n+n×F;

Step 13. Form the reverse input feature of the Bilstm model:

Splicing the _jth reversed word vector s′ _j , the jth reversed part-of-speech vector p′j and the word vectors corresponding to the F aspect words in sequence to obtain the reversed feature vector bw_cell _j , which is used as another input of the reverse Bilstm model ;

Step 14. Establish F correction matrices and identify aspect information:

Let any f-th correction matrix be a matrix of (2+F) rows multiplied by 3 columns, where the element of the first column and the first row is "1", the element of the second column and the second row is "1", and the element of the first column is "1". The element in row f+2 of the third column is "1", and the rest of the elements are set to "0";

Inputting the word vector set t=(t ₁ , t ₂ ,...,t _i ,...,t _k ) into the pre-training XLNET model to obtain the aspect information contained in the word vector set t;

Obtain the corresponding aspect word through the aspect information contained in the word vector set t, so as to obtain the corresponding correction matrix according to the aspect word;

Multiply the feature vector fw_cell _j with the corresponding correction matrix of the word vector set t to obtain an input vector I _j ;

Multiply the inverse order feature vector bw_cell _j and the correction matrix corresponding to the word vector set t to obtain an inverse order input vector I′ _j ;

Step 15, using F aspect words to train F Bilstm models respectively, to obtain F Bilstm models after training;

The input vector I _j is respectively input into the Bilstm model after the training of the corresponding aspect word, thereby obtaining the output vector hf _j corresponding to the aspect information contained in the word vector set t;

Inputting the reverse-order input vectors I′ _j into the trained Bilstm models of the corresponding aspect words, respectively, to obtain the reverse-order output vector hb _j corresponding to the aspect information contained in the word vector set t;

Splicing the output vector hf _j and the reverse-order output vector hb _j to form an implicit vector h _j ;

Step 16: Splicing the implicit vector h _j , the j-th part-of-speech vector p _j and the word vectors corresponding to the F aspect words in sequence to obtain the implicit vector h′ _j of the Attention mechanism:

Step 17. Use the word vectors corresponding to the F aspect words to train the Attention mechanism network, and obtain the Attention mechanism network after training;

Input the latent vector h′ _j of the Attention mechanism into the trained Attention mechanism network to obtain the weight corresponding to the jth word vector s _j ;

Step 18. Predict the classification result:

Multiply each hidden vector h _j and its corresponding weight and sum up, get h ^* input into the fully connected layer, and get the scores corresponding to positive emotion and negative emotion; The corresponding scores are input into the softmax layer, the probability corresponding to the positive emotion and the negative emotion is obtained, and the emotion with a larger probability is used as the predicted classification result.

Compared with the prior art, the beneficial effects of the present invention are:

1. The present invention uses the XLNET model to identify the aspects contained in the short text in a fine-grained manner, and converts the short text into word vectors, part-of-speech vectors, and aspect vectors derived from the aspects identified by XLNET. In the Bilstm model with the attention mechanism, the emotion corresponding to the short text in this aspect is obtained, and the fine-grained emotion classification is completed by performing label processing first and then emotion classification processing, and the emotion classification of the short text also becomes more precise. At the same time, through the fusion of word vector, part-of-speech vector, and aspect vector, the present invention more comprehensively retains the information contained in the short text in the process of converting it into a vector, and improves the accuracy of aspect recognition and sentiment classification.

2. The core of natural language processing is to convert language into machine language for recognition and judgment, and the present invention considers both the content of the text and the part-of-speech of the text to recognize text through triple splicing of word vector, part-of-speech vector, and aspect vector. As a result, aspect vectors are also considered for fine-grained sentiment classification of short texts.

3. In the process of identifying aspect labels, the present invention proposes a correction matrix. By using the correction matrix, an input vector in which word vectors, part-of-speech vectors, and aspect vectors corresponding to various aspects related to short texts are generated in sequence is generated. Using the correction matrix, after automatically splicing aspect-related aspect vectors to word vectors and part-of-speech vectors, the aspect vectors that are not related to the aspect are automatically discarded, simplifying the input vector, while retaining all aspect-related content, and reducing the computational complexity.

Description of drawings

Fig. 1 is the overall flow chart of the method of the present invention.

Detailed ways

In the present embodiment, a sentiment classification method of the Bilstm model that fuses word vectors, aspect vectors, and part-of-speech vectors and adds the Attention mechanism is carried out according to the following steps:

Step 1. Obtain all the short texts in the comment data and use them as corpus, and perform preprocessing operations of classification, cleaning, and word segmentation on any short text in the corpus, and obtain the word vector set of the corresponding short text, which is denoted as t=(t ₁ ,t ₂ ,…,t _i ,…,t _k ), t _i represents the ith word, i∈[1,k], k represents the total number of words in the short text;

Step 2. Perform part-of-speech recognition on the word vector set t=(t ₁ ,t ₂ ,...,t _i ,...,t _k ), and obtain the part-of-speech representation vector set p″′=(p″′ ₁ ,p″′ ₂ , …,p″′ _i ,…,p″′ _k ), p″′ _i represents the part of speech corresponding to the i-th word t _i ;

Step 3. Preprocess all the short texts in the corpus according to Step 1, and after deleting the repeated words, a dictionary is obtained, and each word in the dictionary is numbered as the index position key of the corresponding word;

Step 4. Use the index position key of each word in the dictionary to index the word vector set t=(t ₁ ,t ₂ ,...,t _i ,...,t _k ) to obtain the index vector set s″′=(s″ ′ ₁ ,s″′ ₂ ,…,s″′ _i ,…,s″′ _k ); s″′ _i represents the index position corresponding to the i-th word t _i ;

Step 5. Record the total number of words corresponding to the short text with the largest total number of words in the corpus as max; according to the total number of words max, the part-of-speech representation vector set p"' and the index vector set s"' are respectively filled with "0", so that The total number of words in the part-of-speech representation vector set p"' and the index vector set s"' is equal to max; the complemented part-of-speech representation vector set is denoted as p" and the complemented index vector set is denoted as s";

Step 6. As shown in Figure 1, use the pre-training XLNET model to convert the jth part of speech in the filled part of speech representation vector set p" to the n-dimensional word vector, and obtain the jth part of speech vector, denoted as p _j ＝(p _j,1 ,p _j,2 ,…,p _j,d ,…,p _j,n ), p _j,d represents the d-th dimension value in the j-th part-of-speech vector, d∈[1,n ];

Step 7, as shown in Figure 1, use the pre-training XLNET model to perform the transformation of the n-dimensional word vector on the jth word in the index vector set s" after the completion, to obtain the jth word vector, denoted as s _j = (s _j,1 ,s _j,2 ,…,s _j,d ,…,s _j,n ), s _j,d represents the d-th dimension value in the j-th word vector;

Step 8. Select F aspect words from the corpus, input the f aspect word into the pre-training XLNET model, as shown in Figure 1, obtain the word vector a _f =(a _f,1 , a _f,2 ,…,a _f,d ,…,a _f,n ), a _f,d represents the d-th dimension value corresponding to the f-th aspect word;

Step 9. Arrange the elements in the part-of-speech representation vector set p"' and the index vector set s"' respectively in reverse order, and fill them with "0" according to the total number of words max; the completed reversed-order part-of-speech representation vector set is recorded as p"" and the complemented reverse index vector set is denoted as s"";

Step 10, use the pre-training XLNET model to perform the transformation processing of the n-dimensional word vector on the jth part of speech in the complemented reverse order part of speech representation vector set p"", and obtain the jth reverse order part of speech vector, denoted as p' _j = ( p′ _j,1 ,p′ _j,2 ,…,p′ _j,d ,…,p′ _j,n ), p′ _j,d represents the d-th dimension value in the j-th reverse part-of-speech vector, d∈ [1,n];

Step 11. Use the pre-trained XLNET model to perform the transformation of the n-dimensional word vector on the jth word in the complemented reverse order index vector set s"" to obtain the jth reverse order word vector, denoted as s' _j = (s ′ _j,1 ,s′ _j,2 ,…,s′ _j,d ,…,s′ _j,n ), s′ _j,d represents the d-th dimension value in the j-th reversed word vector;

Step 12. Form the forward input features of the Bilstm model:

The jth word vector s _j , the jth part of speech vector p _j and the word vectors corresponding to the F aspect words are sequentially spliced to obtain the feature vector fw_cell _j and used as the forward input of the Bilstm model, the dimension of the feature vector fw_cell _j is 2×n+n×F; compared to using only word vectors or just splicing word vectors, part-of-speech vectors, and aspect vectors in pairs, by splicing word vectors, part-of-speech vectors, and aspect vectors, the original short text Ben retains more semantic information.

Step 13. Form the reverse input feature of the Bilstm model:

The jth reversed word vector s' _j , the _jth reversed part-of-speech vector p'j and the word vectors corresponding to the F aspect words are sequentially spliced to obtain the reversed feature vector bw_cell _j , which is used as the reverse input of the Bilstm model (Bilstm The model automatically converts sequential input into reverse-order input), and the reverse-order input helps to identify the following information, so that the context can be considered during the input process of short text;

Step 14. Establish F correction matrices and identify aspect information:

Let any f-th correction matrix be a matrix of (2+F) rows multiplied by 3 columns, where the element of the first column and the first row is "1", the element of the second column and the second row is "1", and the element of the first column is "1". The element in row f+2 of the third column is "1", and the rest of the elements are set to "0";

Input the word vector set t=(t ₁ ,t ₂ ,...,t _i ,...,t _k ) into the pre-training XLNET model to obtain the aspect information contained in the word vector set t;

The corresponding aspect words are obtained through the aspect information contained in the word vector set t, so as to obtain the corresponding correction matrix according to the aspect words;

Multiply the feature vector fw_cell _j and the correction matrix corresponding to the word vector set t to obtain the input vector I _j ;

Multiply the inverse order feature vector bw_cell _j and the correction matrix corresponding to the word vector set t to obtain the inverse order input vector I′ _j ;

Step 15, using F aspect words to train F Bilstm models respectively, to obtain F Bilstm models after training;

The input vector I _j is respectively input into the Bilstm model after the training of the corresponding aspect word, thereby obtaining the output vector hf _j corresponding to the aspect information contained in the word vector set t;

Input the reverse-order input vectors I′ _j into the trained Bilstm models of the corresponding aspect words respectively, so as to obtain the reverse-order output vector hb _j corresponding to the aspect information contained in the word vector set t;

Splicing the output vector hf _j and the reverse-order output vector hb _j to form an implicit vector h _j ;

Step 16: Splicing the implicit vector h _j , the j-th part-of-speech vector p _j and the word vectors corresponding to the F aspect words in sequence to obtain the implicit vector h′ _j of the Attention mechanism:

Step 17. Use the word vectors corresponding to the F aspects to train the Attention mechanism network, and obtain the Attention mechanism network after training (using the Attention mechanism allows the machine to decide which part of the input needs to be paid attention to, and allocate limited information processing resources to important parts);

Input the latent vector h′ _j of the Attention mechanism into the trained Attention mechanism network to obtain the weight corresponding to the jth word vector s _j ;

Step 18. Predict the classification result:

Multiply each hidden vector h _j and its corresponding weight and sum up, get h ^* input into the fully connected layer, and get the scores corresponding to positive emotion and negative emotion; The corresponding scores are input into the softmax layer, the probability corresponding to the positive emotion and the negative emotion is obtained, and the emotion with a larger probability is used as the predicted classification result.

Claims (1)

1. A short text aspect level emotion classification method is characterized by comprising the following steps:

step 1, acquiring all short texts in the comment data and using the short texts as a corpus, and performing preprocessing operations of classifying, cleaning and word segmentation on any one short text in the corpus to obtain a word vector set of the corresponding short text, wherein the word vector set is marked as t ═ t (t ═ t) ₁ ,t ₂ ,…,t _i ,…,t _k )，t _i Denotes the ith word, i ∈ [1, k ]]K represents a total number of words of the short text;

step 2, the word vector set t is equal to (t) ₁ ,t ₂ ,…,t _i ,…,t _k ) And performing part-of-speech recognition to obtain a part-of-speech characterization vector set p ' (p ') ' ₁ ,p″′ ₂ ,…,p″′ _i ,…,p″′ _k )，p″′ _i Denotes the ith word t _i The corresponding part of speech;

step 3, carrying out preprocessing operation on all short texts in the corpus according to the step 1, deleting repeated words to obtain a dictionary, numbering each word in the dictionary, and using the number as an index position key of the corresponding word;

step 4, utilizing the index position key of each word in the dictionary to set t (t) as the word vector set ₁ ,t ₂ ,…,t _i ,…,t _k ) Index processing is performed to obtain an index vector set s ' (s ') ' ₁ ,s″′ ₂ ,…,s″′ _i ,…,s″′ _k )；s″′ _i Denotes the ith word t _i The corresponding index position;

step 5, recording the total number of words corresponding to the short text containing the most total number of words in the corpus as max; filling the part of speech characterization vector set p 'and the index vector set s' with '0' according to the total number of words max, so that the total number of words in the part of speech characterization vector set p 'and the index vector set s' is equal to max; recording the complemented part-of-speech characterization vector set as p 'and the complemented index vector set as s';

step 6, utilizing a pre-trained XLNET model to represent the filled parts of speechThe j part of speech in the vector set p' is converted into an n-dimensional word vector to obtain a j part of speech vector which is marked as p _j ＝(p _j,1 ,p _j,2 ,…,p _j,d ,…,p _j,n )，p _j,d Represents the d-th dimension value in the j-th part-of-speech vector, d ∈ [1, n [ ]]；

Step 7, utilizing a pre-training XLNET model to convert the jth word in the complemented index vector set s' into an n-dimensional word vector to obtain a jth word vector which is marked as s _j ＝(s _j,1 ,s _j,2 ,…,s _j,d ,…,s _j,n )，s _j,d Representing the d dimension value in the j word vector;

step 8, selecting F aspect words from the corpus, inputting the F-th aspect word into the pre-training XLNET model to obtain a word vector a corresponding to the F-th aspect word _f ＝(a _f,1 ,a _f,2 ,…,a _f,d ,…,a _f,n )，a _f,d Representing the d dimension value corresponding to the f aspect word;

step 9, respectively carrying out reverse order arrangement on elements in the part of speech characterization vector set p 'and the index vector set s' and filling up the elements by '0' according to the total number max of the words; obtaining a word characteristic vector set of the filled reverse order as p 'and an index vector set of the filled reverse order as s';

step 10, utilizing a pre-training XLNET model to convert the j (th) part of speech in the complemented reverse-order part of speech characterization vector set p 'into an n-dimensional word vector to obtain a j (th) reverse-order part of speech vector which is marked as p' _j ＝(p′ _j,1 ,p′ _j,2 ,…,p′ _j,d ,…,p′ _j,n )，p′ _j,d Represents the d-th dimension value in the j-th reverse-order part-of-speech vector, and belongs to [1, n ]]；

Step 11, utilizing a pre-training XLNET model to convert the j-th word in the complemented reverse-order index vector set s 'into an n-dimensional word vector to obtain a j-th reverse-order word vector which is recorded as s' _j ＝(s′ _j,1 ,s′ _j,2 ,…,s′ _j,d ,…,s′ _j,n )，s′ _j,d Representing the d-th of the j-th reverse-order word vectorA dimension value;

and 12, forming the forward input characteristics of the Bilstm model:

the jth word vector s _j The jth part-of-speech vector p _j Sequentially splicing word vectors corresponding to the F aspect words to obtain a feature vector fw _ cell _j And as input to the forward bitstm model, the feature vector fw _ cell _j The dimension of (a) is 2 xn + nxf;

step 13, forming the reverse input characteristics of the Bilstm model:

j 'th reverse order word vector s' _j Jth reverse order part of speech vector p' _j Sequentially splicing word vectors corresponding to the F aspect words to obtain a reverse-order feature vector bw _ cell _j And is used as the other input of the reverse Bilstm model;

step 14, establishing F correction matrixes and identifying information in the aspect:

let any F-th rectification matrix be a matrix of multiplying (2+ F) rows by 3 columns, wherein the element of the first row of the first column is "1", the element of the second row of the second column is "1", the element of the F +2 th row of the third column is "1", and the rest elements are all set to "0";

setting the word vector set t ═ t (t) ₁ ,t ₂ ,…,t _i ,…,t _k ) Inputting the information into a pre-training XLNET model to obtain aspect information contained in the word vector set t;

acquiring corresponding aspect words through aspect information contained in the word vector set t, so as to obtain corresponding correction matrixes according to the aspect words;

the feature vector fw _ cell is used for carrying out the processing _j Multiplying with the correction matrix corresponding to the word vector set t to obtain an input vector I _j ；

The reverse order feature vector bw _ cell is processed _j Multiplying the correction matrix corresponding to the word vector set t to obtain an inverted-order input vector I' _j ；

Step 15, respectively training F Bilstm models by utilizing F side words to obtain F trained Bilstm models;

inputting the vector I _j Respectively input corresponding toObtaining an output vector hf corresponding to the aspect information contained in the word vector set t in the trained Bilstm model of the aspect word _j ；

Inputting the reverse order into vector I' _j Respectively inputting the training result into the Bilstm model of the corresponding aspect words so as to obtain the reverse order output vector hb corresponding to the aspect information contained in the word vector set t _j ；

Output vector hf _j And the reverse order output vector hb _j Splicing is carried out to form an implicit vector h _j ；

Step 16, converting the implicit vector h _j The jth part-of-speech vector p _j And sequentially splicing word vectors corresponding to the F aspect words to obtain an implicit vector h 'of an Attention mechanism' _j ：

Step 17, training an Attention mechanism network by using word vectors corresponding to F number of aspect words to obtain a trained Attention mechanism network;

implicit vector h 'of Attentition mechanism' _j Inputting the trained Attention mechanism network to obtain the jth word vector s _j A corresponding weight;

step 18, predicting classification results:

respectively dividing each implicit vector h _j Multiplying the obtained product by corresponding weight and summing the product to obtain h ^* Inputting the scores into the full-connection layer to obtain scores corresponding to positive emotions and negative emotions; and inputting the scores corresponding to the positive emotion and the negative emotion into the softmax layer to obtain the probabilities corresponding to the positive emotion and the negative emotion, and taking the emotion with higher probability as a prediction classification result.

Images