CN111259127A - Long text answer selection method based on transfer learning sentence vector - Google Patents

Abstract

The invention discloses a long text answer selection method based on the transfer learning sentence vector. A two-stage method is used to construct a transfer learning sentence vector network and a training prediction network. The transfer learning sentence vector network includes a twin network structure, an attention aggregation structure and a Classification layer; training prediction network includes Siamese network structure and distance metric layer. First of all, the present invention does not need to perform word segmentation on the text sequence of the data set, and directly uses the complete question answer sentence as the input, thereby avoiding the error propagation caused by the word segmentation tool. Second, the training prediction network in the second stage has a simple structure and high computational efficiency. Finally, the transfer learning method is introduced combined with the twin network structure and attention mechanism to obtain sentence vector model weights with more similar semantics, and the second-stage training prediction network provides sentence-level semantic vectors, which is better than traditional methods and ordinary deep learning. The network method has better effect, especially for long text data, its effect is more prominent.

Description

A long text answer selection method based on transfer learning sentence vector

technical field

The present invention relates to natural language processing, pre-training language model in deep learning, and attention mechanism. Specifically, it is a long text answer selection method based on transfer learning sentence vectors.

Background technique

With the rapid development of the Internet in recent years, various information platforms have exploded in a "blowout" manner. According to incomplete statistics from Hootsuite and wearesocial, as of 2019, the number of Internet users in the world has exceeded 350 million, and 45% of the world's population are users of social media. The data shows that from 2018 to 2019, there were 4.39 million new online users, and social media users increased by 3.48 million during the year. A large amount of data shows that the current global network has reached a very developed and prosperous age, and with the network brings countless Internet knowledge and information. A large number of websites carry network information and are flooded in the Internet environment, which brings about the problem of how to search and utilize effectively. Therefore, the existence of search engines is very important. Now the storage and computing speed of computers have ushered in a golden age. In the past, the computing power and storage capacity of computers had become a stumbling block to the development of search engines. With the advent of high-performance computing and high-performance storage, how to efficiently and accurately search for the most Relevant search results have become the focus of search engine research.

For this research focus, we must overcome the problem of accurately retrieving the most relevant information in massive documents. Throughout the history of search development, the first-generation search engine Archie 3 was mainly used to search for files distributed in various hosts. When the World Wide Web appeared, EINet Galaxy (Tradewave Galaxy) 4 appeared, which functioned as the earliest portal website. After successive generations of search engine technology updates, under the competition of Baidu, Google, and Bing search engines dominated by large Internet companies such as Baidu, Google, and Microsoft, how to search accurately will continue to be a research hotspot in the future. . With the rise of artificial intelligence, machine learning and deep learning methods have brought new solutions to the fields of image recognition, natural language processing, and speech recognition processing. Faced with the unsatisfactory status of search engine retrieval and recall results, many retrieval results require secondary screening and filtering by searchers, so automatic question-answering technology emerges as the times require.

Answer selection technology is an important step in automatic question answering technology, which is widely used in life, such as Xiaomi's Xiao Ai, Iphone's Siri, Microsoft's Xiaobing and Baidu Dubi, etc. are the actual products of automatic question answering technology. . In the field of task-based automatic question answering, the robotic assistants achieved by automatic question answering technology can greatly liberate their hands and can control and complete a series of tasks with only voice commands. In the field of chat-type automatic question and answer, chatbots can add a touch of joy to the boring life. In the field of modern medicine, automatic question answering technology can build a more convenient and efficient way of communication for doctors and patients. Therefore, how to improve the accuracy of question answering in the field of automatic question answering has become particularly important. For the field of automatic question answering of retrieval, the answer selection technology that plays a very important role in it also plays a very important role in the search engine introduced above. .

Existing answer selection methods usually use a siamese network structure to model the question text and the answer text separately, and finally distinguish whether the question and answer match through similarity measures such as cosine distance. However, traditional methods mainly focus on short text matching tasks, lack of research on long text application scenarios, and it is difficult to solve the problems of "semantic transfer" and "semantic gap" in the field of long text applications. Moreover, since the question-and-answer data in the medical field generally has the characteristics of "short questions and long answers", the matching effect and recall precision of the existing answer selection methods cannot meet the online needs. Therefore, in order to better select answers for long text data, The main technical difficulties involved are as follows:

1. How to design a model to model long text sequences;

2. How to use external knowledge to introduce transfer learning methods to improve recall accuracy;

3. How to design the evaluation index to quantify the effect of the model.

SUMMARY OF THE INVENTION

In order to solve the above problems, the present invention proposes a long text answer selection method based on transfer learning sentence vectors, uses BERT as a feature extraction layer to model long text data, and adopts a two-stage task of transfer learning + training prediction. First, the question and answer text sequences are used as input and processed using the BERT input format, which does not require additional word segmentation and avoids error propagation caused by word segmentation. Second, using the transfer learning method supplemented by the Siamese network structure and the attention aggregation structure, the question and answer sentence vectors obtained by transfer learning are more semantically similar. Finally, in the training prediction process, the model weight parameters of transfer learning are used to initialize the sentence vector of the text, and the semantic similarity of the question and answer sentence vectors is simply calculated by the distance measurement method. High recall efficiency and lower video memory occupation, the present invention adopts a two-stage method, and obtains higher recall precision compared to the [CLS] semantic vector that directly uses BERT.

In order to achieve the above object, the present invention adopts the following technical scheme:

A long text answer selection method based on transfer learning sentence vector, the steps are as follows:

1) Use XPATH to design a crawler to crawl the doctor-patient question and answer data of the consultation forum, and do data cleaning; use the answers in the doctor-patient question and answer data as a positive sample; for the questions in the doctor-patient question and answer data, use the Lucene index tool to make relevant answers According to the retrieval and recall, the correlation answers are regarded as negative samples; the point answer selection data set is constructed according to the obtained positive samples and negative samples, and the transfer learning data set and the training prediction data set are divided according to the ratio of 27:1 to 8:1;

2) Establish a transfer learning sentence vector network, including a twin network structure, an attention aggregation structure and a classification layer. The twin network structure includes a paired input layer, a feature extraction layer, and a pooling layer. The attention aggregation structure includes Attention layer and aggregation network layer; the feature extraction layer adopts the BERT model, and loads the full word masking weight BERT parameter for initialization. After feature extraction, the average pooling output is taken, and the features are processed through the attention layer and the aggregation network layer in turn. Aggregated output; the aggregated output vector is spliced with the BERT pooling output vector and input to the classification layer for binary output;

Using the transfer learning data set obtained in step 1), the transfer learning sentence vector network is trained, and the MRR and Precision@K evaluation index methods are used to match the binary value of whether the question and the answer match with the real label, and select the highest matching score. The network parameters corresponding to the model are obtained, and the BertAttTL transfer learning sentence vector model is obtained;

3) Establish a training prediction network, including a twin network structure and a distance measurement layer, and the twin network structure includes a paired input layer, a feature extraction layer, and a pooling layer; the feature extraction layer adopts the BERT model, using step 2) The obtained weight parameters of the BertAttTL transfer learning sentence vector model initialize the BERT model and pooling layer parameters in the training prediction network, output the question sentence vector and the answer sentence vector through the pooling layer, and input the two sentence vectors to the distance measurement layer. Obtain the semantic similarity, and divide the similarity according to the threshold to obtain the similar binary value as the prediction content output; use the training prediction data set obtained in step 1) to train the training prediction network, and use the MRR and Precision@K evaluation index methods , match the final binary classification value with the real label, select the network parameters corresponding to the model with the highest matching score, and obtain a trained training prediction network;

4) Input the question and answer text to be processed into the training prediction network obtained in step 3), output the binary classification values of all candidate answers, and obtain the final answer to the question to be processed.

Further, the described MRR and Precision@K evaluation index methods are specifically:

Denote the output of the transfer learning sentence vector network or the training prediction network as pred=[p ₁ ,p ₂ ,...,p _n ], where pi represents the predicted value of the _ith candidate answer, 0 or 1, and 0 represents no Similar, 1 represents similarity, n represents the number of test samples in the sample set; the real label data is represented as label=[t ₁ ,t ₁ ,,...,t _n ], where t _i represents the ith candidate answer The true label of 0 or 1, 0 indicates dissimilarity, 1 indicates similarity, and n indicates the number of test samples in the sample set; for all candidate answers of a question, obtain binary classification values through transfer learning sentence vector network or training prediction network Then sort to get the rank _i of the correct answer for the i-th question;

The formula for calculating MRR is:

Among them, Q is the set of questions, and |Q| represents the number of all questions;

The formula for Precision@K is:

Among them, Precisiin represents precision, K represents the number of answers considered in the index, and in the present invention, the values are 1, 2, and 3, Num (True Answers) represents the number of correct answers, and Sum (related K Answers) represents recall The total number of relevant answers.

Further, the transfer learning sentence vector network includes a twin network structure, an attention aggregation structure and a classification layer, and the twin network structure includes a paired input layer, a feature extraction layer, and a pooling layer. The attention aggregation layer. The structure includes an attention layer and an aggregation network layer. The attention layer mainly adds an attention mechanism to the Siamese network structure, uses the context of the question to enrich the semantic representation of the answer text, and uses the context of the answer to enrich the semantic representation of the question text. , through the semantic interaction of questions and answers, it can effectively improve the matching effect; the aggregation network layer mainly fuses the question and answer representations after the attention mechanism. The comparison layer and the aggregation layer further deepen the similarity between the question and the answer. Feature modeling of parts and dissimilar parts, which can effectively improve the matching effect based on the attention mechanism. The feature extraction layer is modeled with BERT, and BERT is initialized with the BERT weight parameter covered by the whole word;

Input the paired samples into the Siamese network structure. The paired input layers correspond to two text sequences of question and answer. According to the input format of BERT [CLS]+Question+[SEP], [CLS]+Answer+[SEP], the question text is respectively And the answer text is processed; after the BERT feature modeling, the output of the 12-layer pooling layer is averaged, and the pooled outputs with unified dimensions are obtained respectively: the question pooling output Q pool and the answer pooling output A pool, the dimension length is 768 dimensions;

The question pooling output Q pool and the answer pooling output A pool are input to the attention layer, and the question semantic alignment vector Z ₂ and the answer semantic alignment vector Z ₂ ' are obtained respectively through the attention mechanism; Q pool, A pool, Z ₂ and Z ₂ ' are input to the aggregation network layer, for the problem, Q pool and Z ₂ are transformed by [Q pool, Z ₂ ], [Q pool, Q pool-Z ₂ ], [Q pool, Qpool*Z ₂ ], and then The splicing vector [O ₁ , O ₂ , O ₃ ] is obtained by splicing through a layer of linear transformation. The splicing vector is subjected to a layer of linear transformation and the DropOut mechanism is used to obtain the problem attention aggregation output Fused _Q ; Similarly, for the answer, Apool and Z ₂ ' go through the aggregation network layer to get the answer attention aggregation output Fused _A ;

Further splicing Fused _Q , Fused _A , Q pool, and A pool to obtain [Q pool,A pool,|Q pool-Apool|,Q pool*A pool,Fused Q,Fused A], and input the splicing vector into the classification layer, The predicted output pred=[p ₁ ,p ₂ ,...,p _n ] is obtained by Softmax classification, where p _i represents the predicted value of the ith candidate answer, 0 or 1, 0 means dissimilar, 1 means similar, and n means The number of test examples in the sample set.

Further, the method for calculating the semantic similarity in the step 3) adopts any one of cosine distance, Manhattan distance, Euler metric, and dot product metric.

The beneficial effects that the present invention has:

(1) The present invention uses the pre-trained language model BERT in the natural language processing technology to obtain the word representation of the long text data, does not require an additional data word segmentation stage, avoids the inaccurate word segmentation problem caused by the word segmentation tool, and thus avoids word segmentation due to word segmentation. Semantic error propagation problems caused by inaccuracy;

(2) A two-stage method is designed. In the first stage, the transfer learning method is effectively used to utilize the knowledge of large-scale parallel corpus. In the second stage, a simple training prediction network is used to have higher model inference efficiency. The answer chooses the recall precision;

(3) For a large-scale answer search scenario, the method of directly obtaining all text sequence sentence vectors proposed by the present invention can effectively avoid time-consuming computation among multiple text pairs of the pre-trained language model, and is more efficient. For example: when the pre-trained language model calculates the matching score of the same question and m answers, the question and one of the answers need to be paired into the model for calculation each time, so the question is repeatedly encoded m times, the question and the answer A total of 2*m times are encoded. In the large-scale search scenario, the value of m is very large, and the extra time overhead brought by it is very huge. However, the present invention only needs to obtain the sentence vectors of the question and all the answers, and only needs The coding question is once and the answer is m times a total of m+1 times. Compared with 2*m times of coding work, the method of the present invention reduces the coding time by nearly half, so the efficiency is higher;

(4) The present invention adopts the pre-trained language model BERT as the feature extractor, which can effectively carry out semantic modeling on long text data, and avoids the phenomenon of "semantic migration" and "semantic gap" in the long text data of the existing answer selection methods .

Description of drawings

Fig. 1 is the transfer learning model structure diagram of the long text answer selection method based on the transfer learning sentence vector;

Fig. 2 is the structure diagram of the training prediction model of the long text answer selection method based on the transfer learning sentence vector.

Detailed ways

The present invention will be described in detail below with reference to specific examples.

Since the question and answer data in the medical field generally has the characteristics of "short questions and long answers", the matching effect and recall accuracy of the existing answer selection methods cannot meet the online needs. Therefore, the long text answer based on the transfer learning sentence vector proposed by the present invention The selection method has been experimentally verified and can effectively deal with long text answer selection problems.

As shown in Figure 1, a method for selecting long text answers based on transfer learning sentence vectors proposed by the present invention, the adopted transfer learning sentence vector network includes an input layer, a feature extraction layer, an attention aggregation network layer, and a classification layer, The feature extraction layer is modeled with BERT, and BERT is initialized with the BERT weight parameter covered by the whole word;

The input layer corresponds to two text sequences of question and answer, and processes the two texts according to the BERT input format [CLS]+Question+[SEP], [CLS]+Answer+[SEP]. After the BERT feature modeling, the output of the 12-layer pooling layer is averaged to obtain a pooled output with a unified dimension, and the dimension length is 768 dimensions; the attention aggregation network layer obtains the semantic alignment output of the two text sequences through the attention mechanism, and the alignment vector Z2 And the pooled output Z1 is transformed by [Z1, Z2], [Z1, Z1-Z2], [Z1, Z1*Z2], and is spliced through a layer of linear transformation to obtain [O1, O2, O3], and the splicing vector goes through a Layer linear transformation and use the DropOut mechanism to obtain the question attention aggregation output FusedQ and the answer attention aggregation output FusedA, and splicing the two pooled outputs to obtain [Q pool,A pool,|Q pool-A pool|,Q pool *A pool, Fused Q, FusedA], classify it through Softmax to get the predicted output, and after the transfer learning sentence vector network training, get the sentence vector with more similar semantics.

As shown in Figure 2, a method for selecting long text answers based on transfer learning sentence vectors proposed by the present invention, the training prediction network included in the adopted training prediction network includes an input layer, a feature extraction layer, and a distance measurement layer. The feature extraction layer adopts BERT, and is initialized with the transfer learning weight parameters trained in step 3);

The input layer corresponds to two text sequences of question and answer, and processes the two texts according to the BERT input format [CLS]+Question+[SEP], [CLS]+Answer+[SEP]. After the BERT feature modeling, the output of the 12-layer pooling layer is averaged to obtain a pooled output with a unified dimension, and the dimension length is 768 dimensions; use the transfer learning weight parameters trained in step 3) to initialize, and obtain sentence vectors with more similar semantics. The cosine distance, Manhattan distance, Euler metric, and dot product metric are used to calculate the similarity of two sentence vectors, and a threshold is used to segment the similarity to obtain the binary value of whether they are similar.

In a specific implementation example of the present invention, the above-mentioned transfer learning sentence vector network and training prediction network are used to select an answer on the long text question and answer data, and the steps are as follows:

Step 1. Build a crawler framework through Python and XPATH, grab the doctor-patient question-and-answer data from medical consultation platforms such as Sanjiu Health.com, and take certain rules and methods to remove webpage tags other than text, such as <div>, etc., to remove the data. After processing, about 5.75 million doctor-patient question and answer data are finally obtained, which are stored in the database in the form of (question, condition description, condition answer) triplet.

Step 2. The answer of the condition in the warehouse is the correct answer. Use the Lucene tool to recall the relevant answer to the question, recall 500 negative sample answer sets sorted by relevance, and select one negative sample from the 1st to 5th negative samples. One out of 5 to 50 negative samples, one out of 50 to 100 negative samples, and one out of 100 to 500 negative samples. For the recalled samples with less than 100 relevant negative sample answers, the final sampling between the 100th and 500th is reduced in the construction of the candidate answer set. 4,354,417 full data sets are sampled according to topic categories, and small sample data sets are used as training prediction data sets, including 120,000 training sets, 20,000 validation sets and 20,000 test sets. The labeled data is taken as the transfer learning data at 8:1 of the total. set, where the transfer learning dataset does not intersect with the training prediction dataset.

In a specific embodiment of the present invention, the corpus format is as follows:

Where Question represents the question text and Answer represents the answer text.

Step 3. Use Pytorch to build a transfer learning sentence vector network, and use the whole word to cover the BERT weight parameters for initialization. The network includes an input layer, a feature extraction layer, an attention aggregation network layer, and a classification layer. On the transfer learning data set obtained in step 2 Carry out training prediction, and finally get the weight file of sentence vector model with more similar semantic vectors.

The loss function trained by the transfer learning sentence vector network adopts the cross entropy loss:

loss=-y*logy'

where y represents the true label of whether the answer to the question matches, and y′ is the model prediction vector of whether the sample data matches.

In the test set, for a question q and 3 answers [a ₁ , a ₂ , a ₃ ] for the prediction vector pred=[0.71, 0.68.0.35] and the true label label=[0, 1, 0], according to the following MRR The calculation formula, |Q|=1, takes the threshold value of 0.5 as the division prediction result to get pred=[1,1,0], according to the real label, we can see that the correct answer label is correctly predicted, and at the same time, the answers are sorted according to the predicted probability to get the second The answer prediction probability is the second highest, that is, rank _i = 2, then MRR = 1/2 = 0.5. According to the calculation formula of Precision-K, K takes 1, 2 and 3. It can be known that when k=1, Num(True Answers)=0, then Precision@1=0; when k=2, Num(True Answers)=1 , Sum(related K Answers)=2, then Precision@2=0.5; when k=3, Num(True Answers)=1, Sum(related K Answers)=3, then Precision@3=1/3=0.33 . This example only explains one question and multiple answers. There are multiple questions in the test set, and the final result indicator needs to be calculated by taking the average of the number of questions.

Step 4. Use Pytorch to build a training prediction network, use the step 3 transfer learning sentence vector network weight model to initialize, including the input layer, feature extraction layer, distance measurement layer, and perform training prediction on the training prediction small sample data set obtained in step 2. .

The loss function for training the prediction network uses the mean squared error loss:

loss=(yy') ²

where y represents the true label of whether the answer to the question matches, and y′ is the model prediction vector of whether the sample data matches.

针对测试集中所有样例得出pred预测结果，同时与真实标签label进行比较，按照MRR和Precision@K(k取值1,2,3)计算公式得到测试集上的指标。After obtaining the question sentence vector and the answer sentence vector, use the cosine similarity classifier to calculate the semantic similarity of the two sentence vectors, the formula is as follows, for example: the question sentence vector is [1,1,0,0,1], the answer sentence vector is [0,1,1,0,0], then the similarity calculated according to the cosine similarity formula is

The pred prediction results are obtained for all samples in the test set, and compared with the real label label, and the indicators on the test set are obtained according to the calculation formula of MRR and Precision@K (k value is 1, 2, 3).

Step 5: Use the model trained in Step 4 to perform inference on the test set data, and the final predicted value is divided according to the threshold to obtain whether the answer to the question is semantically similar.

Compared with the prior art, firstly, the present invention does not need to perform word segmentation on the text sequence of the data set, and directly uses the complete question answer sentence as the input, thereby avoiding the error propagation caused by the word segmentation tool. Second, the training prediction network in the second stage has a simple structure and high computational efficiency. Finally, the transfer learning method is introduced combined with the twin network structure and attention mechanism to obtain sentence vector model weights with more similar semantics, and the second-stage training prediction network provides sentence-level semantic vectors, which is better than traditional methods and ordinary deep learning. The network method has better effect, especially for long text data, its effect is more prominent. In order to objectively evaluate the performance of the model of the present invention, the model of the present invention is compared with other models, and the comparative models include Siamese RNN, QACNN, DEATT, Cam, Seq Match Seq, and ESIM. The evaluation indicators used in this embodiment are MRR, Precision@1, Precision@2, and Precision@3. These metrics are used to evaluate the similarity between questions and recalled answers. The larger the value, the better the effect. As shown in Table 1, the present invention integrates two-stage tasks and has higher recall precision for answer selection, and the model effect of the present invention is better than all comparison models. As shown in Table 2, compared with the pre-trained language model BERT, the time-consuming of the inference stage is only 0.5 seconds, and the efficiency is high.

Table 1. Recall precision results of comparative experiments

Model MRR Precision@1 Precision@2 Precision@3 Siamese RNN 0.571769 0.311137 0.580483 0.833433 QACNN 0.612844 0.363327 0.650470 0.873225 DEATT 0.525945 0.258348 0.508098 0.745051 Cam 0.636339 0.415917 0.656469 0.827634 Seq Match Seq 0.631340 0.407518 0.651070 0.828834 ESIM 0.523529 0.254749 0.505299 0.743251 this invention 0.739136 0.543491 0.818636 0.971406

Table 2 Comparison of the calculation time-consuming results of the present invention and the pre-trained language model

Model Time-consuming inference stage (number of answers m=4) Pre-trained language model BERT 4.5 seconds this invention 0.5 seconds

The above embodiment only expresses a specific embodiment of the present invention, and its description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present invention. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can also be made, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention should be subject to the appended claims.

Claims (4)

1. a long text answer selection method based on transfer learning sentence vector, is characterized in that, step is as follows: 1) Obtain authoritative doctor-patient question and answer data, and use the answers in the doctor-patient question and answer data as positive samples; for the questions in the doctor-patient question and answer data, use the Lucene index tool to retrieve and recall relevant answers, and use the relevant answers as negative samples; Construct the answer selection data set according to the obtained positive samples and negative samples, and divide the transfer learning data set and the training prediction data set according to the ratio of 27:1 to 8:1; 2) Establish a transfer learning sentence vector network, including a twin network structure, an attention aggregation structure and a classification layer. The twin network structure includes a paired input layer, a feature extraction layer, and a pooling layer. The attention aggregation structure includes Attention layer and aggregation network layer; the feature extraction layer adopts the BERT model, and loads the full word masking weight BERT parameter for initialization. After feature extraction, the average pooling output is taken, and the features are processed through the attention layer and the aggregation network layer in turn. Aggregated output; the aggregated output vector is spliced with the BERT pooled output vector and input to the classification layer for binary output; Using the transfer learning data set obtained in step 1), the transfer learning sentence vector network is trained, and the MRR and Precision@K evaluation index methods are used to match the binary value of whether the question and the answer match with the real label, and select the highest matching score. The network parameters corresponding to the model are obtained, and the BertAttTL transfer learning sentence vector model is obtained; 3) Establish a training prediction network, including a twin network structure and a distance measurement layer, and the twin network structure includes a paired input layer, a feature extraction layer, and a pooling layer; the feature extraction layer adopts the BERT model, using step 2) The obtained weight parameters of the BertAttTL transfer learning sentence vector model initialize the BERT model and pooling layer parameters in the training prediction network, output the question sentence vector and the answer sentence vector through the pooling layer, and input the two sentence vectors to the distance measurement layer. Obtain the semantic similarity, and divide the similarity according to the threshold to obtain the similar binary value as the prediction content output; use the training prediction data set obtained in step 1) to train the training prediction network, and use the MRR and Precision@K evaluation index methods , match the final binary classification value with the real label, select the network parameters corresponding to the model with the highest matching score, and obtain a trained training prediction network; 4) Input the question and answer text to be processed into the training prediction network obtained in step 3), output the binary classification values of all candidate answers, and obtain the final answer to the question to be processed. 2. a kind of long text answer selection method based on transfer learning sentence vector as claimed in claim 1, is characterized in that, described MRR, Precision@K evaluation index method is specifically: Denote the output of the transfer learning sentence vector network or the training prediction network as pred=[p ₁ ,p ₂ ,...,p _n ], where pi represents the predicted value of the _ith candidate answer, 0 or 1, and 0 represents no Similar, 1 represents similarity, n represents the number of test samples in the sample set; the real label data is represented as label=[t ₁ ,t ₂ ,,...,t _n ], where t _i represents the ith candidate answer The true label of 0 or 1, 0 means dissimilar, 1 means similar; for all candidate answers of a question, after obtaining the binary classification value through the transfer learning sentence vector network or training the prediction network, sort them, and get the correct answer for the i-th question. the answer's rank rank _i ; The formula for calculating MRR is:

Among them, Q is the set of questions, and |Q| represents the number of all questions; The formula for Precision@K is:

Among them, Precision represents precision, K represents the number of answers considered in the index, and in the present invention, the values are 1, 2, and 3, Num (True Answers) represents the number of correct answers, and Sum (related K Answers) represents recall The total number of relevant answers. 3. a kind of long text answer selection method based on transfer learning sentence vector as claimed in claim 1 is characterized in that, described transfer learning sentence vector network comprises twin network structure, attention aggregation structure and classification layer, described The Siamese network structure includes a paired input layer, a feature extraction layer, and a pooling layer. The attention aggregation structure includes an attention layer and an aggregation network layer; the feature extraction layer is modeled by BERT, and BERT uses a full The BERT weight parameters covered by the word are initialized; Input the paired samples into the Siamese network structure. The paired input layers correspond to two text sequences of question and answer. According to the input format of BERT [CLS]+Question+[SEP], [CLS]+Answer+[SEP], the question text is respectively And the answer text is processed; after the BERT feature modeling, the output of the 12-layer pooling layer is averaged, and the pooled outputs with unified dimensions are obtained respectively: the question pooling output Q pool and the answer pooling output A pool, the dimension length is 768 dimensions; The question pooling output Q pool and the answer pooling output A pool are input to the attention layer, and the question semantic alignment vector Z ₂ and the answer semantic alignment vector Z ₂ ' are obtained respectively through the attention mechanism; Q pool, A pool, Z ₂ And Z ₂ ' is input to the aggregation network layer, for the problem, Q pool and Z ₂ are transformed by [Q pool, Z ₂ ], [Q pool, Q pool-Z ₂ ], [Q pool, Q pool*Z ₂ ], Then, the splicing vector [O ₁ , O ₂ , O ₃ ] is obtained by splicing through a layer of linear transformation. The splicing vector is subjected to a layer of linear transformation and the DropOut mechanism is used to obtain the problem attention aggregation output Fused _Q ; Similarly, for the answer , A pool and Z ₂ ' go through the aggregation network layer to get the answer attention aggregation output Fused _A ; Further splicing Fused _Q , Fused _A , Q pool, and A pool to obtain [Q pool,A pool,|Q pool-A pool|,Q pool*A pool,Fused Q,Fused A], and input the splicing vector into the classification layer , the predicted output pred=[p ₁ ,p ₂ ,...,p _n ] is obtained by Softmax classification, where p _i represents the predicted value of the ith candidate answer 0 or 1, 0 means dissimilar, 1 means similar, n Indicates the number of test samples in the sample set. 4. a kind of long text answer selection method based on transfer learning sentence vector as claimed in claim 1, is characterized in that, in described step 3), the calculation method of semantic similarity adopts cosine distance, Manhattan distance, Euler metric , any of the point product metrics.

Images