CN119396986A - Retrieval enhancement generation method based on sequence generation - Google Patents
Retrieval enhancement generation method based on sequence generation Download PDFInfo
- Publication number
- CN119396986A CN119396986A CN202411185369.4A CN202411185369A CN119396986A CN 119396986 A CN119396986 A CN 119396986A CN 202411185369 A CN202411185369 A CN 202411185369A CN 119396986 A CN119396986 A CN 119396986A
- Authority
- CN
- China
- Prior art keywords
- text
- sequence
- sample
- query
- relevant
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/3332—Query translation
- G06F16/3338—Query expansion
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3347—Query execution using vector based model
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a search enhancement generation method based on sequence generation, which comprises the steps of searching a query text to obtain candidate related texts, generating a current predicted text block vector based on an initialization context sequence and a sequence generation model to obtain a most related text sequence from the candidate related texts by matching based on the current predicted text block vector, updating the initialization context sequence based on the current most related text sequence, generating a next predicted text block vector based on the updated initialization context sequence and a sequence generation model to obtain the most related text sequence of the next round until the obtained most related text sequence meets preset conditions, and determining a target prompt word based on the obtained most related text sequence. According to the method provided by the invention, the candidate related text obtained by retrieval is optimized through the sequence generation model, so that coherent and high-quality prompt words are generated, and further, the performance of a downstream task for fine adjustment or prompt learning based on the prompt words is improved.
Description
Technical Field
The invention relates to the technical field of natural language processing, in particular to a retrieval enhancement generation method based on sequence generation.
Background
With the rapid development of natural language processing and deep learning, more and more tasks begin to use pre-trained language models to improve effects, such as information retrieval, question-answering systems, dialog generation, and the like. To accommodate downstream tasks, pre-trained language models typically require fine-tuning or prompt learning in conjunction with task-related prompt words (prompt). The current construction method of the prompt word is mainly characterized in that information related to input is searched in a large-scale corpus, and the searched related information is directly spliced to obtain the prompt word.
But directly splice the search result to input, neglect noise and redundancy of the search text, resulting in low quality of the generated prompt word, and further affecting the training effect of fine tuning or prompt learning based on the prompt word.
Disclosure of Invention
The invention provides a retrieval enhancement generation method based on sequence generation, which is used for solving the defect of low quality of generated prompt words based on retrieval results obtained by direct splicing in the prior art.
The invention provides a retrieval enhancement generation method based on sequence generation, which comprises the following steps:
acquiring a query text, and searching the query text to obtain candidate related texts;
Obtaining an initialization context sequence of a current round based on the query text, generating a model based on the initialization context sequence and the sequence, and generating a current predicted text block vector so as to obtain a most relevant text sequence of the current round from the candidate relevant texts by matching based on the current predicted text block vector;
Updating the initialization context sequence based on the current most relevant text sequence, generating a model based on the updated initialization context sequence and the sequence, and generating a next predicted text block vector so as to obtain the most relevant text sequence of the next round from the candidate relevant texts in a matching way based on the next predicted text block vector until the obtained most relevant text sequence meets a preset condition;
And determining a target prompt word of the query text based on the obtained most relevant text sequence to determine an enhanced reply text of the query text based on the target prompt word, wherein the sequence generation model is obtained by training an initial sequence generation model based on a sample query text.
According to the search enhancement generation method based on sequence generation, the initialization context sequence of the current turn is obtained based on the query text, and the method comprises the following steps:
Generating a model based on the query text, the candidate related text and the sequence, and generating a reply text vector of the query text;
and obtaining an initialization context sequence of the current round based on the reply text vector.
According to the search enhancement generation method based on sequence generation, the candidate related texts comprise a plurality of candidate text segments with different text sources;
The matching, based on the current predicted text block vector, the most relevant text sequence of the current turn from the candidate relevant texts comprises the following steps:
respectively calculating semantic similarity between the current predicted text block vector and each candidate text segment;
and based on the calculated semantic similarity, matching the candidate text segments to obtain the most relevant text sequence of the current turn.
According to the method for generating the search enhancement based on the sequence generation, the searching of the query text is carried out to obtain candidate related texts, and the method comprises the following steps:
sparse retrieval is carried out based on keywords in the query text, so that a key related text is obtained;
Performing intensive search based on the query text to obtain a semantic related text;
and obtaining the candidate related text based on the key related text and the semantic related text.
According to the retrieval enhancement generation method based on sequence generation, the training steps of the sequence generation model comprise:
acquiring the sample query text and a plurality of sample related texts corresponding to the sample query text;
Performing mutual information calculation based on the sample query text and the plurality of sample related texts to obtain optimal upper and lower Wen Pianduan of the sample query text;
inputting the sample query text and the optimal context segment into the initial sequence generation model to obtain a sample prediction text block output by the initial sequence generation model;
And predicting the text block and the optimal context segment based on the sample to obtain a prediction loss, and iterating the initial sequence generation model based on the prediction loss to obtain the sequence generation model.
According to the search enhancement generation method based on sequence generation provided by the invention, mutual information calculation is carried out on the basis of the sample query text and the plurality of sample related texts to obtain an optimal context segment of the sample query text, and the method comprises the following steps:
respectively carrying out segmentation processing on the plurality of sample related texts to obtain sample related text fragments of each sample related text;
Performing mutual information calculation based on the sample query text and each sample related text fragment to obtain a contribution value of each sample related text fragment;
selecting preferable upper and lower Wen Pianduan from each sample related text based on the contribution value of each sample related text fragment;
and aggregating the preferable context fragments in the relevant texts of each sample to obtain the optimal context fragments of the sample query text.
The invention also provides a retrieval enhancement generation device based on the sequence generation, which comprises the following steps:
the acquisition unit is used for acquiring a query text and searching the query text to obtain candidate related texts;
The initial prediction unit is used for obtaining an initialization context sequence of the current round based on the query text, generating a model based on the initialization context sequence and the sequence, and generating a current prediction text block vector so as to obtain a most relevant text sequence of the current round in a matching way from the candidate relevant texts based on the current prediction text block vector;
the cyclic prediction unit is used for updating the initialization context sequence based on the current most relevant text sequence, generating a model based on the updated initialization context sequence and the sequence, and generating a next predicted text block vector so as to obtain the most relevant text sequence of the next round from the candidate relevant texts in a matching mode based on the next predicted text block vector until the obtained most relevant text sequence meets a preset condition;
The generating unit is used for determining target prompt words of the query text based on the obtained most relevant text sequence so as to determine enhanced reply text of the query text based on the target prompt words, and the sequence generating model is obtained by training an initial sequence generating model based on a sample query text.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the retrieval enhancement generation method based on sequence generation as described in any one of the above when executing the program.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a retrieval enhancement generation method based on sequence generation as described in any of the above.
The invention also provides a computer program product comprising a computer program which when executed by a processor implements a retrieval enhancement generation method based on sequence generation as described in any of the above.
According to the search enhancement generation method based on sequence generation, the candidate related texts are obtained through searching the query text, so that the diversity of prompt words is ensured. And then, optimizing the candidate related texts obtained by searching through a sequence generation model, taking the candidate text fragments obtained by searching as basic units of similar words, and optimizing and combining the candidate related texts by utilizing a sequence generation algorithm, so that coherent and high-quality prompt words can be obtained, and further, the performance of downstream tasks for fine tuning or prompt learning based on the prompt words is improved.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow diagram of a search enhancement generation method based on sequence generation provided by the invention;
FIG. 2 is a second flow chart of the method for generating search enhancement based on sequence generation provided by the invention;
FIG. 3 is a schematic diagram of a training flow of the sequence generation model provided by the present invention;
FIG. 4 is a flow chart of a method of constructing sample data provided by the present invention;
FIG. 5 is a schematic diagram of the structure of the search enhancement generation device based on sequence generation provided by the invention;
fig. 6 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The current method for constructing the prompt word mainly depends on manual design or heuristic rules, is time-consuming and labor-consuming, and is difficult to cover diversified query intentions, so that the generated prompt word is difficult to accurately describe the real demands of users. In addition, prompt word learning for downstream tasks lacks an effective optimization mechanism, and the potential of a pre-training language model is difficult to fully mine. To solve this problem, a method of automatically constructing a hint word using a search technique has been developed. The method for enhancing the retrieval can obtain diversified background knowledge by retrieving information related to input in a large-scale corpus, and reduce the dependence on manual annotation data. However, the existing retrieval enhancement method usually directly splices the retrieval result into input, ignores noise and redundancy of the retrieval text, and causes low quality of the generated prompt words.
Aiming at the problem, the invention provides a retrieval enhancement generation method based on sequence generation, so as to realize more coherent, high-quality prompt words which accord with the intention of a user. Fig. 1 is a schematic flow chart of a search enhancement generation method based on sequence generation, provided by the invention, as shown in fig. 1, the method includes:
Step 110, acquiring a query text, and searching the query text to obtain candidate related texts;
Step 120, based on the query text, obtaining an initialization context sequence of the current round, generating a model based on the initialization context sequence and the sequence, and generating a current predicted text block vector so as to obtain a most relevant text sequence of the current round by matching from the candidate relevant texts based on the current predicted text block vector;
Step 130, updating the initialization context sequence based on the current most relevant text sequence, generating a model based on the updated initialization context sequence and the sequence, and generating a next predicted text block vector so as to obtain the most relevant text sequence of the next round from the candidate relevant texts in a matching way based on the next predicted text block vector until the obtained most relevant text sequence meets a preset condition;
And 140, determining target prompt words of the query text based on the obtained most relevant text sequence to determine enhanced reply text of the query text based on the target prompt words, wherein the sequence generation model is obtained by training an initial sequence generation model based on a sample query text.
Here, the query text refers to text included in a query request of a user, and may include question text, instructions, or text information related to the query. Such as historical interactions of the user, related documents, or text information in an external knowledge base. Thus, the query text herein may reflect the user's query intent. Candidate relevant text herein refers to background knowledge related to query intent in query text, and may contain candidate relevant text segments from multiple sources.
In addition, the initialization context sequence herein refers to a predicted text block vector that contains query text and has been generated based on a sequence generation model. It will be appreciated that the predicted text block vectors in the initialization context sequence become increasingly rich as progressive predictions of the hint words are generated.
The sequence generation model here may be based on a decoder-only generation model. Therefore, the query text and the candidate related text obtained by searching can be subjected to vector coding by an encoder such as Bert and the like to obtain a query text vector and a related text segment vector.
Specifically, first, text information in a query request may be taken as query text by acquiring a user query request. The text in the query request can be subjected to data preprocessing including text cleaning, word segmentation, stop word removal and the like, and the processed text is used as the query text, so that the query text can be subjected to subsequent execution steps. The query text may then be retrieved from the large-scale corpus, with background knowledge associated with the query text as candidate relevant text. It should be noted that the number and the objects of the large-scale corpus are not particularly limited, and may be a professional knowledge base in a specific field or a basic knowledge base such as a large language model. Therefore, the candidate related text obtained by the method contains extensive and rich background knowledge, and further the diversification of the prompt words is ensured.
Further, a query text vector obtained after vectorizing the query text may be used as the initial context sequence. Or the query text vector and the reply text vector corresponding to the query text vector can also be used as the initial context sequence. The initialization context sequence may then be input into a sequence generation model, by which the next text block may be predicted based on the initial context sequence, and the next text block with the highest probability may be used as the current predicted text block vector. Then, based on the current predicted text block vector, the most relevant text sequence of the current round is obtained from the candidate relevant texts in a matching mode. For example, the related text segment vector with the highest semantic similarity with the current predicted text block vector may be used as the most relevant text sequence by calculating the semantic similarity between the current predicted text block vector and each related text segment vector in the candidate related texts.
It can be understood that the current predicted text block vector generated by the sequence model screens the candidate related texts obtained by retrieval, selects the text segment most related to the candidate related texts, further improves the quality of the prompting words on the basis of guaranteeing the continuity of the prompting words, and avoids the interference caused by text information with low relevance.
Then, the initialization context sequence is updated by the current most relevant text sequence, which may be that the text vector of the current most relevant text sequence of the current round is added to the initialization context sequence and arranged after the query text vector to obtain the updated initialization context sequence. Then, the updated initialization context sequence is input to a sequence generation model, and a next predicted text block vector is generated based on the updated initialization context sequence through the sequence generation model. For example, generating the next predicted text block vector may be accomplished by the following equation:
In the formula, Representing a next predicted text block vector; representing a decoder-only sequence generation model; representing the initial context sequence.
Similarly, the related text segment vector with the highest semantic similarity with the next predicted text block vector can be used as the most related text sequence of the next round by calculating the semantic similarity between the next predicted text block vector and each related text segment vector in the candidate related texts. And circulating based on the steps until the obtained most relevant text sequence meets the preset condition. For example, the most relevant text sequence generated by a plurality of rounds reaches a preset maximum generation length, or the decoder generates a specific termination symbol, and the symbol may be a quotation mark "". The preset conditions herein may be set correspondingly based on actual requirements for the prompt word, which is not specifically limited in the embodiment of the present invention.
Then, the most relevant text sequences obtained in each round can be arranged according to the sequence of the most relevant text sequences in each round in the initialization context sequence, so as to obtain a final coherent text sequence. And then, integrating and outputting based on the coherent text sequences to obtain the final prompt word. The process of integrating the output may include text processing operations such as grammar rectification, style adjustment, etc., to improve the naturalness of the text and the satisfaction of the user. Finally, the enhanced reply text corresponding to the query text can be generated through the indication of the target prompt word. The enhanced reply text refers to the reply text of the question obtained based on the above-mentioned retrieval enhancement method, and can be regarded as more pertinent reply text to the query intention of the user.
It should be noted that, compared with the prior art, the method directly splices each candidate text segment to obtain the prompt word, which results in the incoherence of the generated prompt word. According to the method, the candidate related text fragments obtained through retrieval are regarded as basic units of similar words, a weight or generation probability is given to each text fragment, and then the text fragments are optimally combined by using a sequence generation algorithm, so that a coherent prompt word can be obtained. The sequence generation algorithm is introduced into the prompt word construction process, so that a brand new search enhancement generation prompt word optimization paradigm is created.
According to the method provided by the embodiment of the invention, the candidate related texts are obtained by searching the query text, so that the diversity of the prompt words is ensured. And then, optimizing the candidate related texts obtained by searching through a sequence generation model, taking the candidate text fragments obtained by searching as basic units of similar words, and optimizing and combining the candidate related texts by utilizing a sequence generation algorithm, so that coherent and high-quality prompt words can be obtained, and further, the performance of downstream tasks for fine tuning or prompt learning based on the prompt words is improved.
In order to further improve the quality of the generated prompting words, the prompting words can more accurately describe the real demands of users. Based on any of the above embodiments, the step 120 of obtaining an initialization context sequence of the current round based on the query text includes:
Generating a model based on the query text, the candidate related text and the sequence, and generating a reply text vector of the query text;
and obtaining an initialization context sequence of the current round based on the reply text vector.
The reply text vector refers to a text vector used for replying to the query text, and when the query text is a question, the reply text can be an answer corresponding to the question, and when the query text is a command, the reply text can be an execution result corresponding to the command.
Specifically, a sequence generation model is input with a query text vector generated in advance based on the query text, and a reply text vector of the query text is generated by the sequence generation model by taking a relevant text segment vector of the candidate relevant text obtained by retrieval as background knowledge. The initialization context sequence for the current round may then be derived after the reply text vector is added to the query text vector to ensure that the predicted text block vector generated subsequently based on the sequence generation model is used for close correlation of the user query.
It should be noted that, the reply text corresponding to the query text can be used as the most dominant query intention of the user, and the generated reply text vector is used as the starting point of the generating process of the sequence generating model, so that the predicted text block vector generated in the subsequent generating process based on the sequence generating model is tightly related to the query intention of the user, the prompt word can more accurately describe the real requirement of the user, and the generated prompt word can obtain better performance in the downstream task.
Based on any of the above embodiments, the candidate related text includes a plurality of candidate text segments having different text sources;
The matching, based on the current predicted text block vector, the most relevant text sequence of the current turn from the candidate relevant texts comprises the following steps:
respectively calculating semantic similarity between the current predicted text block vector and each candidate text segment;
and based on the calculated semantic similarity, matching the candidate text segments to obtain the most relevant text sequence of the current turn.
Here, the candidate related text may include a plurality of candidate text segments having different text sources. Thus, there may be problems between candidate text segments that affect text quality, such as noise, redundancy, and discontinuities. Thus, in order to perform further optimization screening on the retrieved candidate related text, the semantic similarity between the current predicted text block vector and each candidate text segment can be calculated. For example, the cosine similarity can be used as the semantic similarity by calculating the cosine similarity between the current predicted text block vector and each candidate text segment vector. The cosine similarity here can be calculated by the following formula:
In the formula, Representing cosine similarity between the current predicted text block vector u and the candidate text segment vector v; representing the dot product of the two vectors, and u and v representing the euclidean norms of the current predicted text block vector u and the candidate text segment vector v, respectively.
Then, the candidate text segment vector corresponding to the maximum similarity can be selected from the semantic similarity between the current predicted text block vector and each candidate text segment vector as the most relevant text sequence of the current turn. Here, the most relevant text sequence can be obtained by the following formula:
In the formula, Representing the most relevant text sequence; Represent the first Candidate text segment vectors; representing current predicted text block vectors And the firstSemantic similarity between candidate text segment vectors.
According to the method provided by the embodiment of the invention, the current predicted text block vector generated by the sequence generation model is used for carrying out optimization selection on each candidate text segment obtained based on retrieval, the most relevant text sequence of the current round is obtained by matching, redundant information in the candidate relevant texts obtained based on retrieval is removed, the text which is most suitable for the query intention of the user is taken as the most relevant text, and the final prompt word is generated based on the most relevant text of each round, so that the text quality of the prompt word is higher and the query intention of the user is more suitable for the query intention of the user.
Based on any of the above embodiments, in step 110, retrieving the query text to obtain candidate related text includes:
sparse retrieval is carried out based on keywords in the query text, so that a key related text is obtained;
Performing intensive search based on the query text to obtain a semantic related text;
and obtaining the candidate related text based on the key related text and the semantic related text.
Specifically, each keyword in the query text can be indexed through the inverted index to obtain a key related text related to each keyword, so that sparse retrieval is completed. It should be noted that, the sparse search can quickly respond, and accurately match to the background knowledge related to each keyword in the query text, so that the more abundant and diversified background knowledge can be searched. It will be appreciated that the key relevant text obtained through sparse retrieval contains more rich and diversified background knowledge, but may be lacking in semantic understanding of the context.
Thus, the text obtained by decoding the searched text vector can be used as the semantic related text by inputting the query text vector of the query text into the deep learning model and searching the text vector similar to the query text vector through vector similarity. The semantically related text is the background knowledge which is similar to the query text semantically.
Finally, the key related text and the semantic related text can be used as candidate related texts. It should be noted that, by combining two paths of search of sparse search and dense search, the speed and basic relativity of search are ensured, and the richness and diversity of candidate related texts are improved. In addition, accuracy and depth understanding of the search result are improved, so that comprehensive, accurate and efficient search is realized.
In an embodiment, fig. 2 is a second flowchart of the search enhancement generation method based on sequence generation, as shown in fig. 2, first, a query text vector (Query Embedding) and candidate text segment vectors (Context 1 Embedding, context 2 Embedding.) are input into a sequence generation model (Transformer Decoder) to reduce the computation amount of the sequence generation model. In detail, a reply text vector is output through a sequence generation model, and an initialization context sequence of the current round is obtained based on the reply text vector and the query text vector. A current predicted text block vector is then generated based on the initialization context sequence based on the sequence generation model (Predict Embedding). Then, updating the initialization context sequence based on the most relevant text sequence of the current round obtained by matching from each candidate text segment, generating a next predicted text block vector based on the updated initialization context sequence based on a sequence generation model, so as to obtain the most relevant text sequence of the next round from the candidate relevant texts by matching based on the next predicted text block vector until the obtained most relevant text sequence meets a preset condition, and obtaining all the most relevant text sequences, namely all relevant context blocks Embedding in fig. 2. All relevant context blocks Embedding generated are then integrated and output based on the text generation policy, forming the final answer or prompt word sequence. The output text is subjected to proper post-processing such as grammar correction, style adjustment and the like so as to improve the naturalness of the text and the satisfaction of a user.
It should be noted that, the method provided by the embodiment of the invention realizes the rapid positioning of the optimal context of the question in the large-scale candidate related text by combining the latest technology of deep learning and natural language processing, and outputs high-quality prompt words by a high-efficiency sequence generation method, thereby obviously improving the response quality and user satisfaction of the system. The method for generating the prompt word is not only suitable for a question-answering system, but also can be expanded to various dialogue systems, and has wide application prospect.
Based on any of the above embodiments, the training step of the sequence generation model includes:
acquiring the sample query text and a plurality of sample related texts corresponding to the sample query text;
Performing mutual information calculation based on the sample query text and the plurality of sample related texts to obtain optimal upper and lower Wen Pianduan of the sample query text;
inputting the sample query text and the optimal context segment into the initial sequence generation model to obtain a sample prediction text block output by the initial sequence generation model;
And predicting the text block and the optimal context segment based on the sample to obtain a prediction loss, and iterating the initial sequence generation model based on the prediction loss to obtain the sequence generation model.
Specifically, first, a question in a question-answer pair may be taken as a sample query text, and a corresponding context may be taken as a plurality of sample related texts corresponding to the sample query text by collecting the question-answer pair and the corresponding context in different fields such as medical, legal, and scientific. In detail, the method can perform data preprocessing on the acquired question-Answer pairs and the corresponding contexts, can comprise data cleaning to remove low-quality texts and irrelevant text information, and can also comprise standardized format arrangement to ensure that each sample has a complete question-Context Answer format. In addition, sample related text may be subject to a blocking process, and the contextual content of each sample may be partitioned into several smaller, semantically complete text segments using natural language processing tools, such as sentence partitioners. The text fragments obtained by segmentation should keep the whole meaning expression as independent as possible, and the length of each fragment can be adjusted according to the actual needs, and usually, the length does not exceed a few sentences.
And then, carrying out mutual information calculation on the sample query text and each sample related text respectively so as to select a text with higher contribution degree to the answer from a plurality of sample related texts as an optimal context segment. The mutual information calculation here can be calculated by the following formula:
In the formula, Expressed in text containing sample queriesAnd (d)The first sample related textIndividual text segments, and inclusion sample query textIn the case of (1), the probability difference of the expected output answer o, wherein,Representation modelQuerying text at a given sampleAnd (d)The first sample related textGenerating the probability of the answer o in the case of the text segments; Representation model Querying text at a given sampleThe probability of answer o is generated.
Further, the sample query text and the optimal context segment can be input into an initial sequence generation model to obtain a sample prediction text block output by the initial sequence generation model. The initial sequence generation model may be a model only including a decoder, and the sample query text and the optimal context segment may be vectorized in advance based on BERT, GPT, or other natural language processing pre-training models to obtain a sample query text vector and an optimal context segment vector. Thus, the optimal context segment vector can be added after the sample query text vector and input to the initial sequence generation model together, and the embedded vector of the sample prediction text block is output through the initial sequence generation model.
And then, carrying out mean square error calculation through the embedded vector of the sample prediction text block and the optimal context segment vector to obtain the prediction loss. The predictive loss here can be calculated by the following formula:
In the formula, Representing a predicted loss; representing a total number of sample query texts; Represent training set Querying text by using a plurality of samples; Represent the first Predicting the embedded vector of the text block by the samples; Represent the first And searching the optimal context segment vector corresponding to the text by each sample.
Finally, the prediction loss is minimized to train the initial sequence generation model. The model weights can be continuously adjusted by a back propagation algorithm and a selected optimizer (e.g., adam or SGD) to reduce the error between the predicted embedded vector and the true embedded vector, resulting in a predicted accurate sequence generation model.
It should be noted that, in the method provided by the embodiment of the present invention, by training an initial sequence generating model based on Transformer Decoder to generate the embedded vectors (embedding) of the text blocks, the embedded vectors of the generated sample predicted text blocks are as close to the optimal upper and lower Wen Pianduan vectors as possible, which is helpful for constructing an efficient generating model, especially in processing the natural language generating task that needs to be consistent and has rich semantics.
In one embodiment, fig. 3 is a training flow chart of the sequence generation model provided in the present invention, as shown in fig. 3, first, a sample query text vector (Query Embedding) and each relevant text segment vector (Context 1 Embedding, context 2 Embedding.) are input into the sequence generation model (Transformer Decoder), and a sample predicted text block vector (Predict Embedding) is generated by the sequence generation model. And calculating Loss of the optimal context segment with the maximum target answer help degree obtained based on the sample query text through predicting the generated sample prediction text block vector, and training and iterating the initial sequence generation model based on the minimum Loss as a target to obtain a final sequence generation model. In an embodiment, fig. 4 is a flowchart of a method for constructing sample data according to the present invention, as shown in fig. 4, first, text blocks Embedding may be obtained by performing text blocking on sample related text in a training database (Query, context, answer), and vectorizing the text blocks after text blocking. The optimal context sequence may then be determined by conditional mutual information, i.e., the optimal context sequence from the plurality of text blocks Embedding that is most helpful in obtaining the corresponding answer to the query based on the query. Thus, a training corpus (Query, context, best Context, answer) is obtained. Thus, sample query text and sample related text can be selected from the training corpus, and the optimal context segment can be labeled as the most.
Based on any of the foregoing embodiments, the performing mutual information calculation based on the sample query text and the plurality of sample related texts to obtain an optimal context segment of the sample query text includes:
respectively carrying out segmentation processing on the plurality of sample related texts to obtain sample related text fragments of each sample related text;
Performing mutual information calculation based on the sample query text and each sample related text fragment to obtain a contribution value of each sample related text fragment;
selecting preferable upper and lower Wen Pianduan from each sample related text based on the contribution value of each sample related text fragment;
and aggregating the preferable context fragments in the relevant texts of each sample to obtain the optimal context fragments of the sample query text.
The method comprises the steps of firstly, respectively carrying out segmentation processing on a plurality of sample related texts to obtain sample related text fragments of each sample related text, and carrying out mutual information calculation based on a sample query text and each sample related text fragment to obtain a contribution value of each sample related text fragment.
It can be appreciated that the contribution value herein may reflect the contribution degree of the sample related text segment to obtaining the corresponding sample answer based on the sample query text, and may reflect the coincidence degree of the sample related text segment and the query intention of the user. For example, the higher the contribution value of the sample related text segment, the higher the degree of coincidence of the sample related text segment with the query intent of the user may be indicated, whereas the lower the degree of coincidence of the sample related text segment with the query intent of the user is.
Then, the contribution value of each sample related text segment can be selected through a preset threshold value. In detail, for a single sample related text, for each sample related text segment's contribution value, sample related text segments below a preset threshold are culled. It is understood that CXMI scores are calculated for each text segment. A sample related text segment is considered to be significantly helpful in generating an answer only if the score of the sample related text segment is above a threshold. Then, the sample related text segment with the highest score of CXMI in the rest is selected as the preferred context segment of the sample related text. For example, this can be achieved by the following formula:
In the formula, The representation is from the firstThe first of the sample related textsA plurality of sample related text fragments; The representation is from the first Selecting CXMI sample related text fragments with highest scores from the sample related texts。
Likewise, a preferred context segment is selected that yields each sample related text corresponding to the sample query text. And finally, aggregating the preferable context fragments of the relevant texts of the samples corresponding to the sample query text to obtain the optimal context fragments of the sample query text.
It should be noted that, the method provided by the embodiment of the invention not only can systematically evaluate the contribution of each sample related text segment, but also can accurately select the sample related text segment which has a great influence on the performance of the sequence generation model. Therefore, the method has obvious advantages in ensuring the relevance and accuracy of the generated content, especially in the fields of machine translation, automatic question-answering systems and the like which need highly accurate information support.
Based on any of the above embodiments, fig. 5 is a schematic structural diagram of a search enhancement generation device based on sequence generation according to the present invention, as shown in fig. 5, the device includes:
the obtaining unit 510 obtains a query text, and searches the query text to obtain candidate related texts;
An initial prediction unit 520, configured to obtain an initialization context sequence of a current round based on the query text, and generate a current predicted text block vector based on the initialization context sequence and a sequence generation model, so as to obtain a most relevant text sequence of the current round from the candidate relevant texts by matching based on the current predicted text block vector;
A cyclic prediction unit 530, configured to update the initialization context sequence based on the current most relevant text sequence, generate a model based on the updated initialization context sequence and the sequence, and generate a next predicted text block vector, so as to obtain the most relevant text sequence of the next round from the candidate relevant texts by matching based on the next predicted text block vector until the obtained most relevant text sequence meets a preset condition;
And a generating unit 540 for determining a target prompt word of the query text based on the obtained most relevant text sequence to determine an enhanced reply text of the query text based on the target prompt word, wherein the sequence generating model is obtained by training an initial sequence generating model based on a sample query text.
According to the device provided by the embodiment of the invention, the candidate related texts are obtained by searching the query text, so that the diversity of the prompt words is ensured. And then, optimizing the candidate related texts obtained by searching through a sequence generation model, taking the candidate text fragments obtained by searching as basic units of similar words, and optimizing and combining the candidate related texts by utilizing a sequence generation algorithm, so that coherent and high-quality prompt words can be obtained, and further, the performance of downstream tasks for fine tuning or prompt learning based on the prompt words is improved.
Based on any of the above embodiments, the initial prediction unit is specifically configured to:
Generating a model based on the query text, the candidate related text and the sequence, and generating a reply text vector of the query text;
and obtaining an initialization context sequence of the current round based on the reply text vector.
Based on any of the above embodiments, the candidate related text includes a plurality of candidate text segments having different text sources;
the initial prediction unit is specifically configured to:
respectively calculating semantic similarity between the current predicted text block vector and each candidate text segment;
and based on the calculated semantic similarity, matching the candidate text segments to obtain the most relevant text sequence of the current turn.
Based on any of the above embodiments, the obtaining unit is specifically configured to:
sparse retrieval is carried out based on keywords in the query text, so that a key related text is obtained;
Performing intensive search based on the query text to obtain a semantic related text;
and obtaining the candidate related text based on the key related text and the semantic related text.
Based on any of the above embodiments, the apparatus further includes a training unit, where the training unit is specifically configured to:
acquiring the sample query text and a plurality of sample related texts corresponding to the sample query text;
Performing mutual information calculation based on the sample query text and the plurality of sample related texts to obtain optimal upper and lower Wen Pianduan of the sample query text;
inputting the sample query text and the optimal context segment into the initial sequence generation model to obtain a sample prediction text block output by the initial sequence generation model;
And predicting the text block and the optimal context segment based on the sample to obtain a prediction loss, and iterating the initial sequence generation model based on the prediction loss to obtain the sequence generation model.
Based on any of the above embodiments, the training unit is further specifically configured to:
respectively carrying out segmentation processing on the plurality of sample related texts to obtain sample related text fragments of each sample related text;
Performing mutual information calculation based on the sample query text and each sample related text fragment to obtain a contribution value of each sample related text fragment;
selecting preferable upper and lower Wen Pianduan from each sample related text based on the contribution value of each sample related text fragment;
and aggregating the preferable context fragments in the relevant texts of each sample to obtain the optimal context fragments of the sample query text.
Fig. 6 illustrates a physical schematic diagram of an electronic device, which may include a processor 610, a communication interface Communications Interface, a memory 630, and a communication bus 640, as shown in fig. 6, where the processor 610, the communication interface 620, and the memory 630 communicate with each other via the communication bus 640. The processor 610 may invoke logic instructions in the memory 630 to perform a sequence-based search enhancement generation method, which includes obtaining a query text, retrieving the query text to obtain a candidate related text, obtaining an initialization context sequence of a current round based on the query text, generating a model based on the initialization context sequence and the sequence generation model, generating a current predicted text block vector to obtain a most relevant text sequence of the current round based on the current predicted text block vector by matching from the candidate related text, updating the initialization context sequence based on the current most relevant text sequence, generating a model based on the updated initialization context sequence and the sequence generation model, generating a next predicted text block vector to obtain a most relevant text sequence of the next round based on the next predicted text block vector by matching from the candidate related text until the obtained most relevant text sequence meets a preset condition, determining a target prompt word of the query text based on the target prompt word, determining that the query text is enhanced based on the target prompt word, and generating a training word based on the model of the sequence generated by matching from the candidate related text.
Further, the logic instructions in the memory 630 may be implemented in the form of software functional units and stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. The storage medium includes a U disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, an optical disk, or other various media capable of storing program codes.
In another aspect, the invention further provides a computer program product, the computer program product comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, when the computer program is executed by a processor, the computer can execute a search enhancement generation method based on sequence generation provided by the above methods, the method comprises the steps of obtaining a query text, searching the query text to obtain candidate related texts, obtaining an initialization context sequence of a current round based on the query text, generating a model based on the initialization context sequence and the sequence, generating a current prediction text block vector to obtain a most relevant text sequence of the current round based on the current prediction text block vector, updating the initialization context sequence based on the current most relevant text sequence, generating a next prediction text block vector based on the updated initialization context sequence and the sequence, obtaining a most relevant text sequence of the next round based on the next prediction text block vector, generating a training text model based on the preset word, and the most relevant text block vector, generating a training text model based on the most relevant text sequence, and the most relevant text block vector of the next round, generating a training text model based on the candidate text, and the training text block vector, and generating a most relevant text word based on the search text block vector.
In yet another aspect, the present invention further provides a non-transitory computer readable storage medium, on which a computer program is stored, the computer program being implemented when executed by a processor to perform a sequence-based search enhancement generation method provided by the above methods, the method comprising obtaining a query text, searching the query text to obtain candidate related texts, obtaining an initialization context sequence of a current round based on the query text, generating a model based on the initialization context sequence and the sequence, generating a current prediction text block vector to obtain a most relevant text sequence of the current round based on the current prediction text block vector, updating the initialization context sequence based on the current most relevant text sequence, generating a model based on the updated initialization context sequence and the sequence, generating a next prediction text block vector to obtain a most relevant text sequence of the next round based on the next prediction text block vector, matching the candidate related texts until the obtained most relevant text sequence meets a preset condition, generating a training word based on the obtained most relevant text sequence, generating a training word based on the query word, and generating a text enhancement model based on the query word, generating a training word based on the obtained target word.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
It should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention, and not for limiting the same, and although the present invention has been described in detail with reference to the above-mentioned embodiments, it should be understood by those skilled in the art that the technical solution described in the above-mentioned embodiments may be modified or some technical features may be equivalently replaced, and these modifications or substitutions do not make the essence of the corresponding technical solution deviate from the spirit and scope of the technical solution of the embodiments of the present invention.
Claims (10)
1. A retrieval enhancement generation method based on sequence generation, comprising:
acquiring a query text, and searching the query text to obtain candidate related texts;
Obtaining an initialization context sequence of a current round based on the query text, generating a model based on the initialization context sequence and the sequence, and generating a current predicted text block vector so as to obtain a most relevant text sequence of the current round from the candidate relevant texts by matching based on the current predicted text block vector;
Updating the initialization context sequence based on the current most relevant text sequence, generating a model based on the updated initialization context sequence and the sequence, and generating a next predicted text block vector so as to obtain the most relevant text sequence of the next round from the candidate relevant texts in a matching way based on the next predicted text block vector until the obtained most relevant text sequence meets a preset condition;
And determining a target prompt word of the query text based on the obtained most relevant text sequence to determine an enhanced reply text of the query text based on the target prompt word, wherein the sequence generation model is obtained by training an initial sequence generation model based on a sample query text.
2. The method for generating search enhancement based on sequence generation according to claim 1, wherein the obtaining an initialization context sequence of a current round based on the query text comprises:
Generating a model based on the query text, the candidate related text and the sequence, and generating a reply text vector of the query text;
and obtaining an initialization context sequence of the current round based on the reply text vector.
3. The method for generating search enhancement based on sequence generation according to claim 1, wherein the candidate related text includes a plurality of candidate text segments having different text sources;
The matching, based on the current predicted text block vector, the most relevant text sequence of the current turn from the candidate relevant texts comprises the following steps:
respectively calculating semantic similarity between the current predicted text block vector and each candidate text segment;
and based on the calculated semantic similarity, matching the candidate text segments to obtain the most relevant text sequence of the current turn.
4. A method of generating search enhancement based on sequence generation according to any one of claims 1 to 3, wherein the searching the query text to obtain candidate related text includes:
sparse retrieval is carried out based on keywords in the query text, so that a key related text is obtained;
Performing intensive search based on the query text to obtain a semantic related text;
and obtaining the candidate related text based on the key related text and the semantic related text.
5. A sequence generation-based search enhancement generation method according to any one of claims 1 to 3, wherein the training step of the sequence generation model comprises:
acquiring the sample query text and a plurality of sample related texts corresponding to the sample query text;
Performing mutual information calculation based on the sample query text and the plurality of sample related texts to obtain optimal upper and lower Wen Pianduan of the sample query text;
inputting the sample query text and the optimal context segment into the initial sequence generation model to obtain a sample prediction text block output by the initial sequence generation model;
And predicting the text block and the optimal context segment based on the sample to obtain a prediction loss, and iterating the initial sequence generation model based on the prediction loss to obtain the sequence generation model.
6. The method for generating search enhancement based on sequence generation according to claim 5, wherein the performing mutual information calculation based on the sample query text and the plurality of sample related texts to obtain an optimal context segment of the sample query text comprises:
respectively carrying out segmentation processing on the plurality of sample related texts to obtain sample related text fragments of each sample related text;
Performing mutual information calculation based on the sample query text and each sample related text fragment to obtain a contribution value of each sample related text fragment;
selecting preferable upper and lower Wen Pianduan from each sample related text based on the contribution value of each sample related text fragment;
and aggregating the preferable context fragments in the relevant texts of each sample to obtain the optimal context fragments of the sample query text.
7. A retrieval enhancement generation device based on sequence generation, comprising:
the acquisition unit is used for acquiring a query text and searching the query text to obtain candidate related texts;
The initial prediction unit is used for obtaining an initialization context sequence of the current round based on the query text, generating a model based on the initialization context sequence and the sequence, and generating a current prediction text block vector so as to obtain a most relevant text sequence of the current round in a matching way from the candidate relevant texts based on the current prediction text block vector;
the cyclic prediction unit is used for updating the initialization context sequence based on the current most relevant text sequence, generating a model based on the updated initialization context sequence and the sequence, and generating a next predicted text block vector so as to obtain the most relevant text sequence of the next round from the candidate relevant texts in a matching mode based on the next predicted text block vector until the obtained most relevant text sequence meets a preset condition;
The generating unit is used for determining target prompt words of the query text based on the obtained most relevant text sequence so as to determine enhanced reply text of the query text based on the target prompt words, and the sequence generating model is obtained by training an initial sequence generating model based on a sample query text.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the sequence generation based search enhancement generation method of any one of claims 1 to 6 when the computer program is executed.
9. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the sequence generation based search enhancement generation method of any of claims 1 to 6.
10. A computer program product comprising a computer program which, when executed by a processor, implements a retrieval enhancement generation method based on sequence generation as claimed in any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411185369.4A CN119396986B (en) | 2024-08-27 | 2024-08-27 | Search enhancement generation method based on sequence generation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411185369.4A CN119396986B (en) | 2024-08-27 | 2024-08-27 | Search enhancement generation method based on sequence generation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN119396986A true CN119396986A (en) | 2025-02-07 |
| CN119396986B CN119396986B (en) | 2025-12-05 |
Family
ID=94426939
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411185369.4A Active CN119396986B (en) | 2024-08-27 | 2024-08-27 | Search enhancement generation method based on sequence generation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN119396986B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210365500A1 (en) * | 2020-05-19 | 2021-11-25 | Miso Technologies Inc. | System and method for question-based content answering |
| CN117573815A (en) * | 2024-01-17 | 2024-02-20 | 之江实验室 | A retrieval enhancement generation method based on vector similarity matching optimization |
| CN118132677A (en) * | 2024-03-21 | 2024-06-04 | 山东大学 | Image and text retrieval method and system based on cross-modal semantic analysis |
| CN118210908A (en) * | 2024-05-21 | 2024-06-18 | 上海普华科技发展股份有限公司 | Retrieval enhancement method and device, electronic equipment and storage medium |
| CN118394890A (en) * | 2024-06-28 | 2024-07-26 | 宏景科技股份有限公司 | Knowledge retrieval enhancement generation method and system based on large language model |
| KR102694794B1 (en) * | 2024-07-04 | 2024-08-16 | 아이비테크 주식회사 | Question-and-answer system, server and operation method for generating answer to user question based on rag technique |
| CN118520854A (en) * | 2024-06-17 | 2024-08-20 | 腾讯科技（北京）有限公司 | Text generation method, apparatus, computer device, storage medium, and program product |
| CN118520074A (en) * | 2024-07-23 | 2024-08-20 | 浙江省北大信息技术高等研究院 | Real-time retrieval enhancement generation method and device based on industrial brain |
| CN118538372A (en) * | 2024-07-25 | 2024-08-23 | 四川省肿瘤医院 | Multi-mode large model-based intraoperative information extraction method and surgery guidance system |
-
2024
- 2024-08-27 CN CN202411185369.4A patent/CN119396986B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210365500A1 (en) * | 2020-05-19 | 2021-11-25 | Miso Technologies Inc. | System and method for question-based content answering |
| CN117573815A (en) * | 2024-01-17 | 2024-02-20 | 之江实验室 | A retrieval enhancement generation method based on vector similarity matching optimization |
| CN118132677A (en) * | 2024-03-21 | 2024-06-04 | 山东大学 | Image and text retrieval method and system based on cross-modal semantic analysis |
| CN118210908A (en) * | 2024-05-21 | 2024-06-18 | 上海普华科技发展股份有限公司 | Retrieval enhancement method and device, electronic equipment and storage medium |
| CN118520854A (en) * | 2024-06-17 | 2024-08-20 | 腾讯科技（北京）有限公司 | Text generation method, apparatus, computer device, storage medium, and program product |
| CN118394890A (en) * | 2024-06-28 | 2024-07-26 | 宏景科技股份有限公司 | Knowledge retrieval enhancement generation method and system based on large language model |
| KR102694794B1 (en) * | 2024-07-04 | 2024-08-16 | 아이비테크 주식회사 | Question-and-answer system, server and operation method for generating answer to user question based on rag technique |
| CN118520074A (en) * | 2024-07-23 | 2024-08-20 | 浙江省北大信息技术高等研究院 | Real-time retrieval enhancement generation method and device based on industrial brain |
| CN118538372A (en) * | 2024-07-25 | 2024-08-23 | 四川省肿瘤医院 | Multi-mode large model-based intraoperative information extraction method and surgery guidance system |
Non-Patent Citations (3)
| Title |
|---|
| CIFALDI, C , ET AL: "Clinical, Immunological, and Molecular Variability of RAG Deficiency:A Retrospective Analysis of 22 RAG patients", JOURNAL OF CLINICAL IMMUNOLOGY, 27 October 2022 (2022-10-27), pages 130 - 145 * |
| 陈浩泷;陈罕之;韩凯峰;朱光旭;赵奕晨;杜滢: "垂直领域大模型的定制化：理论基础与关键技术", 数据采集与处理, 17 June 2024 (2024-06-17), pages 524 - 546 * |
| 陈雪雯;: "基于子词单元的深度学习摘要生成方法", 计算机应用与软件, no. 03, 12 March 2020 (2020-03-12), pages 208 - 214 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN119396986B (en) | 2025-12-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114580382B (en) | Text correction method and device | |
| CN116719520B (en) | Code generation method and device | |
| CN112287670A (en) | Text error correction method, system, computer device and readable storage medium | |
| CN112800170A (en) | Question matching method and device and question reply method and device | |
| CN109840287A (en) | A kind of cross-module state information retrieval method neural network based and device | |
| CN111666764B (en) | Automatic abstracting method and device based on XLNet | |
| CN114154487A (en) | Text automatic error correction method and device, electronic equipment and storage medium | |
| CN118778941B (en) | Method, device, electronic equipment and storage medium for generating code enhanced by search | |
| WO2024045475A1 (en) | Speech recognition method and apparatus, and device and medium | |
| KR20170122755A (en) | Hammer pair expansion device, a computer program for it, and a question answering system | |
| WO2023175089A1 (en) | Generating output sequences with inline evidence using language model neural networks | |
| CN119322838B (en) | Context-based answer generation methods, devices, computer equipment, and storage media | |
| CN118861244A (en) | A method, device and apparatus for generating an answer | |
| CN120450035A (en) | A method and system for generating mathematical problems based on adaptive multi-level retrieval enhancement | |
| CN113378553B (en) | Text processing method, device, electronic equipment and storage medium | |
| CN117057414B (en) | Text generation-oriented multi-step collaborative prompt learning black box knowledge distillation method and system | |
| KR102703538B1 (en) | System and Method for correcting Context sensitive spelling error using predictive information for subsequent sentences in a real-time chatting environment | |
| CN116187317A (en) | Text generation method, device, device and computer readable medium | |
| CN119396986B (en) | Search enhancement generation method based on sequence generation | |
| CN119740024A (en) | A language model training method, device, storage medium and electronic device | |
| CN113934450A (en) | Method, apparatus, computer device and medium for generating annotation information | |
| KR102541806B1 (en) | Method, system, and computer readable record medium for ranking reformulated query | |
| CN111737422B (en) | Entity linking method and device, electronic equipment and storage medium | |
| CN119621896B (en) | Search enhancement generation improvement method based on Embedding-FineTuning | |
| CN116680390B (en) | Vocabulary association recommendation method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |