CN118093849B - Dialogue method and device, storage medium, terminal and computer program product - Google Patents
Dialogue method and device, storage medium, terminal and computer program product Download PDFInfo
- Publication number
- CN118093849B CN118093849B CN202410525974.5A CN202410525974A CN118093849B CN 118093849 B CN118093849 B CN 118093849B CN 202410525974 A CN202410525974 A CN 202410525974A CN 118093849 B CN118093849 B CN 118093849B
- Authority
- CN
- China
- Prior art keywords
- text
- text block
- block
- target
- candidate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Databases & Information Systems (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Human Computer Interaction (AREA)
- Life Sciences & Earth Sciences (AREA)
- Machine Translation (AREA)
Abstract
A dialogue method and device, a storage medium, a terminal, a computer program product, the method comprising: receiving an input sentence of a user; searching a target text block matched with the input sentence in a target knowledge base, wherein the target knowledge base comprises a plurality of candidate text blocks with sequences and text vectors of each candidate text block; for each target text block, performing similarity calculation on the text vector of the target text block and the text vector of one or more candidate text blocks in the sequence in front to determine the above semantic association text block, and/or performing similarity calculation on the text vector of one or more candidate text blocks in the sequence in rear to determine the following semantic association text block; and generating a final answer according to the target text block, the upper Wen Yuyi associated text block and/or the lower semantic associated text block. By the scheme, the completeness and accuracy of the answer can be improved.
Description
Technical Field
The present invention relates to the field of intelligent dialogue technology, and in particular, to a dialogue method and apparatus, a storage medium, a terminal, and a computer program product.
Background
In the field of intelligent dialogue technology, information in a knowledge base to be retrieved is usually stored in the form of text blocks (or called information blocks or information fragments), and the text blocks are usually obtained by parsing, knowledge extraction and knowledge splitting an original document (for example, a document in a PDF, WORD, HTML, EXCEL format).
The text blocks with smaller units are obtained through information splitting, so that the text blocks can be matched and recalled more efficiently, answers corresponding to user input questions can be obtained, and knowledge or information with the same or similar semantics and the same large section can be split into a plurality of fragmented text blocks in the information splitting process. This may result in the recalled text blocks also being fragmented, incomplete during the intelligent conversation, thereby reducing the quality of the answers generated to the user questions.
Disclosure of Invention
The technical problem solved by the embodiment of the invention is how to improve the integrity and accuracy of answers generated for user input sentences.
In order to solve the above technical problems, an embodiment of the present invention provides a dialogue method, including the following steps: receiving an input sentence of a user; searching a target text block matched with the input sentence in a target knowledge base, wherein the target knowledge base comprises a plurality of candidate text blocks with sequences and text vectors of each candidate text block; for each target text block, performing similarity calculation on the text vector of the target text block and the text vector of one or more candidate text blocks in the sequence in front to determine the above semantic association text block, and/or performing similarity calculation on the text vector of one or more candidate text blocks in the sequence in rear to determine the following semantic association text block; and generating a final answer according to the target text block, the upper Wen Yuyi associated text block and/or the lower semantic associated text block.
Optionally, the calculating the similarity between the text vector of the target text block and the text vector of the one or more candidate text blocks in the preceding order to determine the above semantically associated text block includes: taking the target text block as a text block to be matched, performing similarity calculation on the text vector of the current text block to be matched and the text vector of a candidate text block adjacent to the current text block to be matched, taking the candidate text block adjacent to the current text block to be matched as the current text block to be matched when the calculated similarity is larger than or equal to a first threshold value, and continuously performing similarity calculation on the text vector of the current text block to be matched and the text vector of the candidate text block adjacent to the previous text block until the total calculation times reach the preset times; and taking each candidate text block which is participated in calculation and has the similarity larger than or equal to the first threshold value as the upper Wen Yuyi associated text block.
Optionally, each candidate text block of the target knowledge base has a respective affiliated text unit, and each text unit contains a plurality of candidate text blocks; before each candidate text block that has participated in the calculation and has a similarity greater than or equal to the first threshold is taken as the upper Wen Yuyi associated text block, the method further includes: judging whether the number of candidate text blocks which are participated in calculation and have the similarity larger than or equal to the first threshold value in the text units to which the text blocks to be matched belong is larger than or equal to the preset number or not when the calculated similarity is smaller than the first threshold value; if the result is yes, continuing to sequentially perform similarity calculation on the text vector of the current text block to be matched and the text vector of the candidate text block in the previous adjacent text unit until the total calculated times reach the preset times or until the similarity calculated at present is smaller than the first threshold value and the number of the text blocks which are participated in calculation and have the similarity larger than or equal to the first threshold value in the text unit to which the current text block to be matched belongs is smaller than the preset number.
Optionally, the target knowledge base further includes one or more keywords extracted from each candidate text block, and an attribute of each keyword; the searching the target text block matched with the input sentence in the target knowledge base comprises the following steps: extracting one or more target keywords from the input sentence, and determining the attribute of each target keyword; matching each target keyword and attribute thereof of the input sentence with each keyword and attribute thereof of each candidate text block in the target knowledge base; if the target knowledge base has each keyword and the attribute of at least one candidate text block and contains all target keywords and the attribute of the target keywords of the input sentence, the at least one candidate text block is used as a preliminary screening text block; and determining a target text block matched with the input sentence from the preliminary screening text blocks.
Optionally, extracting one or more target keywords from the input sentence, and determining the attribute of each target keyword includes: performing intent analysis on the input sentence to determine an intent category to which the input sentence belongs, wherein each intent category is provided with a corresponding keyword set to be matched, and the keyword set to be matched comprises at least one candidate keyword and attributes thereof; and carrying out text matching on the input sentence and a keyword set to be matched corresponding to the intention category so as to determine one or more candidate keywords matched with the input sentence and attributes thereof from the keyword set to be matched as the target keywords and the attributes thereof.
Optionally, before searching the target text block matched with the input sentence in the target knowledge base, the method further includes: acquiring a to-be-processed document set, wherein the to-be-processed document set comprises at least one document; identifying the document type of the document set to be processed to determine the document type of each document; for each document, adopting a resource analysis strategy corresponding to the document type to which the document belongs, and carrying out resource type identification on the document to obtain a plurality of resource blocks, wherein each resource block has a resource type to which each resource block belongs; for each resource block, adopting a text extraction strategy corresponding to the resource type of the resource block to extract the text of the resource block; determining a plurality of candidate text blocks based on text extraction results of the resource blocks, and constructing and obtaining the target knowledge base; the document types and the resource analysis strategies have one-to-one correspondence, and the resource types and the text extraction strategies have one-to-one correspondence.
The embodiment of the invention also provides a dialogue device, which comprises: the input sentence receiving module is used for receiving input sentences of a user; searching a target text block matched with the input sentence in a target knowledge base, wherein the target knowledge base comprises a plurality of candidate text blocks with sequences and text vectors of each candidate text block; for each target text block, performing similarity calculation on the text vector of the target text block and the text vector of one or more candidate text blocks in the sequence in front to determine the above semantic association text block, and/or performing similarity calculation on the text vector of one or more candidate text blocks in the sequence in rear to determine the following semantic association text block; and generating a final answer according to the target text block, the upper Wen Yuyi associated text block and/or the lower semantic associated text block.
The embodiment of the invention also provides a storage medium, on which a computer program is stored, which executes the steps of the above-mentioned dialog method.
The embodiment of the invention also provides a terminal, which comprises a memory and a processor, wherein the memory stores a computer program capable of running on the processor, and the processor executes the steps of the dialogue method when running the computer program.
The embodiment of the invention also provides a computer program product, comprising a computer program which, when being executed by a processor, performs the steps of the above-mentioned dialog method.
Compared with the prior art, the technical scheme of the embodiment of the invention has the following beneficial effects:
In practical applications, knowledge blocks or text blocks adjacent to each other in a document (for example, adjacent sentences or paragraphs in a PDF document, or adjacent page contents in a PPT document) generally have relevance of up and down Wen Yuyi, while text blocks for constructing the target knowledge base are obtained by splitting original information of the document, and each text block obtained by splitting is often fragmented, so that context semantic association information in the original document is lost. In this case, if the answer is generated based only on the target text blocks that are matched in the target knowledge base, the answer to the user input sentence will not be sufficiently comprehensive and accurate. To solve the foregoing problem, after matching a target text block in a target knowledge base, the embodiment of the present invention further performs vector similarity analysis on a plurality of candidate text blocks before and/or after the target text block, and uses the text blocks semantically related to the target text block and the text blocks semantically related to the context and/or context to generate a final answer of an input sentence. Therefore, the method and the device are beneficial to improving the completeness and accuracy of the answer and improving the user experience.
Further, compared with the method that the preset matching number is used for carrying out semantic similarity analysis on the target text block and the candidate text blocks of the previous matching number and/or the next matching number so as to determine the text blocks related to the upper text block and the lower text block Wen Yuyi, the method of calculating the semantic similarity sequentially and one by one is adopted in the embodiment, that is, when the calculated similarity is greater than or equal to a first threshold value, the similar text blocks are used as new text blocks to be matched, and semantic similarity calculation is carried out on the new text blocks to be matched and the candidate text blocks adjacent to the new text blocks to be matched. Therefore, the method is beneficial to reducing the total times of invalid computation and the operation cost; and the semantic similarity calculation is carried out sequentially forward, so that the semantic continuity (or continuity) rule of the context is more accordant, and the accuracy of the obtained text blocks related to the context Wen Yuyi can be improved.
Further, for a text unit including a plurality of text blocks, if there are a plurality of consecutive candidate text blocks in the text unit, there is a high probability that there is a context content related to the text unit adjacent to the text unit, so when the semantic similarity between a certain text block (e.g., text block T) and a previous adjacent text block is not high, the embodiment further determines whether there are a plurality of consecutive related text blocks in the text unit (e.g., text unit T) to which the text block T belongs, and if so, continues to search for text blocks with similar semantics in the text unit F adjacent to the text unit T. Thus, the integrity and accuracy of text block recalls can be further improved.
Drawings
FIG. 1 is a flow chart of a dialog method in an embodiment of the present invention;
FIG. 2 is a flow chart of one embodiment of step S12 of FIG. 1;
FIG. 3 is a flow chart of one embodiment of step S13 of FIG. 1;
fig. 4 is a schematic structural diagram of a dialogue device according to an embodiment of the invention.
Detailed Description
In order to make the above objects, features and advantages of the present invention more comprehensible, embodiments accompanied with figures are described in detail below.
Referring to fig. 1, fig. 1 is a flowchart of a dialogue method in an embodiment of the present invention. The method can be applied to a terminal device with a dialogue function (hereinafter referred to as dialogue terminal or dialogue robot), and can be selected from, but not limited to, a computer, a mobile phone, a tablet computer, a smart wearable device (such as a smart watch), a vehicle-mounted terminal device, a question-answering robot and the like.
The method comprises the following steps S11 to S14:
Step S11: receiving an input sentence of a user;
Step S12: searching a target text block matched with the input sentence in a target knowledge base, wherein the target knowledge base comprises a plurality of candidate text blocks with sequences and text vectors of each candidate text block;
Step S13: for each target text block, performing similarity calculation on the text vector of the target text block and the text vector of one or more candidate text blocks in the sequence in front to determine the above semantic association text block, and/or performing similarity calculation on the text vector of one or more candidate text blocks in the sequence in rear to determine the following semantic association text block;
step S14: and generating a final answer according to the target text block, the upper Wen Yuyi associated text block and/or the lower semantic associated text block.
In the implementation of step S11, the input sentence may be a sentence in a voice form or a sentence in a text form. The input sentence may be a question sentence, a statement sentence, an anti-question sentence, an exclamation sentence, or the like.
In the implementation of step S12, the target knowledge base may be constructed by using a plurality of candidate text blocks obtained by splitting in advance, where the plurality of candidate text blocks may be obtained by splitting knowledge based on one or more documents to be processed.
In one embodiment, the target knowledge base may be constructed in the following manner: acquiring a to-be-processed document set, wherein the to-be-processed document set comprises at least one document; identifying the document type of the document set to be processed to determine the document type of each document; for each document, adopting a resource analysis strategy corresponding to the document type to which the document belongs, and carrying out resource type identification on the document to obtain a plurality of resource blocks, wherein each resource block has a resource type to which each resource block belongs; for each resource block, adopting a text extraction strategy corresponding to the resource type of the resource block to extract the text of the resource block; determining a plurality of candidate text blocks based on text extraction results of the resource blocks, and constructing and obtaining the target knowledge base; the document types and the resource analysis strategies have one-to-one correspondence, and the resource types and the text extraction strategies have one-to-one correspondence.
Wherein the document type may be used to indicate different formats of the document, which may also be referred to as document format types. There are often a wide variety of document formats in practical applications, including but not limited to PDF, WORD, EXCEL, PPT, HTML, markdown, plain text, and so on. In addition, even within the same document, different resource types may be included, for example, in the same PDF document, resource types such as, but not limited to, pictures, forms, endorsement text, body text, directories, hyperlinks, etc. may be included.
Wherein determining a plurality of candidate text blocks based on the text extraction results for each resource block may include: and performing operations such as text splitting and merging on text extraction results of each resource block to obtain the plurality of candidate text blocks. For example, text splitting and merging can be performed according to a preset number of characters and/or punctuation marks; as another example, text extraction results of one or more resource blocks belonging to the same page (e.g., the same page of PPT) may be combined into a single candidate text block in units of pages of the original document.
In the prior art, for a to-be-processed document set containing a plurality of documents, manually classifying the formats of the documents in the document set is generally relied on to determine the document type of each document; then, for a document of a certain type, text extraction tools corresponding to the document type are adopted, and text extraction is directly carried out with the whole document as granularity (namely, a strategy of single extraction by adopting the same extraction tool). The method has high labor cost, and the whole document is subjected to one-time text extraction by adopting the same tool, so that specific types of resource information can be omitted, and the text extraction has insufficient integrity and accuracy.
In contrast, in the embodiment of the present invention, a "dual-layer parsing policy" is designed, a one-to-one correspondence is established between a document type and a resource parsing policy, and between a document type and a text extraction policy, and knowledge extraction is performed with different resource information in the document as granularity, specifically: for documents in different formats, adaptively adopting respective corresponding resource analysis strategies to identify resource types so as to obtain a plurality of resource blocks; and for resource blocks belonging to different resource types in the same document, adaptively adopting a corresponding text extraction strategy to extract text contents. Therefore, the user can upload all the documents at one time without separate uploading processing according to types; the coverage degree and accuracy of information extraction can be improved, the problem that valuable information cannot be put into a knowledge base due to improper extraction is avoided, and the accuracy and effectiveness of information recall in the subsequent dialogue application process are improved. Further, the embodiment can flexibly support the lateral expansion of the document type and the resource type.
Referring to fig. 2, fig. 2 is a flowchart of one embodiment of step S12 in fig. 1; in this embodiment, the target knowledge base further includes one or more keywords extracted from each candidate text block, and an attribute of each keyword; in the step S12, searching the target text block matching the input sentence in the target knowledge base may include the following steps S121 to S124.
In step S121, one or more target keywords are extracted from the input sentence, and the attribute of each target keyword is determined.
Wherein the attribute of the keyword is used to describe the semantics or properties (or meaning) of the keyword in the belonging sentence or text block. Each keyword and its combination of attributes can be analogically a "key value pair", where the attribute of the keyword can be a "key" and the keyword can be a "value".
It will be appreciated that in natural language, the same keyword may have different properties in different input sentences or text blocks, and the expressed semantics will also tend to be different. The attributes of a keyword may be related to the intent expressed by the user's input sentence or text block, in particular, if multiple input sentences or multiple text blocks express different intentions, the semantics or attributes of the keyword in each input sentence will typically be different even if the multiple input sentences contain the same keyword.
As one non-limiting example, some input sentence a that interrogates marketing-related data: "what is the sales sum of the first brand of shoes 9 months of 2023? ", a certain input sentence B asking for personal information: "is the birth date of the young 2023, 9? "both input sentences a and B contain the same target keyword" 2023 year 9 month ", but the keyword has an attribute of" sales month "or" sales time "in the input sentence a, and an attribute of" birth year month "or" birth time "in the input sentence B.
In a specific implementation, extracting one or more target keywords from the input sentence, and determining the attribute of each target keyword includes: performing intent analysis on the input sentence to determine an intent category to which the input sentence belongs, wherein each intent category is provided with a corresponding keyword set to be matched, and the keyword set to be matched comprises at least one candidate keyword and attributes thereof; and carrying out text matching on the input sentence and a keyword set to be matched corresponding to the intention category so as to determine one or more candidate keywords matched with the input sentence and attributes thereof from the keyword set to be matched as the target keywords and the attributes thereof.
Wherein the intent analysis may be implemented by existing intent recognition or intent classification algorithms. As a non-limiting example, common intent categories for the intelligent dialog field may be selected from, but are not limited to: query ticket information, query weather conditions, query news events, query merchandise information, query marketing related data, and the like. The keyword set to be matched corresponding to each intention category can be obtained by extracting and combining keywords of a large number of sample input sentences belonging to the intention category.
In step S122, each target keyword and its attribute of the input sentence are matched with each keyword and its attribute of each candidate text block in the target knowledge base.
In step S123, if the target knowledge base has keywords and attributes of at least one candidate text block, including all target keywords and attributes of the input sentence, the at least one candidate text block is used as a preliminary screening text block.
In step S124, a target text block that matches the input sentence is determined from the preliminary screening text blocks.
Further, the step S124 may include: calculating the similarity between the text vector of the input sentence and the text vector of each preliminary screening text block; a target text block that matches the input sentence is determined from one or more top-ranked preliminary text blocks of similarity.
As described above, the same keywords often have different attributes or meanings in sentences or text blocks expressing different intentions, and therefore, in the embodiment of the present invention, the combination of "keywords+attributes" is used to match the input sentences with candidate text blocks in the target knowledge base, and compared with the prior art that text blocks are matched in the knowledge base only through "keywords", the present embodiment is beneficial to improving the matching accuracy, and reduces the number of the preliminary screening text blocks obtained after preliminary screening to a greater extent, thereby improving the efficiency of subsequently screening target text blocks from the preliminary screening text blocks. Furthermore, on the basis of performing primary screening on candidate text blocks of the target knowledge base through the combination of the keywords and the attributes, a vector similarity matching method is adopted for the primary screened text blocks with obviously reduced quantity, and compared with a keyword matching or text matching method, the method is beneficial to improving and matching to obtain target text blocks with the most similar semantics to input sentences.
With continued reference to fig. 1, in step S13, for each target text block, a similarity calculation is performed on the text vector of the target text block with the text vector of the one or more candidate text blocks in the preceding order to determine the above semantic-associated text block and/or with the text vector of the one or more candidate text blocks in the following order to determine the below semantic-associated text block.
It is appreciated that knowledge blocks or text blocks that are adjacent in position in a document (e.g., adjacent sentences or paragraphs) typically have a top-bottom Wen Yuyi relevance. For example, a knowledge base is built for documents in PPT format, since each page of content in a PPT generally corresponds to a single topic, each page of content in the PPT (including text, tables, charts, etc. in that page of content) can be treated as a single candidate text block in the knowledge base. However, a PPT is semantically strongly related to its preceding/following page (or pages), such as a background introduction where the preceding page is the current page, a detailed description where the next page is the current page, etc. However, the text blocks for constructing the knowledge base are obtained by splitting the original information of the document, and each text block obtained by splitting is often fragmented, so that context semantic association information in the original document can be lost. In this case, if an answer is generated based on only the information of the current page (corresponding to the matched target text block) and recalled to the user, the information is not comprehensive in solving the query problem of the user.
In order to avoid the above problem, after matching to a target text block, the embodiment of the present invention further performs vector similarity analysis on one or more candidate text blocks before and/or after the target text block, and determines a text block associated with the upper text block and the lower text block Wen Yuyi, which are used together with the target text block to generate a final answer of the input sentence. Therefore, the method and the device are beneficial to improving the completeness and accuracy of the answer and improving the user experience.
Further, referring to fig. 3, fig. 3 is a flowchart of one embodiment of step S13 in fig. 1; the method of determining the above semantic association text block of the target text block according to this embodiment, in the step S13, performs similarity calculation on the text vector of the target text block and the text vector of one or more candidate text blocks that precede the target text block in order to determine the above semantic association text block, and may specifically include the following steps S131 to S132.
In step S131, the target text block is used as a text block to be matched, similarity calculation is performed on the text vector of the current text block to be matched and the text vector of the candidate text block adjacent to the current text block to be matched, and when the calculated similarity is greater than or equal to the first threshold value, the candidate text block adjacent to the current text block to be matched is used as the current text block to be matched, and similarity calculation is performed on the text vector of the current text block to be matched and the text vector of the candidate text block adjacent to the previous text block until the total calculation times reach the preset times.
In step S132, each candidate text block that has participated in the calculation and has a similarity greater than or equal to the first threshold is taken as the upper Wen Yuyi associated text block.
Compared with the method that the semantic similarity analysis is carried out on the target text block and the candidate text blocks of the previous and/or subsequent matching number through the preset matching number so as to determine the text blocks related to the upper and lower Wen Yuyi, the method of calculating the semantic similarity sequentially one by one is adopted in the embodiment, when the calculated similarity is larger than or equal to a first threshold value, the similar text blocks are used as new text blocks to be matched, and the semantic similarity calculation is carried out on the new text blocks to be matched and the candidate text blocks adjacent to the new text blocks to be matched. Therefore, the method is beneficial to reducing the total times of invalid computation and the operation cost; and the semantic similarity calculation is carried out sequentially forward, so that the semantic continuity (or continuity) rule of the context is more accordant, and the accuracy of the obtained text blocks related to the context Wen Yuyi can be improved.
Further, each candidate text block of the target knowledge base has a respective affiliated text unit, and each text unit contains a plurality of candidate text blocks; prior to performing the step S132, the method may further include: judging whether the number of candidate text blocks which are participated in calculation and have the similarity larger than or equal to the first threshold value in the text units to which the text blocks to be matched belong is larger than or equal to the preset number or not when the calculated similarity is smaller than the first threshold value; if the judgment result is yes, continuing to sequentially calculate the similarity between the text vector of the current text block to be matched and the text vector of the candidate text block in the previous adjacent text unit until the condition one is satisfied: calculating the total times to reach the preset times or meeting the second condition: the similarity calculated at present is smaller than the first threshold value, and the number of text blocks which are participating in calculation and have the similarity larger than or equal to the first threshold value in the text units to which the text blocks to be matched at present belong is smaller than the preset number.
It should be noted that, the above condition two is for whether to continue to enter the previous adjacent text unit for similarity calculation, specifically, when the condition two is satisfied, the text vector of the current text block to be matched is not further for similarity calculation with the candidate text block in the previous adjacent text unit (i.e. the previous adjacent text unit of the text unit to which the current text block to be matched belongs), but the above step S132 is directly executed to determine the above semantic association text block of the target text block.
The step of sequentially performing similarity calculation on the current text block to be matched and the candidate text block in the previous adjacent text unit may specifically include: and performing similarity calculation on the text vector of the current text block to be matched and the text vector of the candidate text block which is sequentially last in the previous adjacent text unit, taking the candidate text block as the current text block to be matched if the calculated similarity is greater than or equal to the first threshold value, and continuously performing similarity calculation on the text vector of the candidate text block to be matched and the text vector of the previous adjacent candidate text block. The similarity calculation method may be performed with reference to the content of step S131.
It can be understood that, for a text unit containing multiple text blocks, if the semantics of multiple continuous candidate text blocks in the text unit are similar, then there is a greater probability that there is a context content related to the semantics in text units adjacent to the text unit, so when a situation that the semantic similarity between a certain text block T and a previous adjacent text block is not high is encountered, it is further determined whether the text unit T to which the text block T belongs has multiple continuous related text blocks, and if so, the text unit F adjacent to the text unit T is continuously searched for text blocks with similar semantics. Thus, the integrity and accuracy of text block recalls can be further improved.
The method of how to determine the text blocks of the target text block that are semantically associated with the text blocks of the context is performed with reference to the specific method of determining the text blocks of the context described above, and will not be described in detail herein.
With continued reference to fig. 1, in step S14, a final answer is generated from the target text block, the upper Wen Yuyi-associated text block, and/or the lower semantic-associated text block.
In specific implementation, the target text block, the upper Wen Yuyi associated text block and/or the lower semantic associated text block can be spliced according to the original sequence in the respective knowledge base, and the splicing result is used as a final answer; or, the final answer may be generated after performing operations such as deduplication, format conversion, and templating (e.g., templating using a prompt template (template) tool) on the target text block, the upper Wen Yuyi associated text block, and/or the lower semantic associated text block.
Further, if it is confirmed that the matching target text block is not found in the target knowledge base, the process may be shifted to manual processing, or an answer template corresponding to the type of intention to which the input sentence belongs may be adopted as a final answer.
Further, after step S11 and before step S12, risk verification may also be performed on the input sentence of the user (especially for a conversation robot set in a public place), for example, it may be determined whether the input sentence contains a sensitive word in a preset sensitive word set, and if the input sentence contains a sensitive word, the question-answer flow may be ended, or a preset template answer may be output to the user.
Fig. 4 is a schematic structural diagram of a dialogue device according to an embodiment of the present invention, where the dialogue device may include the following modules:
an input sentence receiving module 41 for receiving an input sentence of a user;
A searching module 42, configured to search a target knowledge base for target text blocks that match the input sentence, where the target knowledge base includes a plurality of candidate text blocks having a sequence, and a text vector of each candidate text block;
a relevance analysis module 43, configured to perform similarity calculation on the text vector of each target text block and the text vector of one or more candidate text blocks in the order before to determine the above semantic associated text block, and/or perform similarity calculation on the text vector of one or more candidate text blocks in the order after to determine the following semantic associated text block;
An answer generation module 44 for generating a final answer based on the target text block, the upper Wen Yuyi associated text block, and/or the lower semantic associated text block.
Regarding the principle, implementation and advantageous effects of the dialogue device, please refer to the related description of the dialogue method shown in the foregoing and any embodiment of fig. 1 to 3, which is not repeated herein.
The embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, performs the steps of the dialog method shown in any of the above-described embodiments of fig. 1 to 3. The computer readable storage medium may include non-volatile memory (non-volatile) or non-transitory memory, and may also include optical disks, mechanical hard disks, solid state disks, and the like.
Specifically, in the embodiment of the present invention, the processor may be a central processing unit (central processing unit, abbreviated as CPU), which may also be other general purpose processors, digital signal processors (DIGITAL SIGNAL processors, abbreviated as DSP), application Specific Integrated Circuits (ASIC), off-the-shelf programmable gate arrays (field programmable GATE ARRAY, abbreviated as FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
It should also be appreciated that the memory in embodiments of the present application may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile memory may be a read-only memory (ROM), a Programmable ROM (PROM), an erasable programmable ROM (erasable PROM EPROM), an electrically erasable programmable ROM (ELECTRICALLY EPROM, EEPROM), or a flash memory. The volatile memory may be a random access memory (random access memory, RAM for short) which acts as an external cache. By way of example, and not limitation, many forms of random access memory (random access memory, RAM) are available, such as static random access memory (STATIC RAM, SRAM), dynamic Random Access Memory (DRAM), synchronous Dynamic Random Access Memory (SDRAM), double data rate synchronous dynamic random access memory (double DATA RATE SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (ENHANCED SDRAM, ESDRAM), synchronous link dynamic random access memory (SYNCHLINK DRAM, SLDRAM), and direct memory bus random access memory (direct rambus RAM, DR RAM).
The embodiment of the invention also provides a terminal, which comprises a memory and a processor, wherein the memory stores a computer program capable of running on the processor, and the processor executes the steps of the dialogue method shown in any embodiment of the figures 1 to 3 when running the computer program.
The embodiments of the present invention also provide a computer program product comprising a computer program which, when run by a processor, performs the steps of the method of dialog shown in any of the embodiments of figures 1 to 3 described above.
It should be understood that the term "and/or" is merely an association relationship describing the associated object, and means that three relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist together, and B exists alone. In this context, the character "/" indicates that the front and rear associated objects are an "or" relationship.
The term "plurality" as used in the embodiments of the present application means two or more.
The first, second, etc. descriptions in the embodiments of the present application are only used for illustrating and distinguishing the description objects, and no order is used, nor is the number of the devices in the embodiments of the present application limited, and no limitation on the embodiments of the present application should be construed.
It should be noted that the serial numbers of the steps in the present embodiment do not represent a limitation on the execution sequence of the steps.
Although the present invention is disclosed above, the present invention is not limited thereto. Various changes and modifications may be made by one skilled in the art without departing from the spirit and scope of the invention, and the scope of the invention should be assessed accordingly to that of the appended claims.
Claims (8)
1. A method of dialog, comprising:
receiving an input sentence of a user;
Searching a target text block matched with the input sentence in a target knowledge base, wherein the target knowledge base comprises a plurality of candidate text blocks with sequences and text vectors of each candidate text block;
For each target text block, performing similarity calculation on the text vector of the target text block and the text vector of one or more candidate text blocks in the sequence in front to determine the above semantic association text block, and/or performing similarity calculation on the text vector of one or more candidate text blocks in the sequence in rear to determine the following semantic association text block;
generating a final answer according to the target text block, the upper Wen Yuyi associated text block and/or the lower semantic associated text block;
Wherein the calculating the similarity between the text vector of the target text block and the text vector of one or more candidate text blocks in the preceding order to determine the above semantically associated text block includes:
Taking the target text block as a text block to be matched, performing similarity calculation on the text vector of the current text block to be matched and the text vector of a candidate text block adjacent to the current text block to be matched, taking the candidate text block adjacent to the current text block to be matched as the current text block to be matched when the calculated similarity is larger than or equal to a first threshold value, and continuously performing similarity calculation on the text vector of the current text block to be matched and the text vector of the candidate text block adjacent to the previous text block until the total calculation times reach the preset times;
each candidate text block which is participated in calculation and has the similarity larger than or equal to the first threshold value is used as the upper Wen Yuyi associated text block;
each candidate text block of the target knowledge base is provided with a text unit to which each candidate text block belongs, and each text unit comprises a plurality of candidate text blocks;
Before each candidate text block that has participated in the calculation and has a similarity greater than or equal to the first threshold is taken as the upper Wen Yuyi associated text block, the method further includes:
Judging whether the number of candidate text blocks which are participated in calculation and have the similarity larger than or equal to the first threshold value in the text units to which the text blocks to be matched belong is larger than or equal to the preset number or not when the calculated similarity is smaller than the first threshold value;
If the result is yes, continuing to sequentially perform similarity calculation on the text vector of the current text block to be matched and the text vector of the candidate text block in the previous adjacent text unit until the total calculated times reach the preset times or until the similarity calculated at present is smaller than the first threshold value and the number of the text blocks which are participated in calculation and have the similarity larger than or equal to the first threshold value in the text unit to which the current text block to be matched belongs is smaller than the preset number.
2. The method of claim 1, wherein the target knowledge base further comprises one or more keywords extracted from each candidate text block, and attributes of each keyword;
The searching the target text block matched with the input sentence in the target knowledge base comprises the following steps:
Extracting one or more target keywords from the input sentence, and determining the attribute of each target keyword;
Matching each target keyword and attribute thereof of the input sentence with each keyword and attribute thereof of each candidate text block in the target knowledge base;
If all keywords and attributes of at least one candidate text block exist in the target knowledge base and are completely matched with all target keywords and attributes of the target keywords of the input sentence, the at least one candidate text block is used as a preliminary screening text block;
And determining a target text block matched with the input sentence from the preliminary screening text blocks.
3. The method of claim 2, wherein extracting one or more target keywords from the input sentence and determining attributes of each target keyword comprises:
performing intent analysis on the input sentence to determine an intent category to which the input sentence belongs, wherein each intent category is provided with a corresponding keyword set to be matched, and the keyword set to be matched comprises at least one candidate keyword and attributes thereof;
and carrying out text matching on the input sentence and a keyword set to be matched corresponding to the intention category so as to determine one or more candidate keywords matched with the input sentence and attributes thereof from the keyword set to be matched as the target keywords and the attributes thereof.
4. The method of claim 1, wherein prior to locating in a target knowledge base a target text block that matches the input sentence, the method further comprises:
Acquiring a to-be-processed document set, wherein the to-be-processed document set comprises at least one document;
identifying the document type of the document set to be processed to determine the document type of each document;
for each document, adopting a resource analysis strategy corresponding to the document type to which the document belongs, and carrying out resource type identification on the document to obtain a plurality of resource blocks, wherein each resource block has a resource type to which each resource block belongs;
For each resource block, adopting a text extraction strategy corresponding to the resource type of the resource block to extract the text of the resource block;
determining a plurality of candidate text blocks based on text extraction results of the resource blocks, and constructing and obtaining the target knowledge base;
The document types and the resource analysis strategies have one-to-one correspondence, and the resource types and the text extraction strategies have one-to-one correspondence.
5. A dialog device, comprising:
the input sentence receiving module is used for receiving input sentences of a user;
The searching module is used for searching a target text block matched with the input sentence in a target knowledge base, wherein the target knowledge base comprises a plurality of candidate text blocks with sequences and text vectors of each candidate text block;
The relevance analysis module is used for carrying out similarity calculation on the text vector of each target text block and the text vector of one or more candidate text blocks in the sequence before to determine the text blocks related to the semantic above, and/or carrying out similarity calculation on the text vector of one or more candidate text blocks in the sequence after to determine the text blocks related to the semantic below;
the answer generation module is used for generating a final answer according to the target text block, the upper Wen Yuyi associated text block and/or the lower semantic associated text block;
Wherein the relevance analysis module further performs:
Taking the target text block as a text block to be matched, performing similarity calculation on the text vector of the current text block to be matched and the text vector of a candidate text block adjacent to the current text block to be matched, taking the candidate text block adjacent to the current text block to be matched as the current text block to be matched when the calculated similarity is larger than or equal to a first threshold value, and continuously performing similarity calculation on the text vector of the current text block to be matched and the text vector of the candidate text block adjacent to the previous text block until the total calculation times reach the preset times;
each candidate text block which is participated in calculation and has the similarity larger than or equal to the first threshold value is used as the upper Wen Yuyi associated text block;
each candidate text block of the target knowledge base is provided with a text unit to which each candidate text block belongs, and each text unit comprises a plurality of candidate text blocks;
Before each candidate text block that has participated in the calculation and has a similarity greater than or equal to the first threshold is taken as the upper Wen Yuyi associated text block, the dialogue device further performs:
Judging whether the number of candidate text blocks which are participated in calculation and have the similarity larger than or equal to the first threshold value in the text units to which the text blocks to be matched belong is larger than or equal to the preset number or not when the calculated similarity is smaller than the first threshold value;
If the result is yes, continuing to sequentially perform similarity calculation on the text vector of the current text block to be matched and the text vector of the candidate text block in the previous adjacent text unit until the total calculated times reach the preset times or until the similarity calculated at present is smaller than the first threshold value and the number of the text blocks which are participated in calculation and have the similarity larger than or equal to the first threshold value in the text unit to which the current text block to be matched belongs is smaller than the preset number.
6. A storage medium having stored thereon a computer program, which when executed by a processor performs the steps of the dialog method of any of claims 1 to 4.
7. A terminal comprising a memory and a processor, said memory having stored thereon a computer program capable of being run on said processor, characterized in that said processor executes the steps of the dialog method of any of claims 1 to 4 when said computer program is run.
8. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, performs the steps of the dialog method as claimed in any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410525974.5A CN118093849B (en) | 2024-04-26 | 2024-04-26 | Dialogue method and device, storage medium, terminal and computer program product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410525974.5A CN118093849B (en) | 2024-04-26 | 2024-04-26 | Dialogue method and device, storage medium, terminal and computer program product |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118093849A CN118093849A (en) | 2024-05-28 |
| CN118093849B true CN118093849B (en) | 2024-08-06 |
Family
ID=91151089
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410525974.5A Active CN118093849B (en) | 2024-04-26 | 2024-04-26 | Dialogue method and device, storage medium, terminal and computer program product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118093849B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116308758A (en) * | 2023-03-20 | 2023-06-23 | 深圳征信服务有限公司 | Financial risk analysis method and system based on big data |
| CN116756298A (en) * | 2023-08-18 | 2023-09-15 | 太仓市律点信息技术有限公司 | Cloud database-oriented AI session information optimization method and big data optimization server |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5680479A (en) * | 1992-04-24 | 1997-10-21 | Canon Kabushiki Kaisha | Method and apparatus for character recognition |
| CN112733545A (en) * | 2020-12-28 | 2021-04-30 | 中电金信软件有限公司 | Text blocking method and device, computer equipment and storage medium |
| CN114863435A (en) * | 2022-05-26 | 2022-08-05 | 北京金山数字娱乐科技有限公司 | Text extraction method and device |
| CN114936276A (en) * | 2022-06-07 | 2022-08-23 | 来也科技(北京)有限公司 | Answer generation method, device, electronic device and storage medium |
| CN117609477B (en) * | 2024-01-22 | 2024-05-07 | 亚信科技（中国）有限公司 | Large model question-answering method and device based on domain knowledge |
-
2024
- 2024-04-26 CN CN202410525974.5A patent/CN118093849B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116308758A (en) * | 2023-03-20 | 2023-06-23 | 深圳征信服务有限公司 | Financial risk analysis method and system based on big data |
| CN116756298A (en) * | 2023-08-18 | 2023-09-15 | 太仓市律点信息技术有限公司 | Cloud database-oriented AI session information optimization method and big data optimization server |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118093849A (en) | 2024-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111859960B (en) | Semantic matching method, device, computer equipment and medium based on knowledge distillation | |
| CA3174601C (en) | Text intent identifying method, device, computer equipment and storage medium | |
| CN118132731B (en) | Dialogue method and device, storage medium, terminal and computer program product | |
| WO2020077896A1 (en) | Method and apparatus for generating question data, computer device, and storage medium | |
| CN111291177B (en) | Information processing method, device and computer storage medium | |
| US8271502B2 (en) | Presenting multiple document summarization with search results | |
| CN111881307A (en) | Demonstration manuscript generation method and device, computer equipment and storage medium | |
| US20200257860A1 (en) | Semantic recognition method, electronic device, and computer-readable storage medium | |
| CN106649612B (en) | Method and device for automatically matching question and answer templates | |
| CN110321537B (en) | Method and device for generating file | |
| CN111767737A (en) | Text intent similarity determination method, apparatus, electronic device and storage medium | |
| CN118296120A (en) | Large-scale language model retrieval enhancement generation method for multi-mode multi-scale multi-channel recall | |
| US11379527B2 (en) | Sibling search queries | |
| CN114138936B (en) | A method, device, electronic device and storage medium for generating text summary | |
| CN114090776A (en) | Document analysis method, system and device | |
| CN111324713A (en) | Automatic replying method and device for conversation, storage medium and computer equipment | |
| CN113704623A (en) | Data recommendation method, device, equipment and storage medium | |
| WO2020133186A1 (en) | Document information extraction method, storage medium, and terminal | |
| CN109086386B (en) | Data processing method, device, computer equipment and storage medium | |
| CN115081457A (en) | Information processing method and system based on artificial intelligence technology | |
| CN118093849B (en) | Dialogue method and device, storage medium, terminal and computer program product | |
| CN113946668A (en) | Semantic processing method, system and device based on edge node and storage medium | |
| CN118536473A (en) | Manual intelligent technology-based tagboard review method, computer device, medium and program product | |
| US11042520B2 (en) | Computer system | |
| CN109344388A (en) | Spam comment identification method and device and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |