CN115129834B

CN115129834B - Question and answer method and system

Info

Publication number: CN115129834B
Application number: CN202210549895.9A
Authority: CN
Inventors: 郑杰文; 陈泽
Original assignee: Netease Hangzhou Network Co Ltd
Current assignee: Netease Hangzhou Network Co Ltd
Priority date: 2022-05-20
Filing date: 2022-05-20
Publication date: 2025-09-16
Anticipated expiration: 2042-05-20
Also published as: CN115129834A

Abstract

The present invention discloses a question-answering method, comprising: identifying entities in an input question and obtaining a list of entity pairs; obtaining alternative relationships for each entity pair in the entity pair list to form an alternative relationship set corresponding to the entity pair list; obtaining a probability distribution of the input question on the alternative relationship set based on the input question and the alternative relationship set; obtaining a probability distribution of known questions containing any entity pair in the entity pair list on the alternative relationship set; calculating the similarity of the probability distributions of the input question and the known questions on the alternative relationship set, and determining an output answer corresponding to the input question based on the similarity of the probability distributions. This method is a question-answering method based on probability distribution and probability distribution similarity, which solves the technical problem in the prior art of being unable to answer multi-entity questions and missing relationship questions, and improves the comprehensiveness of question-answering technology.

Description

Question and answer method and system

Technical Field

The invention relates to the technical field of artificial intelligence, in particular to a question answering method, a question answering system, electronic equipment and a computer readable storage medium.

Background

A Question and answer system (Question ANSWERING SYSTEM, QA) is a high-level form of information retrieval system that can answer questions posed by users in natural language in more accurate and concise natural language with the rapid development of artificial intelligence technology. The question and answer system is widely applied to various fields such as game fields, navigation technical fields, online teaching fields and the like.

The knowledge graph question-answering system (Knowledge Graph Based Question Answering, KGQA) is a current mainstream question-answering system, and the question-answering system based on the Knowledge Graph (KG) has the following problems in the practical application process that one question can only answer a question containing one entity, and the question cannot be answered for multiple entities. Secondly, only the questions of the relationship of the included entities in the knowledge graph can be answered, and the questions of the missing relationship cannot be answered.

Disclosure of Invention

The invention provides a question and answer method and a question and answer system, which are used for solving the technical problem that the existing question and answer method cannot answer the multi-entity problem and the missing relation problem.

The embodiment of the invention provides a question and answer method, which comprises the following steps:

Identifying an entity in the input problem, and acquiring an entity pair list;

acquiring an alternative relation of each entity pair in the entity pair list, and forming an alternative relation set corresponding to the entity pair list;

acquiring probability distribution of the input problem on the alternative relation set according to the input problem and the alternative relation set;

Acquiring probability distribution of known problems including any one entity pair in the entity pair list on the alternative relationship set;

And calculating the similarity of probability distribution of the input problem and the known problem on the alternative relation set, and determining an output answer corresponding to the input problem according to the similarity of probability distribution.

Optionally, the identifying the entity in the input problem, and acquiring the entity pair list includes:

identifying an entity in the input problem by an entity identification model;

and carrying out pairwise team formation on the entities in the input problem to form the entity pair list.

Optionally, the method for obtaining the entity identification model includes:

Taking the entity existing in the knowledge graph as training data, providing the training data for an initial entity identification model, and training the initial entity identification model;

and taking the entity recognition model which reaches the preset standard after training as the entity recognition model which is actually used.

Optionally, the obtaining the candidate relationship of each entity pair in the entity pair list includes taking the relationship of the entity pair existing in the knowledge graph as the candidate relationship.

Optionally, the obtaining the candidate relationship of each entity pair in the entity pair list further includes taking the relationship existing in the knowledge graph of the type pair corresponding to the entity pair as the candidate relationship.

Optionally, the obtaining, according to the input problem and the candidate relationship set, a probability distribution of the input problem on the candidate relationship set includes:

Obtaining a score of the input problem on each alternative relation in the alternative relation set through a relation extraction model;

And forming a probability distribution of the input problem on the alternative relation set according to the score of the input problem on each alternative relation in the alternative relation set.

Optionally, the method for obtaining the relation extraction model includes:

The method comprises the steps of providing a training data of known questions in a question-answer library and corresponding relation descriptions of entities in the known questions to an initial relation extraction model, and training the initial relation extraction model;

and taking the relation extraction model which reaches the preset standard after training as the relation extraction model which is actually used.

Optionally, the obtaining, by a relationship extraction model, a score of the input problem on each candidate relationship in the candidate relationship set includes:

mapping each alternative relationship in the alternative relationship set into a relationship description;

Acquiring correct probability of the input problem on the relation description of each alternative relation through the relation extraction model;

The probability of correctness of the input question on the relation description of each alternative relation is taken as the score of the input question on each alternative relation.

Optionally, mapping each candidate relationship in the candidate relationship set to a relationship description includes correspondingly acquiring a relationship description of each candidate relationship in the candidate relationship set according to a relationship and relationship description list.

Optionally, the obtaining a probability distribution of the known problem including any one entity pair in the entity pair list on the alternative relationship set includes:

acquiring all known questions comprising any one entity pair in the entity pair list from a question-answer library;

And obtaining probability distribution of each known problem on the alternative relation set through a relation extraction model.

Optionally, the calculating the similarity degree of the probability distribution of the input question and the known question on the alternative relationship set, and determining the output answer corresponding to the input question according to the similarity degree includes:

calculating JS divergence values of probability distribution of the input problem on the alternative relation set and probability distribution of the known problem on the alternative relation set;

Judging the similarity between the input problem and the known problem according to the JS divergence value;

And determining an output answer corresponding to the input question according to the similarity of the input question and the known question.

Optionally, the determining the similarity between the input problem and the known problem according to the JS divergence value includes:

comparing whether the JS divergence value is smaller than a preset similarity threshold value;

And judging whether the input problem is similar to the known problem according to the comparison result, wherein if so, the input problem is similar to the known problem, and if not, the input problem is dissimilar to the known problem.

Optionally, the determining the similarity between the input problem and the known problem according to the JS divergence value further includes:

Comparing the magnitudes of a plurality of JS divergence values;

And judging the known problem with the highest similarity with the input problem according to the comparison result, wherein the known problem comprises that the smaller the JS divergence value is, the higher the similarity between the input problem and the known problem is.

Optionally, determining the output answer corresponding to the input question according to the similarity between the input question and the known question comprises taking the answer of the known question with the highest similarity to the input question as the output answer corresponding to the input question.

The embodiment of the invention also provides a question and answer system, which comprises an entity identification unit, an alternative relation acquisition unit, an input problem probability distribution acquisition unit, a known problem probability distribution acquisition unit and a similarity calculation unit;

The entity identification unit is used for identifying the entity in the input problem and acquiring an entity pair list;

The candidate relationship obtaining unit is configured to obtain a candidate relationship of each entity pair in the entity pair list, and form a candidate relationship set corresponding to the entity pair list;

the input problem probability distribution acquisition unit is used for acquiring probability distribution of the input problem on the alternative relation set according to the input problem and the alternative relation set;

The known problem probability distribution obtaining unit is configured to obtain a probability distribution of a known problem including any one entity pair in the entity pair list on the alternative relationship set;

The similarity calculation unit is used for calculating the similarity of the probability distribution of the input problem and the known problem on the alternative relation set, and determining an output answer corresponding to the input problem according to the similarity of the probability distribution.

The embodiment of the invention also provides electronic equipment, which comprises a memory and a processor;

the memory is used for storing one or more computer instructions;

The processor is configured to execute the one or more computer instructions to implement the method described above.

Embodiments of the present invention also provide a computer-readable storage medium having stored thereon one or more computer instructions which, when executed by a processor, perform the above-described method.

Compared with the prior art, the question-answering method comprises the steps of identifying an entity in an input question, obtaining an entity pair list, obtaining an alternative relation of each entity pair in the entity pair list to form an alternative relation set corresponding to the entity pair list, obtaining probability distribution of the input question on the alternative relation set according to the input question and the alternative relation set, obtaining probability distribution of a known question containing any entity pair in the entity pair list on the alternative relation set, calculating similarity of the input question and the probability distribution of the known question on the alternative relation set, and determining output answer corresponding to the input question according to the similarity of the probability distribution. According to the method, a probability distribution of the input questions on the alternative relation set and a probability distribution of the known questions on the alternative relation set are obtained through obtaining the entity pair list of the input questions and the alternative relation set corresponding to the entity pair list, the similarity between the probability distribution of the input questions and the probability distribution of the known questions is further calculated, and output answers of the input questions and answers are determined according to the similarity. The question-answering method provided by the invention is a question-answering method based on probability distribution and probability distribution similarity calculation, and can acquire output answers corresponding to input questions according to the similarity of the input questions and probability distribution of known questions on an alternative relation set. The technical problem that the multi-entity problem and the missing relation problem cannot be solved in the prior art is solved, and the comprehensiveness of the question-answering technology is improved.

Drawings

FIG. 1 is an application system diagram of a question-answering method provided by an embodiment of the present invention;

FIG. 2 is a diagram of an application system of another question-answering method according to an embodiment of the present invention;

FIG. 3 is a flow chart of a question and answer method provided by a first embodiment of the invention;

FIG. 4 is a flowchart of a probability distribution acquisition method provided by a first embodiment of the present invention;

FIG. 5 is a flow chart for obtaining probability distribution of known problems over a set of alternative relationships provided by a first embodiment of the invention;

FIG. 6 is a flow chart of determining an output reply according to probability distribution similarity provided by the first embodiment of the invention;

Fig. 7 is a schematic structural diagram of a question-answering system according to a second embodiment of the present invention;

fig. 8 is a schematic structural diagram of an electronic device according to a third embodiment of the present invention.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. The present invention may be embodied in many other forms than those herein described, and those skilled in the art will readily appreciate that the present invention may be similarly embodied without departing from the spirit or essential characteristics thereof, and therefore the present invention is not limited to the specific embodiments disclosed below.

The following describes terms involved in the embodiments of the present invention:

The question and answer refers to a technical process of analyzing and answering a question which is presented by a user in natural language by utilizing an artificial intelligence technology and outputting the answer of the question in natural language.

The Question answering system (Question ANSWERING SYSTEM, QA) is a system applied to Question answering, is a high-level form of an information retrieval system, and can answer questions in natural language accurately and simply for users. The method is widely applied to various fields, for example, in the field of games, the familiarity degree of different players with games is different, so that a plurality of questions about game playing methods can be generated in the process that the players are familiar with the games, and the players can quickly and accurately acquire answers to the questions through a question-answering system, so that the method has important significance for improving the game retention.

The Knowledge Graph (KG) is a form of Knowledge characterization, is a network formed by connecting Knowledge, is a Graph for describing the Knowledge development process and the structural relationship by using a visualization technology, generally stores the Knowledge in triples, and models the interrelationship between entities according to a head entity-relationship-tail entity mode, wherein one triplet represents one Knowledge.

The knowledge graph question answering system (Knowledge Graph Based Question Answering, KGQA) is a question answering system for answering user questions based on structured information of the knowledge graph, and when a user inputs a question, relevant information of the question can be obtained and answered in the knowledge graph through an inference method.

The artificial intelligence (ARTIFICIAL INTELLIGENCE, AI), a branch of computer science, can create intelligent machines that are close to human intelligence. The research in the field mainly relates to the technologies of robots, language recognition, image recognition and the like. Artificial intelligence is an artificial intelligent robot that analyzes and simulates the process of consciousness and thinking of a person to design a person-like behavior. In a single intelligence, such as computing, it is even possible to override human intelligence.

The question-answering system based on the knowledge graph is an existing mainstream question-answering method, and utilizes the structured information of the knowledge graph to infer and acquire related information and answers of the questions. The knowledge graph-based question-answering method generally comprises the following steps:

first, entity recognition, namely, recognizing an entity from an input problem, and judging the type data of the problem in a knowledge graph.

And secondly, identifying the relationship, namely identifying the candidate relationship of the entity based on the structured information of the knowledge graph, sequencing the candidate relationship, and selecting the relationship with the highest possibility.

Thirdly, recall the subgraph, and output all subgraphs of the entity and the relationship according to the identified entity and the identified relationship.

Fourth, the answers are ranked, ranking is carried out according to the candidate scores of all the subgraphs, the subgraph with the highest score is selected, and the answer corresponding to the subgraph is used as a answer to the question.

Therefore, the question-answering method based on the knowledge graph is actually aimed at the input problem, in the triplet (head entity s-relation r-tail entity o) of the knowledge graph, the head entity s is predicted first, then the relation r is predicted, if the head entity s and the relation r exist in the knowledge graph, then the tail entity o is predicted according to the subgraph output by the head entity s and the relation r, and the predicted result is used as the answer of the problem.

The question answering method based on the knowledge graph can better predict answers to questions, but has two questions:

Firstly, the existing knowledge-graph-based question answering method can only process the situation that one entity appears in the questions, and cannot predict the multi-entity questions.

Secondly, the existing question-answering method based on the knowledge graph only can process the questions of the entity with the relation in the knowledge graph, and the answers cannot be inferred for the questions with the relation missing.

However, in the actual situation, a plurality of entities often appear in a question, and as the knowledge graph is gradually perfected, there is a situation that the relationship between the entities is lost, for which, the question-answering system automatically replies that "your question really cheerio, i can go back to a good study, and can not bring effective information to the user, and can not solve the problem of the user. Particularly in the game field, when a new game is pushed out, a game player can put forward more problems aiming at the new game, and if a plurality of entities or a relation of the entities is absent in the problems, the existing question-answering method can not answer the problems put forward by the game player effectively, so that the understanding of the new game by the game player is not facilitated, and the loss of the game player is caused.

Aiming at the problems existing in the existing question-answering method, the application provides a question-answering method, which takes probability distribution of the problems in a relation space as a starting point, calculates similarity of probability distribution of the input problems and the known problems, and obtains output answers corresponding to the input problems. The method can effectively solve the problem that a plurality of entities exist in the problem and the relation between the entities is absent, and improves the comprehensiveness of the question-answering technology. The question-answering method provided by the application is suitable for any field needing a question-answering system, and is especially suitable for the question-answering system in the game field.

The question answering method, system, electronic device and computer readable storage medium according to the present invention will be described in further detail with reference to specific embodiments and accompanying drawings.

Fig. 1 is an application system diagram of a question-answering method provided by an embodiment of the present invention. As shown in fig. 1, the system includes a client 101 and a server 102. The client 101 and the server 102 are in communication connection through a network. The user terminal 101 may be a touch terminal, such as a smart phone, a tablet pc, a Personal digital assistant (Personal DIGITAL ASSISTANT, PDA), or a computer terminal, such as a notebook computer, a desktop computer, or a plurality of devices. The server 102 is configured to deploy the question-answering system provided by the present invention. The user inputs the questions through the user terminal 101, the questions are transmitted to the server terminal 102 through the network, the server terminal 102 analyzes and solves the questions, answers to the questions are transmitted back to the user terminal 101 through the network, and the user receives the answers to the questions through the user terminal 101. The server 102 may be a question and answer device for the client 101, the client 101 may be combined with the server 102, the user directly inputs the question through the server 102, the server 102 analyzes and answers the question, and the user receives the answer to the question through the server 102.

Fig. 2 is an application system diagram of another question-answering method according to an embodiment of the present invention. As shown in fig. 2, the application system includes a client 201 and a server 202. The user terminal 201 may be a touch terminal, such as a smart phone, a tablet pc, a Personal digital assistant (Personal DIGITAL ASSISTANT, PDA), or a computer terminal, such as a notebook computer, a desktop computer, or other various devices with a voice transmission function or a text transmission function, or one or more devices. The server 202 is used for deploying the question-answering method provided by the invention. The user inputs the questions through the user terminal 201, the questions are transmitted to the server 202 through the network, the questions are analyzed and solved on the server 202, the answers of the questions are transmitted back to the user terminal 201 through the network, and the user receives the answers of the questions through the user terminal 201. The server 202 may be an independent server, deploy the method provided by the present invention, or may be a server group formed by a plurality of servers, where each server deploys a module of the method provided by the present invention. Such as an identification server, a computing server, etc. Of course, the server 202 may also be a cloud server, where the question-answering method provided by the present invention is deployed. The server 202 provides question replies to all users.

The first embodiment of the invention provides a question and answer method.

Fig. 3 is a flowchart of a question-answering method provided in the present embodiment. The question-answering method provided in this embodiment is described in detail below with reference to fig. 3. The examples referred to in the following description are for explaining the technical solution of the present invention and are not intended to be limiting in practical use.

As shown in fig. 3, the question answering method provided in this embodiment includes the following steps:

Step S301, identify the entity in the input problem, and obtain the entity pair list.

The purpose of this step is to identify a number of entities in the user's input problem and to group the identified entities into a list of entity pair forms.

An input question may include one entity or a plurality of entities. Entities in the input question may be identified by an entity identification model.

The entity recognition model is a neural network model capable of recognizing an entity in an input problem. The neural network (Neural Networks, NNs), which is composed of several neurons and their parameters, is a system that "learns" through a large number of examples to perform tasks, typically without programming with task-specific rules. For example, in image recognition, the neural network may learn characteristics of a cat by analyzing example images labeled "cat" or "not cat" and use the learning results to identify whether other images contain a cat. In the learning of the neural network, the characteristics of the cat are not directly input to the neural network, but an example image marked as the cat is input, and the neural network automatically generates characteristic information representing the cat according to the example image through iterative learning.

The entity recognition model provided by the embodiment takes the entity existing in the knowledge graph as training data, provides the training data for the initial entity recognition model, trains the initial entity recognition model, and takes the entity recognition model which reaches the preset standard after training as the entity recognition model for practical use.

That is, the entity recognition model needs to be learned and trained by taking the entity existing in the knowledge graph as training data, and the recognition capability of the trained entity recognition model needs to be detected, and only the entity recognition model reaching the predetermined standard can be used as the entity recognition model used in the question-answering method provided by the embodiment.

The predetermined criterion provided in this embodiment is that an effective entity in the input problem, that is, an entity existing in the knowledge graph can be acquired.

After the entities in the input problem are identified, the entities in the input problem need to be subjected to pairwise team formation to form the entity pair list. For example, the input problem includes three entities (entity a, entity b, entity c), and entity a, entity b, entity c are paired to obtain entity pair ab, entity pair ac, and entity pair bc. The entity pair ab, the entity pair ac and the entity pair bc can form an entity pair list corresponding to the input problem.

If the relationships of the entity pairs in the knowledge graph are directional, the entity pair list includes entity pairs of entity pairs ab, entity pairs ba, entity pairs ac, entity pairs ca, entity pairs bc, and entity pairs cb.

Step S302, obtaining an alternative relation of each entity pair in the entity pair list, and forming an alternative relation set corresponding to the entity pair list.

The purpose of this step is to obtain an alternative relationship or an alternative relationship group corresponding to each entity pair in the entity pair list, and combine the alternative relationship or the alternative relationship group corresponding to each entity pair to form an alternative relationship set corresponding to the entity pair list.

The method for acquiring the corresponding alternative relation or the alternative relation group of the entity pair comprises the step of taking the relation of the entity pair existing in the knowledge graph as the alternative relation.

When the entity pair exists in the knowledge graph, the relationship is used as an alternative relationship of the entity pair. For example, when a certain user plays a certain game, an entity pair of a game role and a big dog exists in the input problem, and in a knowledge graph of the certain game, the relationship between the game role and the big dog is an example, so that the entity pair of the game role and the big dog has a relationship in the knowledge graph, and the example is taken as an alternative relationship of the entity pair.

The method for acquiring the corresponding alternative relation or the alternative relation group of the entity pair further comprises the step of taking the relation existing in the knowledge graph of the type pair corresponding to the entity pair as the alternative relation.

And when the entity pair does not exist in the knowledge graph, expanding the entity pair into a corresponding type pair, and taking the relationship group existing in the knowledge graph of the type pair as an alternative relationship group of the entity pair. For example, when a user plays a game, an entity pair of 'a deer man' and 'a cat bringing in wealth' exists in the input problem, and in a knowledge graph of the game, the entity pair of 'the deer man-the cat bringing in wealth' is expanded into a type pair of 'a role-a prop' (wherein 'the deer man' belongs to the role type and 'the cat bringing in wealth' belongs to the prop type) if the relation between 'the deer man' and the 'the cat bringing in wealth' does not exist. The relationship of the type pair 'role-prop' in the knowledge graph is 'suitable for carrying', 'recommended soul', and the alternative relationship group corresponding to the entity pair 'fawn male-wealth bringing cat' is 'suitable for carrying', 'recommended soul'.

The entity pairs are expanded into type pairs, and the alternative relation groups corresponding to the type pairs are used as the alternative relation groups corresponding to the entity pairs, so that the technical problem that when the entity lacks relations, the problem can not be solved is solved.

By the method, one or more alternative relations corresponding to each entity pair in the entity pair list are obtained, and the alternative relations are combined, so that an alternative relation set corresponding to the entity pair list is formed.

Step S303, obtaining probability distribution of the input problem on the alternative relation set according to the input problem and the alternative relation set.

The purpose of this step is to obtain a probability distribution of the input problem over the set of alternative relations obtained above.

The probability distribution is a correct probability distribution consisting of the correct probability of the problem on each candidate relationship in the set of candidate relationships. Firstly, obtaining a score of the input problem on each alternative relation in the alternative relation set through a relation extraction model, and secondly, combining the scores of the input problem on each alternative relation in the alternative relation set to form probability distribution of the input problem on the alternative relation set.

The relationship extraction model is a neural network model that can score the correct probability of an input problem on each of a set of alternative relationships.

The relation extraction model provided by the embodiment takes the known problems in the question-answer library and the corresponding relation description of the entities in the known problems as training data, provides the training data for the initial relation extraction model, trains the initial relation extraction model, and takes the relation extraction model which reaches the preset standard after training as the relation extraction model for practical use.

The question and answer library is a database for collecting known questions and answers, wherein the database comprises the known questions and corresponding relation descriptions of entities in the questions.

The relation description is a language for describing the relation of the entity pair, and the entity pair relation description have a one-to-one correspondence.

The relation extraction model provided in this embodiment is a neural network model that actually describes the known questions in the question-answer library and the corresponding relation of the entities in the known questions, forms training data in the input format of "[ CLS ] question [ SEP ] relation description [ SEP ]", trains the initial relation extraction model, and can evaluate and score the correct probability of the questions and the relation description. Therefore, in the practical use of the relation extraction model, the data to be evaluated also needs to be in the input format of "[ CLS ] problem [ SEP ] relation description [ SEP ]".

Fig. 4 is a flowchart of a probability distribution acquisition method provided in the present embodiment.

As shown in fig. 4, the probability distribution obtaining method provided in this embodiment includes the following steps:

And step S303-1, mapping each alternative relation in the alternative relation set into a relation description.

The set of candidate relationships is a set formed by combining all entity pairs in the input question with the corresponding candidate relationship or candidate relationship group. Before the correct probability evaluation is performed, each candidate relationship in the candidate relationship set needs to be mapped into a relationship description.

An optional implementation manner provided by this embodiment is that, according to a relationship and a relationship description list, a relationship description of each alternative relationship in the alternative relationship set is correspondingly obtained.

The relationship and relationship description list is a data set for correspondingly collecting relationship and relationship descriptions, and from the relationship and relationship description list, the relationship can be mapped into relationship descriptions.

For example, when a user plays a game, a question of "a cat bringing in wealth may not be possible for a deer man" is entered, and it is known from step S301 that there is one entity pair in the question, that is, the entity pair "a cat bringing in wealth for a deer man". As can be seen from step S302, the corresponding alternative relation group of the entity is "applicable to", "suitable for carrying", "recommended for soul". Because the input problem only comprises one entity pair, the set of alternative relations corresponding to the entity pair list of the input problem is "applicable", "suitable for carrying", "recommended soul". And mapping each alternative relationship in the alternative relationship set into a corresponding relationship description by adopting a relationship and relationship description list. The mapped set of alternative relationships is shown in table 1:

table 1 mapping alternative relationship set table

Step S303-2, obtaining correct probability of the input problem on the relation description of each alternative relation through the relation extraction model.

The input questions and each relationship description in Table 1 are combined into the input data format of the relationship extraction model as shown in Table 2:

Table 2 table of input data formats for relational extraction model

Inputting the input problem corresponding to each relation description into a relation extraction model in a data format shown in table 2, and evaluating the correct probability of the input problem on the relation description of each alternative relation according to the learning training result by the relation extraction model. The correct probability evaluation results are shown in table 3:

TABLE 3 evaluation results of correct probability table

As can be seen from table 3, the input problem "a male of a fawn may not carry a cat bringing in wealth? the input problem" a deer man may not be able to bring a cat bringing in wealth? the degree of correctness is evaluated as superior; evaluation of accuracy first class. That is, it is substantially impossible to input a relational description of the problem "a male of a deer may not be able to bring a cat bringing in wealth.

Step S303-3, taking the correct probability of the input question on the relation description of each alternative relation as the score of the input question on each alternative relation.

In the mapped candidate relationship set, the candidate relationships and the relationship descriptions are in one-to-one correspondence, so that the correct probability of the input problem on the relationship description of each candidate relationship can be used as the score of the input problem on each candidate relationship. That is, the correct probability of an input question on a relationship description is mapped to the score of the input question on an alternative relationship. The scoring results of the input questions on the alternative relationships are shown in table 4:

Table 4 scoring results table of input questions on alternative relationships

And step S303-4, forming a probability distribution of the input problem on the alternative relation set according to the score of the input problem on each alternative relation in the alternative relation set.

The set of alternative relationships includes a plurality of alternative relationships, and the score of the input question on each of the alternative relationships constitutes a probability distribution of the input question on the set of alternative relationships. From the scoring results of the input questions on the alternative relationships shown in table 4, it is possible to obtain a probability distribution of the input question "a male of fawn may not be able to bring a cat bringing in wealth" on the set of alternative relationships [0.689,0.887,0.125].

The candidate relationship set can be regarded as a relationship space, and the relationship space corresponding to the input problem that the deer male can not bring the cat bringing in wealth is a three-dimensional relationship space is present in the candidate relationship set. The relationship space may be multidimensional, depending on the set of alternative relationships to which the input problem corresponds.

The probability distribution of an input problem over a set of alternative relationships is described in detail above taking as an example an input problem comprising two entities. The method provided by the embodiment is applicable to input problems including more entities. For example, the input problem Q comprises an entity a, an entity b and an entity c, and the probability distribution of the input problem Q on the alternative relation set is obtained, which comprises the following steps:

First, a list of entity pairs ab, ac, bc is obtained.

Secondly, acquiring an alternative relation set, wherein the alternative relation of an entity to ab is alpha, the alternative relation of the entity to ac is beta, gamma and delta, and the alternative relation of the entity to bc is eta and theta. Then the set of candidate relationships corresponding to the entity pair list of the input problem is α, β, γ, δ, η, θ.

Thirdly, acquiring a mapped candidate relation set, wherein each candidate relation in the candidate relation set is mapped into a relation description, and the relation description of the candidate relation is alpha ', beta', gamma ', delta', eta ', theta'.

Fourth, the correct probability of the input problem on the relation description of each alternative relation is obtained, namely the input problem Q and each relation description form the input data format of a relation extraction model, and the correct probability of the input problem on each relation description is evaluated in the input relation extraction model. For example, the correct probability distribution of the input problem Q on the relation descriptions alpha ', beta', gamma ', delta', eta ', theta' is u%, v%, w%, x%, y%, z%.

Fifthly, obtaining probability distribution of the input problem on the alternative relation set, namely taking the correct probability of the input problem on the relation description of the alternative relation as the score of the input problem on the alternative relation, and forming the probability distribution of the input problem on the alternative relation set according to the score. For example, the probability distribution of the input problem Q on the alternative relation set is [ u%, v%, w%, x%, y%, z% ].

Step S304, obtaining a probability distribution of a known problem including any one entity pair in the entity pair list on the alternative relationship set.

The purpose of this step is to obtain a probability distribution of the known problem over the set of alternative relations obtained above.

The known questions originate from a question and answer library, and answers corresponding to the known questions and the questions are collected in the question and answer library.

Fig. 5 is a flow chart for obtaining probability distribution of known problems over a set of alternative relationships provided by the present embodiment. As shown in fig. 5, the method for obtaining probability distribution of the known problem on the alternative relationship set according to the present embodiment includes:

step S304-1, all known questions comprising any one entity pair in the entity pair list are obtained from the question-answer library.

And screening all known questions comprising any one entity pair in the entity pair list from the question-answer library according to the entity pair list acquired in the step S301. For example, the entity pair list comprises three entity pairs ab, ac and bc, and then the known questions screened from the question-answer library comprise the known questions of the entity pair ab, the entity pair ac or the entity pair bc.

To illustrate the above-described input problem "a fawn man may not be able to bring a cat bringing in wealth"? the entry question "may not bring a cat bringing in wealth? screening from the question-and-answer library all known questions comprising the" deer male-wealth bringing-up cat "entity pair, including: the first known problem is" does not have a male deer? the second known problem is how do a cat bringing in wealth match a deer male 4 piece suit?

Step S304-2, obtaining probability distribution of each known problem on the alternative relation set through a relation extraction model.

Combining the known problem with the relationship description of each alternative relationship in the alternative relationship set acquired in the step S302 to form an input data format of a relationship extraction model, and acquiring a score of the known problem on each alternative concern in the alternative relationship set in the relationship extraction model to further acquire the distribution probability of the known problem on the alternative relationship set.

To illustrate the above-described input problem "a fawn man may not be able to bring a cat bringing in wealth"? the first known question and the second known question are obtained from the question-answer library. Obtaining the distribution probability of the first known problem and the second known problem on the alternative relation set through relation extraction model distribution, wherein the obtaining steps are as follows:

First, a relationship description of each of the set of candidate relationships and the first known problem and the second known problem distribution are combined into an input data format of the relationship extraction model. As shown in table 5:

table 5 table of input data formats for relational extraction model

Second, the relationship descriptions of the known problem and the alternative relationships are input into the relationship extraction model in the data format of Table 5, which evaluates the probability that the input problem is correct on the relationship description of each alternative relationship. The correct probability evaluation results are shown in table 6:

TABLE 6 evaluation results Table of correct probability

Third, the probability of correctness of the known question on the relationship description of each alternative relationship is taken as the score of the known question on each alternative relationship. The scoring results of the known questions on the alternative relationships are shown in table 7:

TABLE 7 scoring results Table for known questions on alternative relationships

Fourth, a probability distribution of each known problem over the set of alternative relationships is formed based on the score of each known problem over each alternative relationship.

From the scoring results of the known questions on the alternative relationships shown in table 7, it is possible to obtain that the first known question "is suitable for a cat bringing in wealth and does not have a deer man.

Step S305, calculating the similarity of the probability distribution of the input question and the known question on the alternative relationship set, and determining the output answer corresponding to the input question according to the similarity of the probability distribution.

The purpose of this step is to calculate the similarity of the probability distribution of the input question to the known questions, determine the known question most similar to the input question, and output the answer to the known question as the answer to the input question.

The present implementation provides a method of measuring the similarity of two probability distributions, namely, using LS divergence values to measure the similarity of the probability distribution of the input problem and the known problem on the set of alternative relationships.

The JS divergence (Jensen-Shannon divergence) is a method for measuring the similarity between two probability distributions, solves the problem of asymmetry of KL divergence, has symmetrical JS divergence, takes a value between 0 and 1, has the same probability distribution as 0, and has the opposite value between 1 and 0 and 1, so that the similarity between the two probabilities can be represented.

Fig. 6 is a flowchart of determining an output reply according to probability distribution similarity provided by the present embodiment.

As shown in fig. 6, the method for calculating the similarity degree of probability distribution of an input question and a known question on an alternative relationship set and determining an output answer corresponding to the input question according to the similarity degree according to the embodiment includes the following steps:

Step S305-1, a JS divergence value of a probability distribution of the input question on the alternative relation set and a probability distribution of the known question on the alternative relation set is calculated.

The JS divergence value of the probability distribution of the input problem and the known problem is calculated, the probability distribution is firstly subjected to normalization processing, specifically, a Softmax formula (normalized exponential function) is adopted for processing, and then the JS formula is adopted for calculating the similarity of the probability distribution after normalization processing.

The Softmax formula is:

Where z _i represents the ith data in the probability distribution, z _C represents the C-th data in the probability distribution, and C represents the total number of data in the probability distribution.

The JS formula is:

Where p represents the first probability distribution, q represents the second probability distribution, p (x) represents the xth data in the first probability distribution, and q (x) represents the xth data in the second probability distribution.

By way of illustration of the above-described input problem "may not be possible with a cat bringing in wealth" the probability distribution of "may not be possible with a cat bringing in wealth" on the set of alternative relations [0.689,0.887,0.125], the first known problem "does a wealth bringing cat fit a god with or without a fawn? the second known problem is the probability distribution of how a cat bringing in wealth matches a deer male 4-piece suit on the set of alternative relationships 0.653,0.485,0.323. The probability distribution of the input problem, the first known problem, the second known problem over the alternative relationships is represented by P ₀、P₁、P₂, respectively. The probability distributions of the input problem, the first known problem, and the second known problem on the alternative relationships are summarized as shown in table 8:

Table 8 summary of probability distributions

Representative symbol	Problem(s)	Probability distribution
			P₀	Is a male of fawn unable to bring a cat bringing in wealth?	[0.689,0.887,0.125]
P₁	Suitable god for cat bringing in wealth is no deer man?	[0.589,0.789,0.305]
			P₂	What are wealth-bringing cats matched with 4 sets of male deer?	[0.653,0.485,0.323]

The JS divergence values of the probability distribution of the input problem on the alternative relation set and the probability distribution of the known problem on the alternative relation set are calculated, and the specific steps are as follows:

first, the probability distribution is normalized, and normalized probability distribution is obtained.

Taking P ₀ [0.689,0.887,0.125] as an example, the calculation process is as follows:

The probability distribution [0.689,0.887,0.125] of the input problem on the alternative relation after normalization processing is converted into [0.36,0.44,0.20], and the sum of values in the normalized probability distribution is 1. Thus, the normalized probability distribution may be considered as a probability distribution of the problem in different dimensions, e.g., the normalized probability distribution [0.36,0.44,0.20] may be considered as a probability distribution of the input problem in three dimensions.

The probability distribution of the first known problem and the second known problem on the alternative relation is normalized by the same calculation method, and the normalization processing result is shown in table 9:

Table 9 normalized probability distribution results table

Representative symbol	Problem(s)	Probability distribution	Normalized probability distribution
				P₀	Is a male of fawn unable to bring a cat bringing in wealth?	[0.689,0.887,0.125]	[0.36,0.44,0.20]
P₁	Suitable god for cat bringing in wealth is no deer man?	[0.589,0.789,0.305]	[0.34,0.41,0.25]
				P₂	What are wealth-bringing cats matched with 4 sets of male deer?	[0.653,0.485,0.323]	[0.39,0.33,0.28]

Second, the JS divergence value of the probability distribution of the input problem with the known problem over the set of alternative relationships is calculated.

The calculation of the JS divergence value is performed by using the normalized probability distribution, taking the calculation of the JS divergence value of the probability distribution P ₀ of the input problem on the alternative relationship set and the probability distribution P ₁ of the first known problem on the alternative relationship set as an example, and the calculation process is as follows:

JS(P₀||P₁)=0.5×0.002664+0.5×0.002528=0.002596

thus, the JS divergence value of the probability distribution of the input problem over the set of alternative relationships with the first known problem is 0.002528.

The JS divergence values of the probability distribution P ₀ of the input problem on the alternative relation set and the probability distribution P ₂ of the second known problem on the alternative relation set are calculated by the same method, and the calculation result is as follows:

JS(P₀||P₂)=0.01095

Thus, the JS divergence value of the probability distribution of the input problem over the set of alternative relationships with the second known problem is 0.01095.

And step S305-2, judging the similarity between the input problem and the known problem according to the JS divergence value.

The JS divergence value measures the similarity of two probability distributions, the value range of which is [0,1], the value range is 0, and the value range is 1. The similarity between the input question and the known question can be determined based on the JS divergence value of the probability distribution of the input question and the known question over the set of alternative relationships.

The method for judging the similarity of the input problem and the known problem according to the JS divergence value comprises the steps of comparing whether the JS divergence value is smaller than a preset similarity threshold value or not, judging whether the input problem is similar to the known problem or not according to a comparison result, wherein if yes, the input problem is similar to the known problem, and if not, the input problem is dissimilar to the known problem.

The preset similarity threshold is a preset JS divergence threshold, if the JS divergence value of the probability distribution of the calculated input problem and the known problem on the alternative relationship set is smaller than the preset similarity threshold, the input problem is similar to the known problem, and if the JS divergence value of the probability distribution of the calculated input problem and the known problem on the alternative relationship set is greater than or equal to the preset similarity threshold, the input problem is dissimilar to the known problem. The preset similarity threshold can be adjusted according to specific application scenarios and practical accumulation.

To illustrate the above-described input problem "a fawn man may not be able to bring a cat bringing in wealth"? the preset similarity threshold is set to 0.01, and as can be seen from the calculation result in step S305-1, the input question "does a deer man may not be able to bring a cat bringing in wealth? is there a deer male? the JS divergence value of the closed probability distribution is 0.002528. Because 0.002528<0.01, it can be determined that the input problem "a deer man may not be able to bring a cat bringing in wealth" is similar to the first known problem "a cat bringing in wealth is suitable god without a deer man.

Also, as can be seen from the calculation result in step S305-1, the input problem "is a cat bringing in wealth that can not be brought by a deer" and the second known problem "how is a cat bringing in wealth matched with 4 sets of deer men" are the JS divergence value of the probability distribution on the set of alternative relationships 0.01095 ". Because 0.01095>0.01, it can be judged that the input problem "is a cat bringing in wealth that a deer man may not be able to bring" is not similar to the second known problem "how is a cat bringing in wealth matched with a 4-piece set of a deer man.

The method for judging the similarity between the input problem and the known problem according to the JS divergence value further comprises the steps of comparing the magnitudes of a plurality of JS divergence values, and judging the known problem with the highest similarity with the input problem according to a comparison result, wherein the smaller the JS divergence value is, the higher the similarity between the input problem and the known problem is.

For the case where an input question corresponds to multiple known questions, the JS divergence values of the probability distributions of the input question and each of the known questions over the set of alternative relationships may be compared, the smaller the JS divergence values, the higher the similarity of the input question to the known questions.

By way of illustration of the above-described input problem "a male of a fawn may not carry a cat bringing in wealth? the input question" does a deer man may not be able to bring a cat bringing in wealth? is there a deer male? the input question "does a deer man may not bring a cat bringing in wealth? is the JS divergence value for the probability distribution over the set of alternative relationships of 0.01095 for the 4-piece sleeve for a male with a deer. Since 0.002528<0.01095, it can be judged that the input problem "is a cat bringing in wealth that a deer man may not be able to bring," has the highest similarity to the first known problem "is a cat bringing in wealth suitable god without a deer man.

Step S305-3, determining an output answer corresponding to the input question according to the similarity between the input question and the known question.

According to the similarity between the input question and the known question, the method for determining the output answer corresponding to the input question comprises the step of taking the answer of the known question similar to the input question as the output answer corresponding to the input question.

As exemplified by the above-described input problem "a male of a fawn may not bring a cat bringing in wealth? the input question" does a deer man may not bring a cat bringing in wealth? the known problem "does not have a male deer suitable for a cat bringing in wealth? problem-aware" Cat bringing in wealth is suitable "is the spirit that a deer man is not. Thus, an output answer can be made with the answer of the first known question "is a wealth bringing cat suitable god not a fawn man.

The method for determining the output answer corresponding to the input question according to the similarity between the input question and the known question further comprises the step of taking the answer of the known question with the highest similarity to the input question as the output answer corresponding to the input question.

For the case where the input question corresponds to a plurality of known questions and the JS divergence values of the distribution probabilities of the input question and the plurality of known questions over the set of alternative relationships are all less than the preset similarity threshold, the answer to the known question that is most similar to the input question may be taken as the output answer corresponding to the input question with the minimum JS divergence value.

By way of illustration of the above-described input problem "a male of a fawn may not carry a cat bringing in wealth? the input question" does a deer man may not be able to bring a cat bringing in wealth? "does a cat bringing in wealth have a suitable god without a deer man. Thus, an output answer can be made with the answer of the first known question "is a wealth bringing cat suitable god not a fawn man.

The first embodiment above describes in detail the question-answering method provided by the present invention in an alternative implementation manner, and the question-answering method provided by the present invention includes, but is not limited to, the implementation manner given in the first embodiment.

The second embodiment of the invention provides a question and answer system. Fig. 7 is a schematic structural diagram of the question-answering system provided in the present embodiment.

As shown in fig. 7, the question-answering system provided by the present embodiment includes an entity recognition unit 701, an alternative relationship acquisition unit 702, an input question probability distribution acquisition unit 703, a known question probability distribution acquisition unit 704, and a similarity calculation unit 705.

The entity identifying unit 701 is configured to identify an entity in the input problem, and obtain an entity pair list.

identifying an entity in the input problem by an entity identification model;

Optionally, the method for obtaining the entity identification model includes:

The candidate relationship obtaining unit 702 is configured to obtain a candidate relationship of each entity pair in the entity pair list, and form a candidate relationship set corresponding to the entity pair list.

The input question probability distribution obtaining unit 703 is configured to obtain a probability distribution of the input question on the candidate relationship set according to the input question and the candidate relationship set.

Optionally, the method for obtaining the relation extraction model includes:

The known problem probability distribution obtaining unit 704 is configured to obtain a probability distribution of a known problem including any one entity pair in the entity pair list on the alternative relationship set.

The similarity calculation unit 705 is configured to calculate a similarity of probability distributions of the input question and the known question on the alternative relationship set, and determine an output answer corresponding to the input question according to the similarity of probability distributions.

Comparing the magnitudes of a plurality of JS divergence values;

A third embodiment of the present invention provides an electronic device. Fig. 8 is a schematic structural diagram of an electronic device provided in the present embodiment.

As shown in fig. 8, the electronic device provided in this embodiment includes a memory 801 and a processor 802.

The memory 801 is used for storing computer instructions for executing the question-answering method.

The processor 802 is configured to execute computer instructions stored in the memory 801 and perform the following operations:

Identifying an entity in the input problem, and acquiring an entity pair list;

identifying an entity in the input problem by an entity identification model;

Optionally, the method for obtaining the entity identification model includes:

Optionally, the method for obtaining the relation extraction model includes:

Comparing the magnitudes of a plurality of JS divergence values;

A fourth embodiment of the invention provides a computer-readable storage medium comprising computer instructions which, when executed by a processor, are adapted to carry out the solution according to the first embodiment of the invention.

It is noted that the relational terms such as "first," "second," and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Furthermore, the terms "comprise," "have," "include," and other similar terms, are intended to be inclusive and open-ended in that any one or more items following any one of the terms described above, neither of which indicates that the one or more items have been enumerated, as an exhaustive list, or limited to only those one or more items so enumerated.

As used herein, unless expressly stated otherwise, the term "or" includes all possible combinations, except where not possible. For example, if expressed as a database may include a or B, then unless specifically stated or not possible, the database may include a, or B, or a and B. In a second example, if expressed as a database might include A, B or C, the database may include database A, or B, or C, or A and B, or A and C, or B and C, or A and B and C, unless otherwise specifically stated or not possible.

It is noted that the above-described embodiments may be implemented by hardware or software (program code), or a combination of hardware and software. If implemented by software, it may be stored in the computer-readable medium described above. The software, when executed by a processor, may perform the methods disclosed above. The computing units and other functional units described in this disclosure may be implemented by hardware or software, or a combination of hardware and software. Those of ordinary skill in the art will also appreciate that the above-described modules/units may be combined into one module/unit, and each of the above-described modules/units may be further divided into a plurality of sub-modules/sub-units.

In the foregoing detailed description, embodiments have been described with reference to numerous specific details that may vary from implementation to implementation. Certain adaptations and modifications of the described embodiments can be made. Other embodiments will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. The specification and examples are for illustrative purposes only, with the true scope and nature of the invention being indicated by the following claims. The order of steps shown in the figures is also for illustrative purposes only and is not meant to be limited to any particular step, order. Accordingly, those skilled in the art will recognize that the steps may be performed in a different order when performing the same method.

In the drawings and detailed description of the application, exemplary embodiments are disclosed. Many variations and modifications may be made to these embodiments. Accordingly, although specific terms are employed, they are used in a generic and descriptive sense only and not for purposes of limitation.

Claims

1. A question-answering method, comprising:

Identifying an entity in the input problem, and acquiring an entity pair list;

Acquiring probability distribution of the input problem on the alternative relationship set according to the input problem and the alternative relationship set, wherein the probability distribution of the input problem on the alternative relationship set comprises the steps of acquiring a score of the input problem on each alternative relationship in the alternative relationship set;

2. The method of claim 1, wherein identifying the entity in the input question, obtaining the entity pair list, comprises:

identifying an entity in the input problem by an entity identification model;

3. The method of claim 2, wherein the method of obtaining the entity identification model comprises:

4. The method of claim 1, wherein the obtaining the candidate relationship for each of the entity pairs in the list of entity pairs comprises taking the relationship of the entity pairs in the knowledge-graph as a candidate relationship.

5. The method of claim 1, wherein the obtaining the candidate relationship for each entity pair in the list of entity pairs further comprises taking the relationship existing in the knowledge graph for the type pair corresponding to the entity pair as the candidate relationship.

6. The method of claim 1, wherein the obtaining a score for the input question on each candidate relationship in the set of candidate relationships comprises:

And obtaining the score of the input problem on each alternative relation in the alternative relation set through a relation extraction model.

7. The method of claim 6, wherein the method of obtaining the relationship extraction model comprises:

8. The method of claim 6, wherein the obtaining, by a relationship extraction model, a score for the input problem on each of the set of alternative relationships comprises:

9. The method of claim 8, wherein mapping each of the set of alternative relationships to a relationship description comprises correspondingly obtaining a relationship description for each of the set of alternative relationships from a list of relationship and relationship descriptions.

10. The method of claim 1, wherein the obtaining a probability distribution over the set of alternative relationships of a known problem comprising any one of the entity pairs in the list of entity pairs comprises:

11. The method of claim 1, wherein the calculating a degree of similarity of the input question to a probability distribution of the known question over the set of alternative relationships and determining an output answer corresponding to the input question based on the degree of similarity comprises:

12. The method of claim 11, wherein said determining a similarity of said input question to said known question based on said JS divergence value comprises:

13. The method of claim 11, wherein said determining a similarity of said input question to said known question based on said JS divergence value further comprises:

Comparing the magnitudes of a plurality of JS divergence values;

14. The method of claim 11, wherein determining an output answer corresponding to the input question based on the similarity of the input question to the known question comprises responding to the output answer corresponding to the input question with the answer to the known question having the highest similarity to the input question.

15. The question-answering system is characterized by comprising an entity identification unit, an alternative relation acquisition unit, an input problem probability distribution acquisition unit, a known problem probability distribution acquisition unit and a similarity calculation unit;

the input problem probability distribution obtaining unit is used for obtaining the probability distribution of the input problem on the alternative relationship set according to the input problem and the alternative relationship set, and comprises obtaining the score of the input problem on each alternative relationship in the alternative relationship set;

16. An electronic device is characterized by comprising a memory and a processor;

the memory is used for storing one or more computer instructions;

The processor being configured to execute the one or more computer instructions to implement the method of any of claims 1-14.

17. A computer readable storage medium having stored thereon one or more computer instructions which, when executed by a processor, perform the method of any of claims 1-14.