CN110008476B - Semantic analysis method, device, equipment and storage medium - Google Patents

Abstract

The present disclosure provides a semantic parsing method, comprising: acquiring the characteristics related to each word in the input sequence to obtain a sequence containing the characteristics related to each word; generating a first vector representation of the input sequence based on the sequence comprising features associated with each word; generating an intent attention vector and a semantic slot attention vector based on the first vector representation; and obtaining a semantic analysis result corresponding to the input sequence according to the intention attention vector and the semantic slot attention vector. The disclosure also provides a semantic parsing device, an electronic device and a readable storage medium.

Description

Semantic analysis method, device, equipment and storage medium

technical field

The disclosure relates to a semantic analysis method, a word segmentation device, electronic equipment and a readable storage medium.

Background technique

The traditional semantic analysis method divides intent prediction and semantic slot filling into two steps. The first step is to classify the intent of the input sentence, and the second step is to fill the semantic slot under a specific intent. The relationship between intentions and semantic slots in the same sentence is often overlooked, but semantic slots are often strongly associated with intentions.

In the prior art, a method of simultaneously performing intent prediction and semantic slot filling by using an attention mechanism to simulate the relationship between semantic slots and intent has been proposed, in order to improve the accuracy of semantic slot filling and intent prediction. But on the semantic slot filling task, only using the attention mechanism cannot achieve good results.

Contents of the invention

In order to solve at least one of the above technical problems, the present disclosure provides a semantic analysis method, a semantic analysis device, electronic equipment and a readable storage medium.

According to one aspect of the present disclosure, a semantic parsing method includes obtaining features related to each word in the input sequence, and obtaining a sequence containing features related to each word; based on the sequence containing features related to each word , generate the first vector representation of the input sequence; based on the first vector representation, generate an intention attention vector and a semantic slot attention vector; according to the first vector representation, the intention attention vector and the semantic slot attention vector A semantic analysis result corresponding to the input sequence is obtained, wherein the semantic analysis result includes an intent prediction result and a semantic slot filling result.

According to an aspect of the present disclosure, the method further includes that the sequence containing the features related to each word is obtained by fusing the word embedding of the input sequence with the related features.

According to an aspect of the present disclosure, the method further includes, based on the sequence containing the features related to each word, generating the first vector representation of the input sequence includes: the sequence containing the features related to each word is input to a bidirectional long-short-term memory network (BLSTM), and concatenate the forward hidden sequence and the reverse hidden sequence to obtain the first vector representation.

According to an aspect of the present disclosure, the method further includes, based on the first vector representation, generating an intention attention vector and a semantic slot attention vector includes: calculating an attention weight for each hidden state in the first vector representation, Based on each hidden state and its corresponding attention weight, the semantic slot attention vector is obtained, and based on the hidden state of the last time step and its corresponding attention weight, the intent attention vector is obtained.

According to an aspect of the present disclosure, the method further includes, based on the intention attention vector and the semantic slot attention vector, obtaining a weighted feature related to each hidden state, based on the weighted feature, the first vector representation and the semantic slot Attention vectors get the semantic slot filling results.

According to an aspect of the present disclosure, the method further includes inputting the weighted feature, the first vector representation and the semantic slot attention vector into a conditional random field (CRF) layer to obtain a semantic slot filling result.

According to one aspect of the present disclosure, the method further includes obtaining features related to each word in the input sequence from the knowledge graph, and merging the word embedding of the input sequence with the related features according to the position to obtain a sequence of features.

According to one aspect of the present disclosure, a semantic parsing device is provided, including: a word-feature sequence generation module, used to obtain features related to each word in the input sequence, and obtain a sequence containing features related to each word ; The first vector representation generation module is used to generate the first vector representation of the input sequence based on the sequence containing the features related to each word; the attention vector generation module is used to represent based on the first vector representation, Generating an intention attention vector and a semantic slot attention vector; a semantic analysis result generation module, configured to obtain a semantic analysis result corresponding to the input sequence according to the first vector representation, the intention attention vector and the semantic slot attention vector, Wherein, the semantic analysis results include intent prediction results and semantic slot filling results.

According to yet another aspect of the present disclosure, an electronic device includes: a memory storing computer-executable instructions; and a processor executing the computer-executable instructions stored in the memory so that the processor executes the above method.

According to another aspect of the present disclosure, a readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to implement the above method when executed by a processor.

Description of drawings

The accompanying drawings illustrate exemplary embodiments of the present disclosure and, together with the description, serve to explain the principles of the disclosure, are included to provide a further understanding of the disclosure, and are incorporated in and constitute this specification. part of the manual.

Fig. 1 is a schematic flowchart of a semantic parsing method according to an embodiment of the present disclosure.

FIG. 2 is a schematic flowchart of an attention vector generation process according to an embodiment of the present disclosure.

Fig. 3 is a schematic block diagram of a semantic parsing device according to an embodiment of the present disclosure.

Fig. 4 is a schematic block diagram of an attention vector generation device according to an embodiment of the present disclosure.

FIG. 5 is a schematic view of an electronic device according to one embodiment of the present disclosure.

Detailed ways

The present disclosure will be further described in detail below with reference to the drawings and embodiments. It can be understood that the specific implementation manners described here are only used to explain relevant content, rather than to limit the present disclosure. It should also be noted that, for ease of description, only parts related to the present disclosure are shown in the drawings.

It should be noted that, in the case of no conflict, the implementation modes and the features in the implementation modes in the present disclosure can be combined with each other. The present disclosure will be described in detail below with reference to the drawings and embodiments.

In related technologies in this field, semantic parsing is a very important module in natural language processing (NLP), including intent prediction and semantic slot filling. Intent prediction refers to the judgment of the user's intention. Common intentions include "weather query", "music on demand", "video on demand" and so on. Semantic slot filling refers to extracting corresponding entities to further accurately parse semantics under specific intentions. For example, in the "weather query" intent, the slots can be "city name" and "time", in the "music on demand" intent, the slots can be "song name", "artist name", "album name" and "song type ", etc., in the "Video on Demand" intent, the slots can be "Video Name", "Director" and "Actor", etc.

Fig. 1 is a schematic flowchart of a semantic parsing method according to an embodiment of the present disclosure.

In one embodiment of the present disclosure, the semantic analysis method includes, S11 acquiring features related to each word in the input sequence to obtain a sequence containing features related to each word; S12 based on the Sequence of features, generating a first vector representation of the input sequence; S13 generating an intention attention vector and a semantic slot attention vector based on the first vector representation; S14 generating an intention attention vector and a semantic slot attention vector according to the first vector representation, the intention attention vector and the semantic slot attention vector to obtain a semantic analysis result corresponding to the input sequence, wherein the semantic analysis result includes an intent prediction result and a semantic slot filling result.

In step S11, the features related to each word in the input sequence are obtained, and the sequence containing the features related to each word is obtained by fusing the word embedding of the input sequence with the related features.

In this disclosure, for an input sequence x={x ₁ ,x ₂ ,...x _T }, at each time step i, word embedding (wordembedding) is fused with one or more related features (such as splicing) Get ( _xi , f _i ), thus generating a sequence containing features associated with each word. In the present disclosure, the above-mentioned sequence including features related to each word is called a word-feature sequence.

Specifically, one or more features related to each word in the input sequence are queried in the knowledge graph. The features may include parts of speech, attributes, relationships to other entities, and the like. Next, word embeddings of the input sequence are fused with associated features positionally to obtain a sequence containing features associated with each word.

In step S22, the word-feature sequence is input into a neural network model, such as a bidirectional long short-term memory network (BLSTM), so as to obtain a first vector representation. The first vector representation models the context information in the input word-feature sequence. Those skilled in the art should understand that the above bidirectional long short-term memory network (BLSTM) implementation is exemplary, and other neural network models that can achieve similar or more functions can also be used.

Specifically, in a bidirectional long-short-term memory network (BLSTM), the forward hidden sequence and the reverse hidden sequence are concatenated to obtain the first vector representation. For example, at each time step i, it can be expressed as a hidden state h _i .

Next, in step S13, the first vector representation distribution is input to the intention attention layer and the semantic slot attention layer, so as to obtain the intention attention vector and the semantic slot attention vector.

Finally, in step S14, the first vector representation, the intent attention vector and the semantic slot attention vector are input to the output layer, so as to obtain the semantic analysis result corresponding to the input sequence, wherein the semantic analysis result Including intent prediction results and semantic slot filling results.

For example, for the input sequence "Qilixiang playing XYZ" (XYZ is the singer's name, represented by XYZ here), it can be predicted that the intent prediction result of the above input sentence is "music on demand", and its corresponding slot includes "singer name ", "Song Name", fill the "Singer Name" slot with "XYZ", and fill the "Song Name" slot with "Qilixiang".

FIG. 2 is a schematic flowchart of an attention vector generation process according to an embodiment of the present disclosure.

In step S21, in the intention attention layer and the semantic slot attention layer, attention weights are obtained by using an attention mechanism.

In steps S22 and S23, at each time step i, the hidden state h _i and the attention weight are weighted and summed to obtain the semantic slot attention vector. The intention attention vector is obtained by weighting the hidden state value of the last time step with the corresponding attention weight.

Next, in step S24, weighted features related to each hidden state are obtained based on the semantic slot attention vector and the intent attention vector. For example via semantic slot-gated.

Next, the first vector representation, the semantic slot attention vector, and the weighted features associated with each hidden state obtained by fusing the semantic slot attention vector and the intent attention vector are used as the input of the output layer (e.g., conditional random field layer), Get the result of semantic slot filling. Here, by fusing semantic-slot attention vectors and intent-attention vectors, the relationship between intent and semantic slots is modeled, while providing the performance of both intent prediction and semantic slot filling.

According to yet another embodiment of the present disclosure, a semantic analysis device 30 is provided. Fig. 3 shows a semantic parsing device according to one embodiment of the present disclosure, which includes a word-feature sequence generation module 31 for obtaining features related to each word in the input sequence, and obtaining features related to each word sequence; the first vector representation generation module 32 is used to generate the first vector representation of the input sequence based on the sequence containing the features related to each word; the attention vector generation module 33 is used to generate the first vector representation based on the first A vector representation, generating an intention attention vector and a semantic slot attention vector; a semantic parsing result generation module 34, used to obtain the input sequence corresponding to the input sequence according to the first vector representation, the intention attention vector and the semantic slot attention vector Semantic parsing results, wherein the semantic parsing results include intent prediction results and semantic slot filling results.

According to yet another embodiment of the present disclosure, an attention vector generation device 40 is provided, which is a specific implementation of the semantic analysis device 30 . FIG. 4 is an attention vector generation device 40 according to another embodiment of the present disclosure. It includes, attention weight generation module 41, is used for generating the attention weight corresponding to each hidden state; Semantic groove attention vector generation module 42, is used for based on each hidden state and its corresponding attention weight, obtains Semantic slot attention vector; intent attention vector generation module 43, based on the hidden state of the last time step and its corresponding attention weight, obtain intent attention vector; joint weighted feature generation module 44, based on the intent attention vector and semantic slot attention vectors to obtain weighted features associated with each hidden state.

And the processing procedures executed in the above modules respectively correspond to the corresponding procedures specifically described in the above method.

The present disclosure also provides an electronic device, as shown in FIG. 5 , the device includes: a communication interface 1000 , a memory 2000 and a processor 3000 . The communication interface 1000 is used for communicating with external devices and performing interactive data transmission. Computer programs executable on the processor 3000 are stored in the memory 2000 . When the processor 3000 executes the computer program, the methods in the foregoing implementation manners are implemented. The number of the memory 2000 and the processor 3000 may be one or more.

The memory 2000 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one magnetic disk memory.

If the communication interface 1000, the memory 2000, and the processor 3000 are implemented independently, the communication interface 1000, the memory 2000, and the processor 3000 may be connected to each other through a bus to complete mutual communication. The bus can be an Industry Standard Architecture (ISA, Industry Standard Architecture) bus, a Peripheral Component Interconnect (PCI, Peripheral Component) bus or an Extended Industry Standard Architecture (EISA, Extended Industry Standard Component) bus, etc. The bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one thick line is used in this figure, but it does not mean that there is only one bus or one type of bus.

Optionally, in terms of specific implementation, if the communication interface 1000, the memory 2000, and the processor 3000 are integrated on one chip, the communication interface 1000, the memory 2000, and the processor 3000 can communicate with each other through internal interfaces.

The present invention mainly combines the actual business needs, improves the shortcomings of the existing word segmentation algorithm, and combines the machine learning algorithm with the field-customized dictionary. On the one hand, it can improve the accuracy of word segmentation, and on the other hand, it can target actual application scenarios. , to improve its domain adaptability.

Any process or method descriptions in flowcharts or otherwise described herein may be understood to represent modules, segments or portions of code comprising one or more executable instructions for implementing specific logical functions or steps of the process , and the scope of preferred embodiments of the present disclosure includes additional implementations in which functions may be performed out of the order shown or discussed, including substantially concurrently or in reverse order depending on the functions involved, which shall It is understood by those skilled in the art to which the embodiments of the present disclosure belong. The processor executes the various methods and processes described above. For example, method embodiments in the present disclosure may be implemented as a software program tangibly embodied on a machine-readable medium, such as memory. In some implementations, part or all of the software program may be loaded and/or installed via memory and/or a communication interface. One or more steps in the methods described above may be performed when a software program is loaded into memory and executed by a processor. Alternatively, in other implementation manners, the processor may be configured to perform one of the above-mentioned methods in any other suitable manner (for example, by means of firmware).

The logic and/or steps shown in the flowcharts or otherwise described herein can be embodied in any readable storage medium for instruction execution systems, devices or devices (such as computer-based systems, processor-included system or other systems that may fetch and execute instructions from an instruction execution system, device, or device), or be used in conjunction with such an instruction execution system, device, or device.

As far as this specification is concerned, a "readable storage medium" may be any device that can contain, store, communicate, propagate or transmit programs for instruction execution systems, devices or devices or use in conjunction with these instruction execution systems, devices or devices. More specific examples (non-exhaustive list) of readable storage media include the following: electrical connection with one or more wires (electronic device), portable computer disk case (magnetic device), random access memory (RAM), Read Only Memory (ROM), Erasable and Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Read Only Memory (CDROM). In addition, the readable storage medium may even be paper or other suitable medium on which the program can be printed, since it may be possible, for example, by optical scanning of the paper or other medium, followed by editing, interpretation or other suitable processing if necessary. The program is processed electronically and then stored in memory.

It should be understood that various parts of the present disclosure may be realized by hardware, software or a combination thereof. In the embodiments described above, various steps or methods may be implemented by software stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or combination of the following techniques known in the art: Discrete logic circuits, ASICs with suitable combinational logic gates, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

Those of ordinary skill in the art can understand that all or part of the steps to realize the method of the above-mentioned embodiment can be completed by instructing related hardware through a program, and the program can be stored in a readable storage medium, and the program can be executed when executed , comprising one or a combination of the steps of the method embodiments.

In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing module, each unit may exist separately physically, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. If the integrated modules are realized in the form of software function modules and sold or used as independent products, they can also be stored in a readable storage medium. The storage medium may be a read-only memory, a magnetic disk or an optical disk, and the like.

In the description of this specification, descriptions referring to the terms "one embodiment/mode", "some embodiments/modes", "examples", "specific examples", or "some examples" mean that the embodiments/modes are combined Specific features, structures, materials or features described in or examples are included in at least one implementation/mode or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment/mode or example. Moreover, the described specific features, structures, materials or characteristics may be combined in any one or more embodiments/modes or examples in an appropriate manner. In addition, those skilled in the art can combine and combine different implementations/modes or examples and features of different implementations/modes or examples described in this specification without conflicting with each other.

In addition, the terms "first" and "second" are used for descriptive purposes only, and cannot be interpreted as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include at least one of these features. In the description of the present application, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

It should be understood by those skilled in the art that the above-mentioned embodiments are only for clearly illustrating the present disclosure, rather than limiting the scope of the present disclosure. For those skilled in the art, other changes or modifications can be made on the basis of the above disclosure, and these changes or modifications are still within the scope of the present disclosure.

Claims (8)

1. A semantic parsing method, comprising:

acquiring the characteristics related to each word in the input sequence to obtain a sequence containing the characteristics related to each word;

generating a first vector representation of the input sequence based on the sequence comprising features associated with each word;

generating an intent attention vector and a semantic slot attention vector based on the first vector representation comprises:

an attention weight is calculated for each hidden state in the first vector representation,

obtaining a semantic slot attention vector based on the attention weights corresponding to each hidden state and the hidden states, and

obtaining an intention attention vector based on the hidden state of the last time step and the corresponding attention weight thereof; and

according to the first vector representation, the intention attention vector and the semantic slot attention vector obtain semantic analysis results corresponding to the input sequence,

the semantic analysis result comprises an intention prediction result and a semantic slot filling result;

acquiring features associated with each word in the input sequence, the sequence including the features associated with each word, comprises:

and acquiring the features related to each word in the input sequence from the knowledge graph, and embedding the words of the input sequence into the related features according to the positions to fuse the related features to obtain a sequence containing the features related to each word.

2. The method of claim 1, wherein the sequence containing the features associated with each word is obtained by word embedding of the input sequence fused with the associated features.

3. The method of claim 1 or 2, wherein generating a first vector representation of the input sequence based on the sequence comprising each word-related feature comprises:

the sequence containing the relevant characteristics of each word is input into a two-way long-short-term memory network, and the forward hiding sequence and the reverse hiding sequence are spliced to obtain the first vector representation.

4. The method of claim 1 or 2, further comprising:

a weighted feature associated with each hidden state is obtained based on the intent attention vector and the semantic slot attention vector,

and

And obtaining the semantic slot filling result based on the weighted features, the first vector representation and the semantic slot attention vector.

5. The method as recited in claim 4, further comprising:

and inputting the weighted features, the first vector representation and the semantic slot attention vector into a conditional random field layer to obtain a semantic slot filling result.

6. A semantic parsing apparatus, comprising:

the word-feature sequence generation module for obtaining features related to each word in the input sequence includes: acquiring features related to each word in the input sequence from the knowledge graph, embedding the words of the input sequence into the related features according to the positions to obtain a sequence containing the features related to each word, and obtaining a sequence containing the features related to each word;

a first vector representation generation module for generating a first vector representation of the input sequence based on the sequence of features that include each word association;

an attention vector generation module for generating an intended attention vector and a semantic slot attention vector based on the first vector representation, comprising: calculating an attention weight for each hidden state in the first vector representation, obtaining a semantic slot attention vector based on the attention weight corresponding to each hidden state and the hidden state, and obtaining an intended attention vector based on the hidden state of the last time step and the attention weight corresponding to the hidden state;

a semantic analysis result generation module, configured to obtain a semantic analysis result corresponding to the input sequence according to the first vector representation, the intent attention vector and the semantic slot attention vector,

the semantic analysis results comprise an intention prediction result and a semantic slot filling result.

7. An electronic device, comprising:

a memory storing execution instructions; and

a processor executing the memory-stored execution instructions, causing the processor to perform the method of any one of claims 1 to 5.

8. A readable storage medium having stored therein execution instructions which when executed by a processor are adapted to carry out the method of any one of claims 1 to 5.

Images