CN115858747A - Clustering-combined Prompt structure intention identification method, device, equipment and storage medium - Google Patents

Abstract

The invention provides a clustering-combined Prompt structure intention identification method, a clustering-combined Prompt structure intention identification device, clustering-combined Prompt structure equipment and a clustering-combined Prompt structure storage medium. In addition, in the process of constructing the Prompt structure, a clustering method is used for discovering the mode of the intention center text, and the Prompt structure required by intention identification is automatically constructed. The problem of fluctuation of an intention prediction result generated when a prompt structure is constructed manually can be avoided.

Description

Prompt structural intent recognition method, device, equipment and storage medium combined with clustering

technical field

The present invention relates to the field of computer technology, in particular to a method, device, device and storage medium for recognizing Prompt structure intentions combined with clustering.

Background technique

In human-computer dialogue, many methods have been proposed for each round of dialogue text representation. Currently, the more commonly used text representation methods generally use unsupervised training of language models on large-scale corpus, and then fine-tune parameters on the models trained by themselves. Implement a text representation process for intent classification tasks. Using the method based on pre-trained language model plus parameter fine-tuning requires obtaining a large amount of labeled data in downstream tasks, which requires a lot of time and labor costs. By using the pre-trained language model and the structure of the prompt prompt, the user's intention can be obtained based on 0 samples, but the acquisition of the prompt prompt structure requires a lot of manual experience to make, which may cause fluctuations in the intention prediction results.

Contents of the invention

The object of the present invention is to provide a method, device, equipment and storage medium for recognizing Prompt structure intention combined with clustering.

The present invention provides a kind of Prompt structural intent recognition method combined with clustering, which comprises steps:

Obtaining a dialogue text, performing unsupervised model training on the dialogue text, and obtaining a pre-trained language model;

Perform clustering processing on the dialogue text, obtain the cluster center according to the clustering result, and obtain the text closest to the cluster center as the intent center text;

Build a Prompt template based on the intent center text, input the dialogue text into the Prompt template slot, and output the Prompt construction text;

The Prompt constructed text is judged by the pre-trained language model whether the input dialogue text is consistent with the intent of the intent center text.

As a further improvement of the present invention, said acquiring the dialogue text, performing unsupervised model training on the dialogue text to obtain a pre-trained language model, further includes:

According to the dialogue text, construct a sentence pattern template, and automatically generate training corpus;

The dialogue text and the training corpus are trained to obtain a pre-trained language model.

As a further improvement of the present invention, the unsupervised model training is performed on the training dialogue text to obtain a pre-trained language model, which specifically includes:

A pre-trained language model is obtained by training the dialogue text through BERT, ELECTRA, and GPT models.

As a further improvement of the present invention, the clustering of the dialogue text specifically includes:

Carry out clustering to described dialogue text by K-means clustering algorithm, select cluster center K value, and divide text cluster class;

Select the text closest to the K value of the cluster center in each text cluster as the representative text in the text cluster, and combine the representative texts to generate the intent center text.

As a further improvement of the present invention, the selection of the cluster center K value specifically includes:

Calculate the cluster center silhouette coefficient, and select the optimal cluster center K value based on the size of the silhouette coefficient.

As a further improvement of the present invention, the clustering processing of the dialogue text also includes:

For the representative texts in the text clusters, the intent center text is formed by manually discriminating the text categories.

The present invention also provides a Prompt structural diagram recognition device combined with clustering, which includes:

A model training module, which is configured to acquire dialogue text, perform unsupervised model training on the dialogue text, and obtain a pre-trained language model;

A clustering module, which is configured to perform clustering processing on the dialogue text, obtain a clustering center according to the clustering result, and obtain the text closest to the clustering center as the text of the intention center;

A Prompt template construction module configured to construct a Prompt template based on the intent center text, input the dialogue text into the Prompt template slot, and output the Prompt construction text;

A judging module configured to use the Prompt constructed text to judge whether the input dialog text is consistent with the intent of the intent center text through the pre-trained language model.

The present invention also provides an electrical device, which includes:

memory for storing executable instructions;

The processor is configured to implement the above-mentioned Prompt structure intent recognition method combined with clustering when running the executable instructions stored in the memory.

The present invention also provides a refrigerator, which includes:

memory for storing executable instructions;

The processor is configured to implement the above-mentioned Prompt structure intent recognition method combined with clustering when running the executable instructions stored in the memory.

The present invention also provides a computer-readable storage medium, which stores executable instructions, and when the executable instructions are executed by a processor, the above-mentioned Prompt structure intent recognition method combined with clustering is implemented.

The beneficial effects of the present invention are: the present invention can recognize the text intent by constructing a zero-sample Prompt prompt structure without labeling a large number of text intent tags, thereby saving the cost of dialogue intent recognition. Moreover, in the process of constructing the prompt prompt structure, the clustering method is used to discover the intent center text, and the prompt prompt structure required for intent recognition is automatically constructed. It can avoid the problem of fluctuating intention prediction results when the prompt structure is artificially constructed.

Description of drawings

Fig. 1 is a schematic diagram of the steps of the method for identifying the structural intent of Prompt combined with clustering in an embodiment of the present invention.

Fig. 2 is a schematic diagram of the steps of clustering the dialogue text in an embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present invention and corresponding drawings. Apparently, the described embodiments are only some, not all, embodiments of the present invention. Based on the implementation manners in the present invention, all other implementation manners obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of the present invention.

Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

This embodiment provides a clustering-based Prompt structural intent recognition method. Prompt is prompt learning, which can add additional information to the input section of the language model by constructing a Prompt template without significantly changing the structure and parameters of the pre-trained language model. text, increase the information used for prompting, and perform intent recognition on downstream tasks. In this embodiment, by clustering the dialogue text, automatically generating the intent-centered text, and automatically constructing a semantic prompt template based on the intent-centered text, it is possible to avoid the problem of fluctuations in intent prediction results when manually constructing the prompt structure. In this implementation manner, this method is described based on a smart refrigerator, and the application of this method in other devices may refer to this implementation manner.

In the intelligent application of electrical equipment such as refrigerators, text classification is a common natural language processing task, which requires the correct intention judgment of the input text data. The intention judgment is to judge and identify the specific usage of the user based on the input corpus data Intent, for example, according to different usage intentions, the input corpus data can be divided into usage intentions belonging to several major fields as a whole, and each intent classification has corresponding training corpus for training the intent classification model. In this embodiment, for a smart refrigerator, the usage intention may include: recipe query, food reminder in the refrigerator, music playing, news broadcasting, etc.

As shown in Figure 1, the Prompt structural intent recognition method combined with clustering includes steps:

S1: Obtain the dialogue text, and perform unsupervised model training on the dialogue text to obtain a pre-trained language model.

S2: Perform clustering processing on the dialog text, obtain a cluster center according to the clustering result, and obtain the text closest to the cluster center as the intent center text.

S3: Construct a Prompt template based on the intent center text, input the dialog text into the slot of the Prompt template, and output the Prompt constructed text.

S4: Using the Prompt constructed text to judge whether the input dialogue text is consistent with the intent of the intent center text through the pre-trained language model.

In step S1, it specifically includes:

A pre-trained language model is obtained by training the dialogue text through BERT, ELECTRA, and GPT models.

The dialogue text mentioned here refers to the text transcribed by the user currently speaking to the smart electronic device or to the client terminal device communicated with the smart electronic device, etc., such as query or instructional sentences. As in this embodiment, the user can ask questions such as "what vegetables are in the refrigerator today", "what recipes are recommended today", or the user can send questions such as "remind the yogurt in the refrigerator that is about to expire", "give seasonal fruit" and other commands. Based on the above information, the processor of the smart refrigerator judges the user's usage intention after performing speech recognition through the method provided by the present invention.

Unsupervised model training refers to the ability of the model to learn from unlabeled data to data feature extraction, representation, and prediction, and to achieve the role of data enhancement. Unsupervised model training based on large-scale text library pre-training can better learn general language representation, which is helpful for downstream application tasks, and unsupervised language model can provide better model initialization methods, so as to obtain more A model with good generalization ability, and accelerates the convergence rate of the target task.

According to different language models used, different steps of text preprocessing are also included before model training. For example, when using the BERT model, the dialogue text is first processed into text data in the input format of the BERT model, and then the text data is filled to make the length of each text data consistent.

In other embodiments of the present invention, common unsupervised training models can also be used, and the steps of text preprocessing can be adjusted according to the model type.

Further, in some embodiments of the present invention, step S1 also includes:

According to the dialogue text, construct a sentence pattern template, and automatically generate training corpus;

The dialogue text and the training corpus are trained to obtain a pre-trained language model.

In actual use, due to the small amount of corpus in the user's real dialogue text or the uneven distribution of the corpus of different intent categories, it is necessary to automatically generate corpus data as supplementary text data for model training. For the corpus generation method, reference may be made to the prior art, which will not be repeated here.

By combining the actual user dialogue text and the large-scale corpus generated based on it to train the language model, you can

As shown in Figure 2, in step S2, it specifically includes:

S21: Clustering the dialogue texts through a K-means clustering algorithm, selecting the best clustering center K value through the silhouette coefficient and dividing the text clusters.

S22: Select the text closest to the optimal cluster center K value in each text cluster as the representative text in the text cluster, and combine the representative texts to generate the intent center text.

The intention-centered text mentioned here is the structured text in a type of dialogue text that can best reflect the intention of this type of dialogue text, and it can be regarded as representing the intention of this type of dialogue text.

By clustering the dialogue text and taking the text closest to the K value of the cluster center as the most representative text in the text cluster, it is possible to effectively automatically generate an intent center that reflects the intention of the dialogue text for a large amount of text data The text generation method is simple and efficient with high accuracy, and in the subsequent process, the intention center text is constructed as a prompt template, which can effectively avoid the fluctuation of intention prediction results generated when the prompt template is manually constructed.

Specifically, in step S21, it specifically includes: performing preprocessing on the dialogue text such as word segmentation and removing stop words, calculating text features and constructing a vector space model, and performing clustering using the K-means algorithm.

Further, in step S21, by calculating the cluster center silhouette coefficient, based on the size of the silhouette coefficient, the maximum value of the silhouette coefficient is calculated, and the optimal cluster center K value is selected.

Further, in step S2, the clustering processing of the dialogue text also includes:

For the representative texts in the text clusters, the intent center text is formed by manually discriminating the text categories.

For the clustered text, the text category of the text is manually identified to further improve the text accuracy of the intent center text.

In other embodiments of the present invention, other commonly used clustering algorithms may also be used to cluster the dialog text.

In step S3, the Prompt template is constructed through the intent center text, which is described here based on a specific example. The central text of the intention obtained in step S2 is "put the eggplant into the refrigerator", then the constructed Prompt template is: [X] Is it consistent with putting the eggplant into the refrigerator? [MASK]. Among them, [X] is the input dialogue text, and [MASK] is used to indicate the prediction result, that is, to indicate consistency or inconsistency.

For the dialog text "Add potatoes to the refrigerator", input it into the above prompt template, and the output Prompt construction text is "Is adding potatoes to the refrigerator consistent with putting eggplants in the refrigerator? [MASK]". Input the text constructed by Prompt into the pre-trained language model to judge the sentence intent matching. If the output is consistent or inconsistent, the intent judgment of the dialogue text can be completed based on the intent center text.

Based on the same inventive idea, the present invention also provides a Prompt structural diagram recognition device combined with clustering, which includes:

A model training module, which is configured to acquire dialogue text, perform unsupervised model training on the dialogue text, and obtain a pre-trained language model;

A clustering module, which is configured to perform clustering processing on the dialogue text, obtain a clustering center according to the clustering result, and obtain the text closest to the clustering center as the text of the intention center;

A Prompt template construction module configured to construct a Prompt template based on the intent center text, input the dialogue text into the Prompt template slot, and output the Prompt construction text;

A judging module configured to use the Prompt constructed text to judge whether the input dialog text is consistent with the intent of the intent center text through the pre-trained language model.

Based on the same inventive idea, the present invention also provides an electrical device, which includes:

memory for storing executable instructions;

The processor is configured to implement the above-mentioned Prompt structure intent recognition method combined with clustering when running the executable instructions stored in the memory.

Based on the same inventive idea, the present invention also provides a refrigerator, which includes:

memory for storing executable instructions;

The processor is configured to implement the above-mentioned Prompt structure intent recognition method combined with clustering when running the executable instructions stored in the memory.

Based on the same inventive idea, the present invention also provides a computer-readable storage medium, which stores executable instructions, and is characterized in that, when the executable instructions are executed by a processor, the above-mentioned Prompt structure intent recognition method combined with clustering is realized. .

To sum up, in this embodiment, by constructing a zero-sample Prompt prompt structure, it is possible to identify the intent of the text without labeling a large number of text intent tags, thereby saving the cost of dialogue intent recognition. Moreover, in the process of constructing the prompt prompt structure, the clustering method is used to discover the intent center text, and the prompt prompt structure required for intent recognition is automatically constructed. It can avoid the problem of fluctuating intention prediction results when the prompt structure is artificially constructed.

It should be understood that although this description is described according to implementation modes, not each implementation mode only contains an independent technical solution, and this description in the description is only for clarity, and those skilled in the art should take the description as a whole, and each The technical solutions in the embodiments can also be properly combined to form other embodiments that can be understood by those skilled in the art.

The series of detailed descriptions listed above are only specific descriptions of feasible implementations of the present invention, and are not intended to limit the scope of protection of the present invention. Any equivalent implementation or change that does not depart from the technical spirit of the present invention All should be included within the protection scope of the present invention.

Claims (10)

1. A Prompt structure intention recognition method combined with clustering is characterized by comprising the following steps:

obtaining a dialogue text, and carrying out unsupervised model training on the dialogue text to obtain a pre-trained language model;

clustering the dialog texts, acquiring a clustering center according to a clustering result, and acquiring a text closest to the clustering center as an intention center text;

constructing a Prompt template based on the intention center text, inputting the dialog text into a Prompt template slot position, and outputting a Prompt construction text;

and judging whether the intentions of the input dialog text and the intention center text are consistent or not by the Prompt construction text through the pre-trained language model.

2. The method of claim 1, wherein the obtaining of the dialog text, the unsupervised model training of the dialog text to obtain a pre-trained language model, and the method further comprises:

constructing a sentence pattern template according to the dialogue text, and automatically generating a corpus for training;

and training the dialogue texts and the training speech materials to obtain a pre-trained language model.

3. The method for combined cluster Prompt structural intention recognition according to claim 1, wherein the unsupervised model training of the training dialog text to obtain a pre-trained language model specifically comprises:

and training the dialog text through BERT, ELECTRA and GPT models to obtain a pre-trained language model.

4. The method for identifying a Prompt structure with cluster according to claim 1, wherein the clustering process is performed on the dialog text, and specifically comprises:

clustering the dialog texts by a K-means clustering algorithm, selecting a clustering center K value, and dividing text clusters;

selecting the text closest to the K value of the cluster center in each text cluster class as the representative text in the text cluster class, and combining the representative texts to generate the intention center text.

5. The method for identifying a Prompt structure intention of a connected cluster according to claim 4, wherein the selecting a K value of a cluster center specifically comprises:

and calculating the contour coefficient of the clustering center, and selecting the optimal K value of the clustering center based on the size of the contour coefficient.

6. The method of Prompt structure intent recognition in conjunction with clustering of claim 5, wherein the clustering of the dialog text further comprises:

and forming the intention center text by manually judging the text category for the representative text in the text cluster class.

7. A Prompt structural intent recognition apparatus incorporating clustering, comprising:

the model training module is configured to acquire a dialogue text, and perform unsupervised model training on the dialogue text to obtain a pre-trained language model;

the clustering module is configured to perform clustering processing on the dialog text, acquire a clustering center according to a clustering result, and acquire a text closest to the clustering center as an intention center text;

a Prompt template construction module configured to construct a Prompt template based on the intention center text, input the dialog text into the Prompt template slot, and output a Prompt construction text;

a judging module configured to judge whether the inputted dialog text and the intention center text are consistent through the pre-trained language model.

8. An electrical device, comprising:

a memory for storing executable instructions;

a processor for implementing the method of Prompt structure intent recognition of a joined cluster of any of claims 1 to 6 when executing the executable instructions stored in the memory.

9. A refrigerator, characterized by comprising:

a memory for storing executable instructions;

a processor for implementing the method of Prompt structure intent recognition of a bonded cluster of any of claims 1 to 6 when executing the executable instructions stored in the memory.

10. A computer readable storage medium storing executable instructions, wherein the executable instructions when executed by a processor implement the method of Prompt structural intent recognition of a joined cluster of any of claims 1 to 6.

Images