CN118210926B

CN118210926B - Text label prediction method and device, electronic equipment and storage medium

Info

Publication number: CN118210926B
Application number: CN202410627771.7A
Authority: CN
Inventors: 刘岩
Original assignee: Shandong Yunhai Guochuang Cloud Computing Equipment Industry Innovation Center Co Ltd
Current assignee: Shandong Yunhai Guochuang Cloud Computing Equipment Industry Innovation Center Co Ltd
Priority date: 2024-05-21
Filing date: 2024-05-21
Publication date: 2024-08-20
Anticipated expiration: 2044-05-21
Also published as: CN118210926A

Abstract

The invention provides a text label prediction method, a device, electronic equipment and a storage medium, and relates to the field of machine learning, wherein the method comprises the following steps: acquiring a text to be detected, and determining neighbor texts similar to the text to be detected in a preset corpus; each text in the preset corpus is provided with a plurality of category labels; setting a class label of the neighbor text as a neighbor class label, and determining an associated class label corresponding to the neighbor class label by using preset label structure knowledge; the preset tag structure knowledge records the association relationship among category tags; setting the label text of the neighbor class label and the label text of the associated class label together as guiding knowledge; inputting the text to be tested and the guide knowledge into a label prediction model together for label prediction to obtain a class label corresponding to the text to be tested; the label prediction accuracy of the text to be detected can be improved, meanwhile, the long tail problem can be relieved, and the prediction effect of the tail label is improved.

Description

Text label prediction method and device, electronic equipment and storage medium

Technical Field

The present invention relates to the field of machine learning, and in particular, to a text label prediction method, a text label prediction device, an electronic device, and a storage medium.

Background

Text classification is one of the basic tasks of natural language processing, and specifically uses a label prediction model to determine the category to which the text belongs. Multi-tag text classification is an important subtask in the above text classification, which specifically associates a piece of text with a subset of tags that contain multiple tags. With the enrichment of text classification application scenes, the number of category labels is no longer in units of one hundred or more, but gradually in units of one thousand, ten thousand, and even one hundred thousand. And the task of Multi-tag text classification with a huge number of category tags can be generally called XMTC task (eXtreme Multi-label Text Classification, limit Multi-tag text classification). When dealing with a limited multi-label text classification task, a label prediction model is easy to have serious long tail problem, namely a small number of class labels are frequently labeled to a text (labels which are frequently labeled can be called head labels), and a large number of class labels are not easy to label (labels which are not easy to label can be called tail labels). Furthermore, this not only affects the accuracy of prediction of text labels, but also reduces the reliability of multi-label text classification.

Disclosure of Invention

The invention aims to provide a text label prediction method, a device, electronic equipment and a storage medium, which can not only improve label prediction accuracy of a text to be detected, but also relieve the problem of long tail and improve the prediction effect of tail labels.

In order to solve the technical problems, the invention provides a text label prediction method, which comprises the following steps:

acquiring a text to be detected, and determining a neighbor text corresponding to the text to be detected in a preset corpus; each text in the preset corpus is provided with a plurality of category labels, and the adjacent text is determined according to the similarity between the adjacent text and the text to be detected;

setting the class label of the neighbor text as a neighbor class label, and determining an associated class label corresponding to the neighbor class label by using preset label structure knowledge; wherein, the preset label structure knowledge records the association relation between class labels;

setting the label text of the neighbor class label and the label text of the associated class label together as guiding knowledge;

And inputting the text to be tested and the guide knowledge into a label prediction model together for label prediction to obtain a class label corresponding to the text to be tested.

Optionally, the preset tag structure knowledge is a tag hierarchy relationship and/or a tag cluster relationship.

Optionally, determining the association class label corresponding to the neighbor class label by using preset label structure knowledge includes:

determining an associated tag cluster in which the neighbor class tag is located in a plurality of tag clusters recorded in the tag cluster relation;

and setting other class labels except the neighbor class label in the associated label cluster as the associated class label.

Optionally, the construction process of the tag cluster relationship includes:

Acquiring text information of each type of tag; the text information is label text and/or description text of the category labels, and the description text contains interpretation information of the category labels;

Performing vector conversion on the text information to obtain tag embedded vectors of all types of tags;

And clustering the tag embedding vectors to obtain a plurality of tag clusters.

Searching a parent node of the node where the neighbor class label is located in a label tree for recording the label hierarchical relationship; wherein, the nodes in the label tree are in one-to-one correspondence with the category labels;

and setting the class label corresponding to the father node as the associated class label.

Optionally, determining the neighboring text corresponding to the text to be tested in the preset corpus includes:

performing vector conversion on the text to be detected to obtain an embedded vector of the text to be detected;

Determining the vector similarity between the text embedding vector to be detected and the text embedding vector of each text in the preset corpus;

and determining the neighbor text corresponding to the text to be tested according to the vector similarity.

Optionally, determining, according to the vector similarity, a neighboring text corresponding to the text to be tested, including:

And setting the text to which the text embedded vector corresponding to the maximum vector similarity belongs as the neighbor text corresponding to the text to be tested.

Optionally, performing vector conversion on the text to be tested to obtain an embedded vector of the text to be tested, including:

vector conversion is carried out on the text to be detected according to the following formula, and an embedded vector of the text to be detected is obtained:

；

Wherein, Representing the text to be tested,And c-th words of the text to be tested are represented, the Encode () represents a word vector embedding function, and the Len () represents a word number counting function.

Optionally, determining the vector similarity between the text embedding vector to be tested and the text embedding vector of each text in the preset corpus includes:

and determining cosine similarity between the text embedding vector to be detected and the text embedding vector according to the following formula:

；

Wherein, Representing the text-to-be-tested embedded vector,Representing the text-embedding vector;

And setting the cosine similarity as the vector similarity.

Optionally, before determining the neighboring text corresponding to the text to be tested in the preset corpus, the method further includes:

word segmentation is carried out on the text to be detected, and stop words in the text to be detected are removed, so that a processed text to be processed is obtained;

And entering the step of determining the neighbor text corresponding to the text to be tested in the preset corpus based on the processed text to be tested.

Optionally, the training process of the label prediction model includes:

Acquiring training texts from a training set, and determining neighbor training texts corresponding to the training texts in the preset corpus; the preset corpus is composed of a training set and a verification set, the training text is marked with a plurality of category labels, the training text is different from the neighbor training text, and the neighbor training text is determined according to the similarity between the training text and the neighbor training text;

setting the class label of the neighbor training text as a neighbor training class label, and determining an associated training class label corresponding to the neighbor training class label by utilizing the preset label structure knowledge;

setting the label text of the neighbor training class label and the label text of the associated training class label together as training guide knowledge;

The training text and the training guide knowledge are input into the label prediction model together to conduct label prediction, and a prediction type label corresponding to the training text is obtained;

and determining a loss value according to the labeled class label of the training text and the predicted class label, and updating model parameters of the label prediction model by using the loss value.

Optionally, the label prediction model comprises a coding module, a feature extraction module and an output module;

The text to be tested and the guide knowledge are input into a label prediction model together to conduct label prediction, and a class label corresponding to the text to be tested is obtained, and the method comprises the following steps:

Converting the text to be tested into a text embedded vector to be tested by using the coding module, and converting the guide knowledge into a guide knowledge embedded vector by using the coding module;

Extracting features of the text embedded vector to be detected by using the feature extraction module to obtain a text feature vector, and extracting features of the guide knowledge embedded vector by using the feature extraction module to obtain a guide knowledge feature vector; the feature extraction module comprises a network branch for extracting features of the text embedding vector to be detected and a network branch for guiding the knowledge embedding vector to extract features;

Fusing the text feature vector and the guide knowledge feature vector to obtain a fused feature vector;

and processing the fusion feature vector by using an output module to determine a category label corresponding to the text to be detected.

Optionally, performing feature extraction on the text embedded vector to be detected by using the feature extraction module to obtain a text feature vector, and performing feature extraction on the guide knowledge embedded vector by using the feature extraction module to obtain a guide knowledge feature vector, including:

Inputting the text embedding vector to be tested into a first text convolution network branch to perform feature extraction, so as to obtain the text feature vector;

and inputting the guide knowledge embedded vector into a second text convolution network branch to perform feature extraction, so as to obtain the guide knowledge feature vector.

Optionally, the first text convolution network branch and the second text convolution network branch have the same network structure.

Optionally, converting the guiding knowledge into a guiding knowledge embedding vector comprises:

Respectively carrying out vector conversion on the label text of each neighbor class label and the label text of each associated class label to obtain a word embedding vector corresponding to each neighbor class label and a word embedding vector of each associated class label;

and carrying out average processing on all word embedding vectors to obtain the guide knowledge embedding vector.

Optionally, fusing the text feature vector and the guiding knowledge feature vector to obtain a fused feature vector, including:

and splicing the text feature vector and the guide knowledge feature vector to obtain the fusion feature vector.

Optionally, processing the fusion feature vector by using an output module, and determining a category label corresponding to the text to be tested includes:

processing the fusion feature vector by using an output layer weight matrix and a regression function to obtain an output vector; wherein, each position of the output vector corresponds to the class label one by one, and each position of the output vector records the probability that the corresponding class label belongs to the text to be tested;

and setting the class label corresponding to the position, with the numerical value larger than the preset threshold value, in the output vector as the class label corresponding to the text to be detected.

The invention also provides a text label prediction device, which comprises:

The neighbor text searching module is used for acquiring a text to be detected and determining a neighbor text corresponding to the text to be detected in a preset corpus; each text in the preset corpus is provided with a plurality of category labels, and the adjacent text is determined according to the similarity between the adjacent text and the text to be detected;

The label searching module is used for setting the class label of the neighbor text as a neighbor class label and determining an associated class label corresponding to the neighbor class label by utilizing preset label structure knowledge; wherein, the preset label structure knowledge records the association relation between class labels;

The guiding knowledge setting module is used for setting the label text of the neighbor class label and the label text of the association class label together as guiding knowledge;

And the prediction module is used for inputting the text to be detected and the guide knowledge into a preset label prediction model together to perform label prediction, so as to obtain a class label corresponding to the text to be detected.

The present invention also provides an electronic device including:

a memory for storing a computer program;

And a processor for implementing the text label prediction method as described above when executing the computer program.

The present invention also provides a computer program product comprising computer programs/instructions which when executed by a processor implement the steps of the text label prediction method as described above.

The present invention also provides a computer readable storage medium having stored therein computer executable instructions that, when loaded and executed by a processor, implement the text label prediction method as described above.

The invention provides a text label prediction method, which comprises the following steps: acquiring a text to be detected, and determining a neighbor text corresponding to the text to be detected in a preset corpus; each text in the preset corpus is provided with a plurality of category labels, and the adjacent text is determined according to the similarity between the adjacent text and the text to be detected; setting the class label of the neighbor text as a neighbor class label, and determining an associated class label corresponding to the neighbor class label by using preset label structure knowledge; wherein, the preset label structure knowledge records the association relation between class labels; setting the label text of the neighbor class label and the label text of the associated class label together as guiding knowledge; and inputting the text to be tested and the guide knowledge into a label prediction model together for label prediction to obtain a class label corresponding to the text to be tested.

Therefore, before label prediction is carried out on the text to be detected, the method can search the neighbor text similar to the text to be detected in the preset corpus, and set the class label of the neighbor text as the neighbor class label. The neighbor class labels are part of guiding knowledge and are used for guiding and predicting actual class labels of texts to be detected. The method is characterized in that approximate semantics exist between the neighbor text and the text to be tested, and then the category labels of the neighbor text are more attached to the actual semantics of the text to be tested. In other words, the method and the device can dynamically select the neighbor class labels which are more attached to the text to be tested according to the actual semantic scene of the text to be tested. The present invention may then use the preset tag structure knowledge to determine the associated class tag of the neighbor class tag, and may use both the neighbor class tag and the associated class tag as guiding knowledge. The preset identification structure knowledge records the association relation among the category labels. By searching the associated category labels of the neighbor category labels according to the preset label structure knowledge and further introducing the associated category labels into the guide knowledge, the method and the device can increase the number of the category labels in the guide knowledge, and simultaneously can further introduce the association relation among the category labels into the guide knowledge, namely, the semantic radiation range of the guide knowledge can be enlarged, so that the semantic category boundary learning definition of the head label auxiliary tail label under the semantic radiation range of the guide knowledge is ensured, the prediction effect of the tail label is improved, and the long tail problem is relieved. Furthermore, the invention inputs the text to be tested and the guiding knowledge into the label prediction model to perform label prediction, so that the label prediction accuracy of the text to be tested can be improved, the long tail problem can be reduced, and the prediction effect of the tail label can be improved. The invention also provides a text label predicting device, electronic equipment and a computer readable storage medium, which have the beneficial effects.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flowchart of a text label prediction method according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a process flow for classifying limited multi-label text according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of data set partitioning in a training phase of a label prediction model according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of a frame for improving the classification of text with multiple labels according to an embodiment of the present invention;

Fig. 5 is a block diagram of a text label predicting device according to an embodiment of the present invention;

Fig. 6 is a block diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Text classification is one of the basic tasks of natural language processing, and specifically uses a label prediction model to determine the category to which the text belongs. Multi-tag text classification is an important subtask in the text classification task described above, which specifically associates a piece of text with a subset of tags that contain multiple tags. With the enrichment of text classification application scenes, the number of category labels is no longer in units of one hundred or more, but gradually in units of one thousand, ten thousand, and even one hundred thousand. And the task of Multi-tag text classification with a huge number of category tags can be generally called XMTC task (eXtreme Multi-label Text Classification, limit Multi-tag text classification). When dealing with a limited multi-label text classification task, a label prediction model is easy to have serious long tail problem, namely a small number of class labels are frequently labeled to a text (labels which are frequently labeled can be called head labels), and a large number of class labels are not easy to label (labels which are not easy to label can be called tail labels). Furthermore, this not only affects the accuracy of prediction of text labels, but also reduces the reliability of multi-label text classification.

In view of this, the present invention may provide a text label prediction method, which may search for a neighboring text similar to a text to be measured in advance, set a class label of the neighboring text as a neighboring class label, and search for other associated class labels associated with the neighboring class label, and further use the neighboring class label and the associated class label to compose a guiding knowledge corresponding to the text to be measured, and guide label prediction for the text to be measured by using the knowledge, so as to not only improve label prediction accuracy of the text to be measured, but also alleviate long tail problem and improve prediction effect of tail labels.

It should be noted that, the embodiment of the present invention is not limited to the hardware device for executing the method, and may be set according to the actual application requirement, for example, may be a personal computer, a server, etc.

It should be further noted that the method provided by the embodiment of the present invention is applicable to various text classification scenes, for example, the scenes may be emotion classification scenes, theme classification scenes, and the like. In addition, the invention can be applied to the field of health care, and the knowledge in the field of medicine is modeled by using the tag knowledge, so that the classification performance can be improved by using the tag knowledge, and more accurate and reliable diagnosis results can be realized, thereby helping doctors to diagnose diseases more quickly and accurately. The method and the system can be applied to the field of electronic commerce recommendation systems, and because the shopping demands of users are complex and changeable, and the demands and interests of different users are different, the method and the system can accurately understand the demands of the users and recommend proper commodities for the users, and help an electronic commerce platform to improve the accuracy and recommendation effect of product label prediction. The method can be applied to the field of social media, such as social media, and users can describe contents published by the users by using various labels or keywords. In other words, the text processed by the method can be emotion classification text, theme classification text, medical text, e-commerce text, social media text and the like; similarly, the category labels related to the invention can be emotion classification labels, theme classification labels, medical category labels, e-commerce category labels, social media category labels and the like, and can be set according to actual application requirements.

For convenience of understanding, please refer to fig. 1, fig. 1 is a flowchart of a text label prediction method according to an embodiment of the present invention, where the method may include:

S101, acquiring a text to be detected, and determining a neighbor text corresponding to the text to be detected in a preset corpus; each text in the preset corpus is provided with a plurality of category labels, and the adjacent text is determined according to the similarity between the adjacent text and the text to be detected.

In this embodiment, the text to be tested is a text waiting for label prediction, which may be a text input by a user. In the related art, a process of performing label prediction on a text to be tested may refer to fig. 2, and fig. 2 is a schematic diagram of a process of classifying a limited multi-label text according to an embodiment of the present invention. The text to be tested is directly input into a classifier to conduct label prediction after preprocessing and text representation conversion, wherein the text representation is usually vector representation of the text, and the classifier is a label prediction model. Unlike the related art, in this embodiment, after the text to be tested is obtained, the text to be tested is not immediately subjected to label prediction, but guide knowledge is first built for the text to be tested, and then the guide knowledge is used for assisting in predicting the class labels of the text to be tested. The guiding knowledge comprises fine-grained category label information similar to the text to be tested in terms of semantics. In order to construct the guiding knowledge, a preset corpus is specifically added in the embodiment, and in this step, a neighbor text similar to the text to be tested is first determined in the preset corpus. The corpus is preset with a plurality of texts, and each text is provided with a plurality of types of labels. It can be understood that the category labels set by the texts in the preset corpus are verified, i.e. the category labels set by the texts all belong to the true labels of the texts. In other words, the preset corpus stores the prior knowledge required for label prediction. And the neighbor text is a text with similarity to the text to be tested. The purpose of finding neighbor text is to: the neighbor text has semantic similarity with the text to be tested, so that the class label of the neighbor text is closer to the actual semantic of the text to be tested, and a good guiding effect on label prediction can be achieved. In other words, the embodiment can dynamically match the neighbor class label for the text to be tested according to the actual semantics of the text to be tested so as to guide label prediction for the text to be tested based on priori knowledge, thereby effectively improving the label prediction accuracy of the text to be tested.

Of course, the text to be tested can be preprocessed before searching the neighbor text, so that the text to be tested can be conveniently encoded, and the influence of useless information in the text to be tested on subsequent steps can be avoided. The embodiment is not limited to a specific pretreatment mode, and may be selected according to actual application requirements. For example, preprocessing may include word segmentation, and may also include removal of stop words, where stop words refer to words that frequently occur in text but that do not contribute significantly to meaning, such as stop words, function words, punctuation marks, and the like. For the removal of stop words, a stop word dictionary can be preset, and words recorded in the preset stop word dictionary in the text to be tested can be removed.

Based on this, before determining the neighboring text similar to the text to be tested in the preset corpus, the method may further include:

step 11: word segmentation is carried out on the text to be detected, and stop words in the text to be detected are removed, so that a processed text to be processed is obtained;

Step 12: and entering a step of determining neighbor texts corresponding to the text to be tested in a preset corpus based on the processed text to be tested.

Further, the number of texts included in the corpus is not limited, and may be set according to actual application requirements. It can be appreciated that the more text is contained in the preset corpus, the higher the contribution to the label prediction. The number of the recordable class labels and label content in the preset corpus are not limited, and the label content can be set according to actual application requirements. It should be further noted that the preset corpus is provided with a total category label set, and the total category label set may include all category labels appearing in the corpus; in addition, a category label set corresponding to each text in the corpus is preset to be a subset of the total category label set, wherein the category label set records all category labels set by the corresponding text; in addition, the number and variety of category labels set by different texts may be different. In addition, in order to facilitate text searching, text embedded vectors corresponding to the texts can be set in the preset corpus.

Further, in order to conveniently record the category labels marked by the texts in the preset corpus, a label vector with the same dimension can be set for all the texts in the corpus, each position in the label vector corresponds to each category label one by one, and the value of each position is 1 or 0. When the value of one position in the label vector is 1, the text corresponding to the label vector is provided with the label corresponding to the position; when the value of a position in the label vector is 0, the text corresponding to the label vector is not provided with the label corresponding to the position. For example, if the tag vector of a certain text is [0,1,0], and three positions in the tag vector sequentially correspond to the tag a, the tag B and the tag C, it can be determined that the text is only labeled with the tag B according to the values of the three positions of the tag vector. Text ofCan be defined as: wherein L represents both the total number of categories of category labels and the label vector Is a length of (c). Label vectorEach dimension of the list corresponds to a category label, and when the j-th category label isAnd textIn the case of the association with each other,=1。

Further, the embodiment is not limited to how to find the neighboring text corresponding to the text to be tested, for example, the neighboring text containing the same vocabulary can be directly matched based on the vocabulary contained in the text to be tested; the text to be tested can be converted into the text embedded vector to be tested, and the text in the corpus is converted into the text embedded vector, so that the similarity between the text embedded vector to be tested and the text embedded vector is determined, and the neighbor text of the text to be tested is searched from the corpus from the dimension of the embedded vector. In consideration of better reliability of searching based on the embedded vector, the embodiment can convert the text to be detected into the embedded vector of the text to be detected, convert the text in the corpus into the embedded vector of the text, and further search the neighbor text of the text to be detected by determining the similarity between the embedded vector of the text to be detected and the embedded vector of the text.

Based on this, determining the neighboring text corresponding to the text to be tested in the preset corpus may include:

Step 21: performing vector conversion on the text to be detected to obtain an embedded vector of the text to be detected;

Step 22: determining the vector similarity between the text embedding vector to be detected and the text embedding vector of each text in the preset corpus;

step 23: and determining the neighbor text corresponding to the text to be tested according to the vector similarity.

In this embodiment, the text to be tested is an embedded vector corresponding to the text to be tested, and the text embedded vector is an embedded vector corresponding to each text in the preset corpus. It can be understood that the text to be tested and the text of the preset corpus need to be subjected to vector conversion in the same vector conversion mode. The embodiment is not limited to a specific vector conversion method, and may be selected according to practical application requirements. For example, performing vector conversion on the text to be tested to obtain an embedded vector of the text to be tested may include:

step 31: vector conversion is carried out on the text to be detected according to the following formula, and an embedded vector of the text to be detected is obtained:

；

It should be noted that in the moleculeFor textEncoding all the words of (a); in denominator ofFor weakening the effect of text length on coding. The text to be detected is converted into the text embedded vector to be detected by adopting the formula, the situations that a longer text is only matched with a longer text and a shorter text is only matched with a shorter text can be avoided as much as possible, and therefore the reliability of searching neighbor texts can be ensured. Note that, the present embodiment is not limited to a specific word vector embedding function, and reference may be made to a related art of word vector embedding.

Further, the embodiment of the invention is not limited to how to test the vector similarity between the text embedded vector and the text embedded vector, and can refer to the related technology of vector similarity calculation. For example, the similarity of the two may be determined based on the manner in which the cosine similarity between the vectors is determined.

Based on this, determining the vector similarity between the text-embedded vector to be tested and the text-embedded vectors of each text in the preset corpus may include:

Step 41: and determining cosine similarity between the text embedded vector to be tested and the text embedded vector according to the following formula:

；

Wherein, Representing the text-to-be-tested embedded vector,Representing the text-embedded vector.

Step 42: the cosine similarity is set as the vector similarity.

Further, the present embodiment does not limit how to determine a neighboring text similar to the text to be tested according to the vector similarity. For example, texts with vector similarity greater than a preset threshold value can be embedded into the text in the vector and all set as neighbor texts, i.e. the text to be tested can correspond to a plurality of neighbor texts; the text to which the text embedded vector corresponding to the maximum vector similarity belongs can also be set as the neighbor text of the text to be tested, i.e. the text to be tested corresponds to a single neighbor text. Considering that matching a single neighbor text (i.e., nearest neighbor text) for a text under test can promote pertinence of guiding knowledge, in this embodiment, only the text to which the text embedded vector corresponding to the maximum vector similarity belongs may be set as the neighbor text of the text under test.

Based on this, determining the neighbor text corresponding to the text to be tested according to the vector similarity may include:

step 51: and setting the text to which the text embedded vector corresponding to the maximum vector similarity belongs as the neighbor text corresponding to the text to be tested.

S102, setting a class label of a neighbor text as a neighbor class label, and determining an associated class label corresponding to the neighbor class label by using preset label structure knowledge; the preset tag structure knowledge records the association relationship among the category tags.

In this step, in addition to setting the class label of the neighbor text as the neighbor class label, the neighbor class label is used as the guiding knowledge, and the association class label corresponding to the neighbor class label can be further determined by using the preset label structure knowledge, so as to further add the association class label in the guiding knowledge. The preset tag structure knowledge records the association relationship between category tags, and the association relationship can be the subordinate association relationship between category tags, for example, for a 'sports item' tag and a 'skiing' tag, the 'skiing' tag belongs to a 'sports item' tag because skiing is a sports item, namely, the 'skiing' tag and the 'sports item' tag have the subordinate association relationship (namely, father-son relationship); the association relationship can also be a semantic association relationship between category labels, for example, a plurality of category labels correspond to male users or female users. It can be understood that, for the tag subordinate association relationship, the higher the tag hierarchy (such as a sports item), the wider the semantic coverage of the tag hierarchy, the easier it is to be labeled in tag prediction, i.e. the easier it is to be a frequently labeled head tag; the lower the label level (e.g., skiing), the narrower its semantic coverage, the less easily labeled in label prediction, i.e., the more easily becomes a tail label that is not easily labeled. Therefore, the association class labels associated with the neighbor class labels are searched based on the label subordinate association relation, and the neighbor class labels and the association class labels are set as guiding knowledge, so that the association between the tail label and the head label can be enhanced, the semantic category boundary of the tail label can be learned and mined by utilizing the head label auxiliary label prediction model, the tail label prediction effect can be improved, and the long tail problem is relieved. Similarly, for the semantic association relationship of the labels, as the head label and the tail label can have semantic similarity, the association class label associated with the neighbor class label is searched based on the semantic association relationship of the labels, and the neighbor class label and the association class label are set as guiding knowledge, so that the association between the tail label and the head label can be enhanced, and the effects of improving the prediction effect of the tail label and relieving the long tail problem can be achieved.

It should be noted that, the above-mentioned label subordinate association relationship may also be referred to as a label hierarchy relationship in this embodiment, and the above-mentioned label semantic association relationship may also be referred to as a label cluster relationship in this embodiment. In other words, the preset tag structure knowledge may be a tag hierarchy relationship, a tag cluster relationship, or a combination of a tag hierarchy relationship and a tag cluster relationship. It should be noted that, in this embodiment, the setting manner of the tag hierarchy relationship and the tag cluster relationship is not limited, and may be set according to actual application requirements.

The manner in which the associated category labels are determined based on the relationship of the label clusters is described below. Based on this, determining the association class label corresponding to the neighbor class label by using the preset label structure knowledge may include:

Step 61: determining an associated tag cluster in which the neighbor class tag is located in a plurality of tag clusters recorded in the tag cluster relation;

Step 62: and setting other class labels except the neighbor class label in the associated label cluster as associated class labels.

In this embodiment, a plurality of tag clusters are set in the tag cluster relationship, and all category tags with semantic similarity may form one cluster. Therefore, when the association type label is set, the association type label cluster where the neighbor type label is located is only determined in the plurality of label clusters recorded by the label cluster relation, and other type labels except the neighbor type label in the association type label cluster are set as the association type label.

It should be noted that, this embodiment is not limited to how to construct the tag clusters, and reference may be made to the related art of the tag clusters and clustering.

The manner in which the associated category labels are determined based on the label hierarchical relationship is described below. Based on this, determining the association class label corresponding to the neighbor class label by using the preset label structure knowledge may include:

Step 71: searching a parent node of a node where the neighbor class label is located in a label tree of the record label hierarchical relationship; wherein, the nodes in the label tree are in one-to-one correspondence with the category labels;

step 72: and setting the class label corresponding to the father node as an associated class label.

It is understood that the tag hierarchy relationship may be recorded in a tree structure. Each node in the label tree recorded with the label hierarchical relationship corresponds to each class label one by one, and the connection relationship between the nodes represents the subordinate relationship between the class labels. Furthermore, to promote the association between the tail label and the head label, the embodiment may search the parent node of the node where the neighbor class label is located, and set the class label corresponding to the parent node as the association class label.

S103, setting the label text of the adjacent class label and the label text of the associated class label as guiding knowledge.

S104, inputting the text to be tested and the guide knowledge into a label prediction model together for label prediction, and obtaining a class label corresponding to the text to be tested.

After the guiding knowledge is obtained, the text to be detected and the guiding knowledge can be input into the label prediction model together to conduct label prediction, so that accuracy of label prediction is improved by using the guiding knowledge, and the long tail degree can be reduced.

It should be noted that, the specific structure of the label prediction model is not limited in this embodiment, and the related technology of the text classification model may be referred to, and may be set according to the actual application requirement. The embodiment is also not limited to how to input the text to be tested and the guide knowledge into the label prediction model, for example, the text to be tested and the guide knowledge can be input independently, the text to be tested and the guide knowledge can be input after being fused, and the setting can be performed according to the actual application requirements.

Based on the embodiment, before label prediction is performed on the text to be detected, the method can search the neighbor text similar to the text to be detected in the preset corpus, and set the class label of the neighbor text as the neighbor class label. The neighbor class labels are part of guiding knowledge and are used for guiding and predicting actual class labels of texts to be detected. The method is characterized in that approximate semantics exist between the neighbor text and the text to be tested, and then the category labels of the neighbor text are more attached to the actual semantics of the text to be tested. In other words, the method and the device can dynamically select the neighbor class labels which are more attached to the text to be tested according to the actual semantic scene of the text to be tested. The present invention may then use the preset tag structure knowledge to determine the associated class tag of the neighbor class tag, and may use both the neighbor class tag and the associated class tag as guiding knowledge. The preset identification structure knowledge records the association relation among the category labels. By searching the associated category labels of the neighbor category labels according to the preset label structure knowledge and further introducing the associated category labels into the guide knowledge, the method and the device can increase the number of the category labels in the guide knowledge, and simultaneously can further introduce the association relation among the category labels into the guide knowledge, namely, the semantic radiation range of the guide knowledge can be enlarged, so that the semantic category boundary learning definition of the head label auxiliary tail label under the semantic radiation range of the guide knowledge is ensured, the prediction effect of the tail label is improved, and the long tail problem is relieved. Furthermore, the invention inputs the text to be tested and the guiding knowledge into the label prediction model to perform label prediction, so that the label prediction accuracy of the text to be tested can be improved, the long tail problem can be reduced, and the prediction effect of the tail label can be improved.

Based on the above embodiment, the following describes a manner of setting the tag cluster relationship. Based on this, the construction process of the tag cluster relationship may include:

S201, acquiring text information of each type of tag; the text information is label text and/or description text of the category labels, and the description text contains interpretation information of the category labels.

The present embodiment will construct a tag cluster based on the text information of each type of tag. The text information is label text and/or description text of the category labels, and the description text contains interpretation information of the category labels. For example, for the label "skiing", the label text is "skiing", and the description text may be "skiing sport is a racing sport in which an athlete wears a ski on a boot bottom to perform speed, jump, and slide down on a snow surface. It can be seen that the description text contains more semantic information than the label text, which has an important role in determining the degree of semantic association between category labels.

S202, carrying out vector conversion on the text information to obtain tag embedded vectors of tags of all categories.

It should be noted that, the present embodiment is not limited to how to perform vector conversion on text information, and reference may be made to related art of embedding vectors.

S203, clustering is carried out on the tag embedding vectors to obtain a plurality of tag clusters.

It should be noted that, the embodiment is not limited to how to perform clustering processing on the tag embedded vector, for example, a clustering algorithm suitable for text data such as a K-means clustering algorithm and a hierarchical clustering algorithm may be used to cluster the tag set to generate a tag cluster.

Based on the above embodiments, a training process of the tag prediction model is described below. Based on this, the training process of the label prediction model may include:

S301, acquiring training texts from a training set, and determining neighbor training texts corresponding to the training texts in a preset corpus; the corpus is composed of a training set and a verification set, the training text is marked with a plurality of category labels, the training text is different from the neighbor training text, and the neighbor training text is determined according to the similarity between the training text and the neighbor training text.

It will be appreciated that the label predictive model requires training using a training set that contains multiple training texts, each labeled with multiple category labels. It will also be appreciated that the training of the tag prediction model is similar to that of tag prediction using the model, i.e. it is necessary to generate guide knowledge for the training text and to input both the training text and the guide knowledge into the tag prediction model for processing. It should be noted that the pre-set corpus of the model training phase may be composed of the training set and the validation set, considering that the text-class label pairs contained in both the training set and the validation set (the validation set being the same form as the training set) are validated. It should be noted that when searching the neighbor training text of the training text in the preset corpus, it is required to ensure that the training text is different from the neighbor training text, so as to ensure the training effect.

S302, setting the class label of the neighbor training text as a neighbor training class label, and determining the associated training class label corresponding to the neighbor training class label by using preset label structure knowledge.

S303, setting the label text of the adjacent training class labels and the label text of the associated training class labels as training guide knowledge.

S304, the training text and the training guide knowledge are input into a label prediction model together to conduct label prediction, and a prediction type label corresponding to the training text is obtained.

It should be noted that the descriptions of the definitions of steps S302 to S304 are identical to the descriptions of the definitions of steps S102 to S104, and will not be repeated here.

And S305, determining a loss value according to the labeled class labels and the predicted class labels of the training texts, and updating model parameters of the label prediction model by using the loss value.

In this step, since the model parameters of the untrained label prediction model are not adjusted and optimized, and the required prediction effect cannot be achieved, it is necessary to determine a loss value according to the labeled class label and the predicted class label of the training text, and update the model parameters of the label prediction model by using the loss value. It is understood that steps S301 to S305 may be performed multiple times to train the tag prediction model by using all training texts.

It should be noted that, the embodiment of the present invention is not limited to the calculation method of the loss value and the optimization method of the model parameter, and reference may be made to the related art of machine learning.

Of course, in the training stage of the label prediction model, in order to verify the training effect of the model, a test set may be further set, so as to verify the label prediction effect of the model by using the text in the test set. The training set, the verification set and the test set in the embodiment can all come from the same data set and are divided according to different data dividing proportions. The embodiment is not limited to a specific data set dividing manner, and may be set according to actual application requirements. Referring to fig. 3, fig. 3 is a schematic diagram illustrating data set partitioning in a training phase of a label prediction model according to an embodiment of the present invention. Wherein the data set is to be split into a training set, a validation set and a test set. During the training phase, textFrom a training set, the textApproximate training text to be approximated in training set and verification set. In the test and verification link, textBut also from the test set and the validation set.

Based on the above embodiments, the process of providing a priori knowledge guidance for text using guidance knowledge is similar to the teacher driven mechanism (Teacher Forcing) in recurrent neural network training. However, this mechanism has a problem that the label prediction model may depend too much on the instruction of a "teacher" in the training process, resulting in unstable prediction results and even reduced effects. In order to alleviate this problem, the structure of the label prediction model may be further improved in this embodiment, so as to ensure that the guide knowledge lifting strategy adopts a model architecture with dual-flow input, and information may be learned from two branches of text features and guide knowledge at the same time, instead of relying on the guide knowledge singly, so that the model may be more independent in the learning process, and the dependence on "teacher" guidance may be reduced. Meanwhile, the two branches provide information of different sources, the information can be mutually supplemented and corrected, the text features and the guiding knowledge are subjected to network feature extraction and modeling, feature interaction is performed in a high-dimensional semantic space, and finally label prediction is performed, so that the problem that the label prediction may be too dependent on the guiding knowledge is solved. Based on this, the following describes in detail the structural improvement of the label prediction model and the prediction processing procedure. Based on the label prediction model, the label prediction model comprises a coding module, a feature extraction module and an output module; the text to be tested and the guiding knowledge are input into a label prediction model together to conduct label prediction, and category labels corresponding to the text to be tested are obtained, and the method can comprise the following steps:

S401, converting the text to be tested into a text embedded vector to be tested by using the coding module, and converting the guide knowledge into a guide knowledge embedded vector by using the coding module.

The step may first convert both the text to be tested and the guide knowledge into embedded vector form. The embodiment of the invention is not limited to a specific embedded vector conversion mode, and can refer to the related technology and set according to actual application requirements. In addition, the coding module can adopt different branches to respectively carry out vector conversion on the text to be tested and the guiding knowledge.

Further, as described above, the guiding knowledge may include a neighbor class label and an associated class label, so the process of converting the guiding knowledge into a guiding knowledge embedded vector is equivalent to vector fusing the neighbor class label and the associated class label. It should be noted that, the present embodiment is not limited to how to convert the guiding knowledge into the guiding knowledge embedding vector, for example, the label text of each neighboring class label and the label text of each associated class label may be subjected to vector conversion to obtain a word embedding vector corresponding to each neighboring class label and a word embedding vector of each associated class label, and all word embedding vectors may be subjected to average processing to obtain the guiding knowledge embedding vector.

Of course, when all word embedding vectors are fused into the guide knowledge embedding vector, the word embedding vector of the neighbor class label and the word embedding vector of the associated class label can be weighted and fused to adjust the contribution degree of the neighbor class label and the associated class label in guide label prediction, and can be set according to actual application requirements.

Based on this, converting the bootstrapping knowledge into a bootstrapping knowledge embedding vector may include:

step 81: respectively carrying out vector conversion on the label text of each neighbor class label and the label text of each associated class label to obtain a word embedding vector corresponding to each neighbor class label and a word embedding vector of each associated class label;

step 82: and carrying out average processing on all word embedding vectors to obtain the guide knowledge embedding vector.

Specifically, the process of constructing the guide knowledge embedding vector according to the tag hierarchy relationship may be: text ofObtaining nearest neighbor text through nearest neighbor searching stageAfter that, can obtainCorresponding tag vector. First, for the nearest neighbor tag setInitializing; subsequently, the tag vector is appliedThe label corresponding to the upper dimension 1 and the parent label are put inIn, obtain the labelParent tag of (a); Finally, word vector embedding is carried out on the label text corresponding to each label, and summation and average are carried out, so that guiding knowledge is finally obtainedIs embedded in matrix of (a)，Is thatThe corresponding tag text.

The steps of constructing the guide knowledge of the tag cluster structure knowledge are as follows: firstly, clustering labels by using a clustering algorithm such as K-means clustering and hierarchical clustering to form label clusters with certain relevance, and ensuring that the labels in each label cluster have certain relevance semantically; secondly, a nearest neighbor searching method is adopted to realize the association between the text and the tag clusters, and the tag clusters closely related to the text are found; finally, word vector embedding is carried out on each label cluster to form a guiding knowledge embedding vector.

S402, carrying out feature extraction on the text embedded vector to be detected by utilizing a feature extraction module to obtain a text feature vector, and carrying out feature extraction on the guide knowledge embedded vector by utilizing a feature extraction module to obtain a guide knowledge feature vector; the feature extraction module comprises a network branch for extracting features of the text embedding vector to be detected and a network branch for guiding the knowledge embedding vector to extract features.

In this step, to ensure that feature extraction for the text to be tested and feature extraction for the guide knowledge are performed independently and do not interfere with each other, the feature extraction module may include a network branch that performs feature extraction on the text embedded vector to be tested and a network branch that performs feature extraction on the guide knowledge embedded vector separately. Furthermore, the text embedding vector to be detected and the guiding knowledge embedding vector can be input into corresponding network branches to perform feature extraction, so that the problem that tag prediction is too dependent on guiding knowledge is solved.

It should be noted that, the embodiment of the present invention is not limited to the specific type of the network branches, and may be set according to the actual application requirements. For example, the network branches may be text convolutional networks (TextCNN), i.e., the text-to-be-tested embedded vectors may correspond to a first text convolutional network branch and the guided knowledge embedded vectors may correspond to a second text convolutional network branch.

Based on the above, the feature extraction module is utilized to perform feature extraction on the text embedded vector to be detected to obtain a text feature vector, and the feature extraction module is utilized to perform feature extraction on the guide knowledge embedded vector to obtain a guide knowledge feature vector, which comprises the following steps:

Step 91: and inputting the text embedding vector to be tested into a first text convolution network branch to perform feature extraction, so as to obtain a text feature vector.

Specifically, the process of extracting the features of the text embedding vector to be detected can be expressed as follows:

；

Wherein, Representing text to be testedIs used to determine the text feature vector of (c),Representing a first branch of the text convolution network,Representing the text-to-be-tested embedded vector.

Step 92: and inputting the guide knowledge embedded vector into a second text convolution network branch to perform feature extraction, so as to obtain a guide knowledge feature vector.

Specifically, the process of feature extraction of the guide knowledge embedding vector can be expressed as:

；

Wherein, Representing guided knowledgeIs a guide knowledge feature vector of (a),Representing a second branch of the text convolution network,Representing the guide knowledge embedding vector.

It should be noted that, the specific structures of the first text convolution network branch and the second text convolution network branch are not limited in this embodiment, and reference may be made to the related art of the text convolution network. It is noted that to avoid the influence of the difference in network branch structure on the subsequent processing, the first text convolution network branch and the second text convolution network branch have the same network structure.

S403, fusing the text feature vector and the guide knowledge feature vector to obtain a fused feature vector.

In this embodiment, after feature extraction of the text to be tested and the guide knowledge is completed separately, the text feature vector and the guide knowledge feature vector may be fused, so as to determine a category label corresponding to the text to be tested based on the obtained fused feature vector. The embodiment is not limited to a specific fusion mode, and may be, for example, average fusion, weighted fusion, splicing fusion, etc. In order to preserve the feature information in each feature vector as much as possible, the text feature vector and the guide knowledge feature vector can be fused in a splicing mode.

Based on this, fusing the text feature vector and the guide knowledge feature vector to obtain a fused feature vector may include:

Step 1001: and splicing the text feature vector and the guide knowledge feature vector to obtain a fusion feature vector.

Specifically, the text feature vector and the guide knowledge feature vector are spliced, which can be expressed as:

。

Wherein O represents the fusion feature vector.

S404, processing the fusion feature vector by using an output module, and determining a category label corresponding to the text to be tested.

In this step, the fused feature vector may be specifically processed by using an output layer weight matrix and a regression function (such as a Softmax function), so as to obtain a class label of the text to be tested. Specifically, the fusion feature vector can be processed by using an output layer weight matrix and a regression function to obtain an output vector; wherein, each position of the output vector corresponds to the class label one by one, and each position of the output vector records the probability that the corresponding class label belongs to the text to be tested; and then, setting the category label corresponding to the position with the value larger than the preset threshold value in the output vector as the category label corresponding to the text to be tested so as to complete label prediction.

Based on the above, the output module is utilized to process the fusion feature vector, and the category label corresponding to the text to be tested is determined, which comprises the following steps:

step 1101: processing the fusion feature vector by using the output layer weight matrix and the regression function to obtain an output vector; wherein, each position of the output vector corresponds to the class label one by one, and each position of the output vector records the probability that the corresponding class label belongs to the text to be tested;

step 1102: and setting the class label corresponding to the position with the value larger than the preset threshold value in the output vector as the class label corresponding to the text to be detected.

Specifically, the process of processing the fusion feature vector by using the output layer weight matrix and the regression function can be expressed as:

；

wherein Y represents an output vector, Representing the output layer weight matrix.

Based on the above embodiment, the text label prediction method is described below based on a specific policy framework. Referring to fig. 4, fig. 4 is a schematic diagram of a limited multi-label text classification lifting framework according to an embodiment of the invention. The limit multi-label text classification lifting framework based on label knowledge mainly comprises two parts: and guiding knowledge to generate a strategy and guiding knowledge to promote the strategy. Wherein, the guiding knowledge generates a strategy part, textFirst search the data set for its nearest neighbor textBased on nearest neighbor textIs of (3)Modeling of the guiding knowledge is performed by using the tag structure knowledge and the tag content knowledge to generate guiding knowledgeIs embedded in matrix of (a); Guiding the knowledge lifting strategy part to textAnd guiding knowledgeIs embedded in matrix of (a)And respectively inputting the two output vectors into different TextCNN network branches to perform feature extraction, splicing the two output vectors, and finally predicting the label.

The text label predicting device, the electronic device and the computer readable storage medium provided in the embodiments of the present invention are described below, and the text label predicting device, the electronic device and the computer readable storage medium described below and the text label predicting method described above may be referred to correspondingly.

Referring to fig. 5, fig. 5 is a block diagram of a text label predicting device according to an embodiment of the present invention, where the device may include:

the neighbor text searching module 501 is configured to obtain a text to be tested, and determine a neighbor text corresponding to the text to be tested in a preset corpus; each text in a preset corpus is provided with a plurality of category labels, and the adjacent text is determined according to the similarity between the adjacent text and the text to be detected;

The tag searching module 502 is configured to set a class tag of the neighbor text as a neighbor class tag, and determine an associated class tag corresponding to the neighbor class tag by using preset tag structure knowledge; the preset tag structure knowledge records the association relationship among category tags;

A guiding knowledge setting module 503, configured to set the label text of the neighbor class label and the label text of the association class label together as guiding knowledge;

the prediction module 504 is configured to input the text to be tested and the guiding knowledge together into a preset label prediction model to perform label prediction, so as to obtain a class label corresponding to the text to be tested.

Alternatively, the tag lookup module 502 may include:

the associated tag cluster determining submodule is used for determining an associated tag cluster where the neighbor class tag is located in a plurality of tag clusters recorded by the tag cluster relation;

The first association category label setting sub-module is used for setting other category labels except the neighbor category label in the association label cluster as association category labels.

Optionally, the apparatus may further include:

the tag text information acquisition module is used for acquiring text information of each type of tag; the text information is a label text and/or a description text of the category label, and the description text contains explanation information of the category label;

The tag text information conversion module is used for carrying out vector conversion on the text information to obtain tag embedded vectors of the tags of each category;

The label cluster construction module is used for carrying out clustering processing on the label embedded vectors to obtain a plurality of label clusters.

Alternatively, the tag lookup module 502 may include:

the parent node determining submodule is used for searching parent nodes of nodes where neighbor class labels are located in a label tree for recording the label hierarchical relationship; wherein, the nodes in the label tree are in one-to-one correspondence with the category labels;

and the second association category label setting sub-module is used for setting the category label corresponding to the father node as the association category label.

Optionally, the neighbor text search module 501 includes:

the text conversion submodule to be tested is used for carrying out vector conversion on the text to be tested to obtain an embedded vector of the text to be tested;

the vector similarity calculation submodule is used for determining the vector similarity between the text embedding vector to be detected and the text embedding vector of each text in the preset corpus;

And the neighbor text query sub-module is used for determining neighbor texts corresponding to the text to be tested according to the vector similarity.

Optionally, the neighbor text query submodule is specifically configured to:

and setting the text to which the text embedded vector corresponding to the maximum vector similarity belongs as a neighbor text of the text to be tested.

Optionally, the text conversion submodule to be tested is specifically configured to:

；

Optionally, the vector similarity calculation submodule is specifically configured to:

And determining cosine similarity between the text embedded vector to be tested and the text embedded vector according to the following formula:

；

Wherein, Representing the text-to-be-tested embedded vector,Representing a text-embedded vector;

the cosine similarity is set as the vector similarity.

Optionally, the apparatus may further include:

The preprocessing module is used for segmenting the text to be detected, removing stop words in the text to be detected and obtaining the processed text to be processed;

the neighbor text search module 501 is further configured to enter a step of determining a neighbor text corresponding to the text to be tested in a preset corpus based on the processed text to be tested.

Optionally, the apparatus may further include:

The neighbor training text query module is used for acquiring training texts from the training set and determining neighbor training texts corresponding to the training texts in a preset corpus; the method comprises the steps that a preset corpus is composed of a training set and a verification set, a plurality of category labels are marked on training texts, the training texts are different from neighbor training texts, and the neighbor training texts are determined according to the similarity between the neighbor training texts and the training texts;

the training label searching module is used for setting the class label of the adjacent training text as a neighbor training class label and determining an associated training class label corresponding to the neighbor training class label by utilizing preset label structure knowledge;

the training guide knowledge setting module is used for setting the label text of the neighbor training class label and the label text of the associated training class label together as training guide knowledge;

The training prediction module is used for inputting the training text and training guide knowledge into the label prediction model together to perform label prediction, so as to obtain a prediction type label corresponding to the training text;

And the model parameter updating module is used for determining a loss value according to the labeled class label and the predicted class label of the training text and updating the model parameter of the label predicted model by using the loss value.

the prediction module 504 may include:

the first processing sub-module is used for converting the text to be detected into a text embedded vector to be detected by using the coding module and converting the guide knowledge into a guide knowledge embedded vector by using the coding module;

The second processing submodule is used for carrying out feature extraction on the text embedded vector to be detected by utilizing the feature extraction module to obtain a text feature vector, and carrying out feature extraction on the guide knowledge embedded vector by utilizing the feature extraction module to obtain a guide knowledge feature vector; the feature extraction module comprises a network branch for extracting features of the text embedding vector to be detected and a network branch for guiding the knowledge embedding vector to extract features;

The third processing sub-module is used for fusing the text feature vector and the guide knowledge feature vector to obtain a fused feature vector;

And the fourth processing sub-module is used for processing the fusion feature vector by utilizing the output module and determining a class label corresponding to the text to be detected.

Optionally, the first processing sub-module may include:

The first input unit is used for inputting the text embedding vector to be detected into a first text convolution network branch to perform feature extraction so as to obtain a text feature vector;

and the second input unit is used for inputting the guide knowledge embedding vector into a second text convolution network branch to perform feature extraction so as to obtain a guide knowledge feature vector.

Optionally, the first processing sub-module is specifically configured to:

Optionally, the third processing sub-module is specifically configured to:

and splicing the text feature vector and the guide knowledge feature vector to obtain a fusion feature vector.

Optionally, the fourth processing sub-module is specifically configured to:

processing the fusion feature vector by using the output layer weight matrix and the regression function to obtain an output vector; wherein, each position of the output vector corresponds to the class label one by one, and each position of the output vector records the probability that the corresponding class label belongs to the text to be tested;

And setting the class label corresponding to the position with the value larger than the preset threshold value in the output vector as the class label corresponding to the text to be detected.

Referring to fig. 6, fig. 6 is a block diagram of an electronic device according to an embodiment of the present invention, and an electronic device 60 according to an embodiment of the present invention includes a processor 61 and a memory 62; wherein the memory 62 is used for storing a computer program; the processor 61 is configured to execute the text label prediction method provided in the foregoing embodiment when executing the computer program.

For the specific process of the text label prediction method, reference may be made to the corresponding content provided in the foregoing embodiment, and no further description is given here.

The memory 62 may be a carrier for storing resources, such as a read-only memory, a random access memory, a magnetic disk, or an optical disk, and the storage mode may be temporary storage or permanent storage.

In addition, the electronic device 60 further includes a power supply 63, a communication interface 64, an input-output interface 65, and a communication bus 66; wherein the power supply 63 is configured to provide an operating voltage for each hardware device on the electronic device 60; the communication interface 64 can create a data transmission channel between the electronic device 60 and an external device, and the communication protocol to be followed is any communication protocol applicable to the technical solution of the present invention, which is not specifically limited herein; the input/output interface 65 is used for obtaining external input data or outputting external output data, and the specific interface type thereof may be selected according to the specific application requirement, which is not limited herein.

Embodiments of the present invention also provide a computer program product comprising a computer program/instruction which, when executed by a processor, implements the text label prediction method as described in the above embodiments.

Since the embodiments of the computer program product portion and the embodiments of the text label prediction method portion correspond to each other, the embodiments of the computer program product portion are referred to for a description of the embodiments of the text label prediction method portion, and are not repeated herein.

The embodiment of the invention also provides a computer readable storage medium, and a computer program is stored on the computer readable storage medium, and when the computer program is executed by a processor, the text label prediction method is realized.

Since the embodiments of the computer readable storage medium portion and the embodiments of the text label prediction method portion correspond to each other, the embodiments of the storage medium portion are referred to the description of the embodiments of the text label prediction method portion, and are not repeated herein.

In the description, each embodiment is described in a progressive manner, and each embodiment is mainly described by the differences from other embodiments, so that the same similar parts among the embodiments are mutually referred. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.

Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

The text label predicting method, the text label predicting device, the electronic equipment and the storage medium provided by the invention are described in detail. The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to facilitate an understanding of the method of the present invention and its core ideas. It should be noted that it will be apparent to those skilled in the art that the present invention may be modified and practiced without departing from the spirit of the present invention.

Claims

1. A text label prediction method, comprising:

Setting the class label of the neighbor text as a neighbor class label, and determining an associated class label corresponding to the neighbor class label by using preset label structure knowledge; the preset tag structure knowledge records the association relationship among category tags, and is a tag hierarchy relationship and a tag cluster relationship;

The text to be tested and the guide knowledge are input into a label prediction model together to conduct label prediction, and a class label corresponding to the text to be tested is obtained;

determining the neighbor text corresponding to the text to be tested in a preset corpus, including:

Setting texts to which the text embedded vectors with the vector similarity larger than a preset threshold belong as the neighbor texts;

Determining the associated category label corresponding to the neighbor category label by using preset label structure knowledge comprises the following steps:

Setting other class labels except the neighbor class label in the associated label cluster as the associated class label;

setting a category label corresponding to the father node as the associated category label;

the construction process of the label cluster relationship comprises the following steps:

Acquiring text information of each type of tag; the text information is a tag text and a description text of the category tag, and the description text contains explanation information of the category tag;

clustering the tag embedding vectors to obtain a plurality of tag clusters;

The training process of the label prediction model comprises the following steps:

2. The text label prediction method according to claim 1, wherein performing vector conversion on the text to be detected to obtain an embedded vector of the text to be detected comprises:

；

3. The text label prediction method according to claim 1, wherein determining the vector similarity between the text-to-be-detected embedded vector and the text-embedded vectors of the texts in the preset corpus comprises:

；

And setting the cosine similarity as the vector similarity.

4. The text label prediction method according to claim 1, further comprising, before determining a neighboring text corresponding to the text to be tested in a preset corpus:

Word segmentation is carried out on the text to be detected, and stop words in the text to be detected are removed, so that the processed text to be detected is obtained;

And entering a step of determining neighbor texts corresponding to the text to be tested in a preset corpus based on the processed text to be tested.

5. The text label prediction method according to any one of claims 1 to 4, wherein the label prediction model comprises an encoding module, a feature extraction module, and an output module;

6. The text label prediction method according to claim 5, wherein performing feature extraction on the text-to-be-detected embedded vector by using the feature extraction module to obtain a text feature vector, and performing feature extraction on the guide knowledge embedded vector by using the feature extraction module to obtain a guide knowledge feature vector, comprises:

7. The text label prediction method of claim 6, wherein the first text convolution network branch and the second text convolution network branch have the same network structure.

8. The text label prediction method of claim 5, wherein converting the guide knowledge into a guide knowledge embedding vector comprises:

9. The text label prediction method according to claim 5, wherein fusing the text feature vector and the guide knowledge feature vector to obtain a fused feature vector comprises:

10. The text label prediction method according to claim 5, wherein the processing the fused feature vector by using an output module to determine the class label corresponding to the text to be detected includes:

11. A text label predicting apparatus, comprising:

The label searching module is used for setting the class label of the neighbor text as a neighbor class label and determining an associated class label corresponding to the neighbor class label by utilizing preset label structure knowledge; the preset tag structure knowledge records the association relationship among category tags, and is a tag hierarchy relationship and a tag cluster relationship;

the prediction module is used for inputting the text to be detected and the guide knowledge into a preset label prediction model together to perform label prediction, so as to obtain a class label corresponding to the text to be detected;

the neighbor text search module comprises:

a neighbor text query sub-module, configured to set, as the neighbor texts, texts to which the text embedding vector with the vector similarity greater than a preset threshold belongs;

The tag searching module comprises:

A first association category label setting sub-module, configured to set other category labels in the association category label cluster, except for the neighbor category label, as the association category label;

The parent node determining submodule is used for searching parent nodes of nodes where the neighbor class labels are located in the label tree for recording the label hierarchical relationship; wherein, the nodes in the label tree are in one-to-one correspondence with the category labels;

The second association category label setting sub-module is used for setting the category label corresponding to the father node as the association category label;

the text label predicting device further includes:

The tag text information acquisition module is used for acquiring text information of each type of tag; the text information is a tag text and a description text of the category tag, and the description text contains explanation information of the category tag;

The tag text information conversion module is used for carrying out vector conversion on the text information to obtain tag embedded vectors of all types of tags;

the label cluster construction module is used for carrying out clustering treatment on the label embedded vectors to obtain a plurality of label clusters;

the text label predicting device further includes:

The neighbor training text query module is used for acquiring training texts from a training set and determining neighbor training texts corresponding to the training texts in the preset corpus; the preset corpus is composed of a training set and a verification set, the training text is marked with a plurality of category labels, the training text is different from the neighbor training text, and the neighbor training text is determined according to the similarity between the training text and the neighbor training text;

The training label searching module is used for setting the class label of the neighbor training text as a neighbor training class label and determining an associated training class label corresponding to the neighbor training class label by utilizing the preset label structure knowledge;

The training prediction module is used for inputting the training text and the training guide knowledge into the label prediction model together to perform label prediction, so as to obtain a prediction type label corresponding to the training text;

And the model parameter updating module is used for determining a loss value according to the labeled class label of the training text and the predicted class label, and updating the model parameter of the label predicted model by utilizing the loss value.

12. An electronic device, comprising:

a memory for storing a computer program;

a processor for implementing the text label prediction method according to any one of claims 1 to 10 when executing the computer program.

13. A computer program product comprising computer programs/instructions which, when executed by a processor, implement the steps of the text label prediction method of any one of claims 1 to 10.

14. A computer readable storage medium having stored therein computer executable instructions which when loaded and executed by a processor implement the text label prediction method of any one of claims 1 to 10.