CN115964490A - Project label prediction method, system, electronic device and storage medium - Google Patents

Abstract

The invention provides an item label prediction method, system, electronic equipment and storage medium, belonging to the technical field of data processing. The method includes: obtaining character sequences corresponding to key texts representing investment projects to be classified; converting the character sequences into several embedded representations through mapping, and superimposing the several embedded representations to obtain project information word sequences; processing the project through the Bert language model The information word sequence outputs the word vector matrix; perform local feature extraction on the word vector matrix, and normalize the extracted local features to obtain the pooling result; use the full connection to transform the spliced pooling result to obtain the integrated feature; The softmax classifier learns the classification labels for the integrated features. This application uses natural language processing to convert project text into word vectors, and then performs convolutional neural network classification processing on word vectors to complete intelligent label classification, so as to improve the accuracy, comprehensiveness and efficiency of investment project label classification.

Description

Item label prediction method, system, electronic device and storage medium

technical field

The invention belongs to the technical field of text processing, and in particular relates to an item label prediction method, system, electronic equipment and storage medium.

Background technique

The text classification of investment projects is to divide project texts into pre-defined categories according to the text content. Accurate and fast project label classification can save a lot of manpower and material resources, and plays an important role in information retrieval and information storage. With the development and popularization of software technology, investment project management software has been deeply applied in the investment project industry. As far as the current development and reform industry full-caliber investment project data involves 97 large industries and 1380 small industries, the existing technology uses manual labeling to classify project text labels, which makes the processing process limited by personnel restrictions. Among them, The method of manual labeling is to determine the label classification for the item text by the label classifier according to the judgment of his own experience. Due to the difference in the richness of experience among the label classifiers, the accuracy, comprehensiveness, and efficiency of the item label classification There are deficiencies.

Therefore, how to realize the intelligent label classification of investment projects to improve the accuracy, comprehensiveness and efficiency of investment project label classification is particularly important.

Contents of the invention

In order to solve the above technical problems, the present invention provides a project label prediction method, system, electronic equipment and storage medium, which uses natural language processing to convert project text information into word vectors, and then performs deep learning on the convolutional neural network of word vectors Classification processing, and finally complete the intelligent label classification of investment projects, so as to improve the accuracy, comprehensiveness and efficiency of investment project label classification.

In a first aspect, the present invention provides a project label prediction method, including:

Obtaining the character sequence corresponding to the key text characterizing the investment project to be classified;

Converting the character sequence into several embedding representations by means of mapping, and superimposing the several embedding representations to obtain the item information word sequence; wherein, the embedding representations include character embedding, position embedding and sentence type embedding;

Process the item information word sequence output word vector matrix by Bert language model;

Performing local feature extraction for the word vector matrix, and normalizing the extracted local features to obtain a pooling result;

Transforming the spliced pooling results by full connection to obtain integrated features;

A softmax classifier is used to learn the integrated features to obtain the classification labels of the investment items to be classified.

Preferably, the step of obtaining the character sequence corresponding to the key text characterizing the investment project to be classified specifically includes:

Concatenate the project name, main construction content and industry field of the investment project to be classified to obtain the key text of the investment project to be classified;

removing stop words in the key text to obtain character groups;

The first n words in the character group are spliced with identifiers, and the identifiers are placed in the first place to form a character sequence corresponding to the key text.

Preferably, the step of processing the item information word sequence through the Bert language model to output a word vector matrix specifically includes:

Converting the item information word sequence into unicode, and removing illegal characters and redundant spaces in the unicode by Unicode code points, to obtain the information word string;

The Chinese characters in the information word string are separated by spaces, and the loop strip() operation is performed to obtain the initial word segmentation result;

Perform deep processing on the initial word segmentation results to obtain the target word segmentation results;

The English in the target word segmentation result is split according to the preset splitting principle to obtain a word vector matrix.

Preferably, the preset splitting principle is specifically:

Split the English language according to the subword vocabulary, the subword after each word is split as long as possible, adopt the greedy longest-first matching algorithm, for each word, the pointer i=0, j=len are matched from the back to the front, Until the prefix [i:j] of the word is a subword in the subword vocabulary, it is taken out, and then i=j, j=len are set, and the above-mentioned process is circulated.

Preferably, the step of performing local feature extraction on the word vector matrix, and normalizing the extracted local features to obtain a pooling result specifically includes:

Using a convolutional neural network model to perform local feature extraction on the word vector matrix to obtain a feature extraction result;

The feature extraction result is subjected to feature normalization through a maximum pooling operation, so as to select a local optimal feature to obtain a pooling result.

Preferably, the full connection applies a dropout strategy to fix the activation probability of some neurons at a p value, wherein the p value ranges from 0 to 1.

Preferably, the specific steps of learning the integrated features through the softmax classifier to obtain the classification labels of the investment items to be classified include:

The multi-class cross-entropy function is used as the loss function of the convolutional neural network model;

The integrated feature is calculated by a convolutional neural network model to output a corresponding classification label.

In a second aspect, the present invention provides an item label prediction system, comprising:

A characterizing module, configured to obtain character sequences corresponding to key texts characterizing investment projects to be classified;

The mapping module is used to convert the character sequence into several embedded representations by mapping, and superimpose several embedded representations to obtain the item information word sequence; wherein, the embedded representations include character embedding, position embedding and sentence type embedding;

Semantic module, for processing described item information word sequence output word vector matrix by Bert language model;

A processing module, configured to extract local features for the word vector matrix, and normalize the extracted local features to obtain a pooling result;

A transform module, configured to transform the spliced pooling results using full connections to obtain integrated features;

The classification module is used to learn the integrated features through a softmax classifier to obtain the classification labels of the investment items to be classified.

Preferably, the characterization module includes:

The concatenation unit is used to concatenate the project names, main construction contents and industry fields of the investment projects to be classified to obtain the key text of the investment projects to be classified;

A removal unit, configured to remove stop words in the key text to obtain a character group;

A concatenating unit, configured to concatenate the first n words in the character group with identifiers, and put the identifiers in the first place, so as to form a character sequence corresponding to the key text.

Preferably, the semantic module includes:

The conversion unit is used to convert the item information word sequence into unicode, and remove illegal characters and redundant spaces in the unicode through the Unicode code point to obtain the information word string;

The loop unit is used to separate the Chinese characters in the information word string by spaces, and perform a loop strip() operation to obtain the initial word segmentation result;

The deep processing unit is used to perform deep processing on the initial word segmentation result to obtain the target word segmentation result;

The splitting unit is configured to split the English in the target word segmentation result according to a preset splitting principle to obtain a word vector matrix.

Preferably, the processing module includes:

The extraction unit is used to perform local feature extraction for the word vector matrix using a convolutional neural network model to obtain a feature extraction result;

The pooling unit is used to perform feature normalization on the feature extraction result through a maximum pooling operation, so as to select a local optimal feature to obtain a pooling result.

Preferably, the classification module includes:

Define a unit for adopting a multi-class cross-entropy function as a loss function of a convolutional neural network model;

The classification unit is used to calculate the integrated features through the convolutional neural network model to output corresponding classification labels.

In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and operable on the processor. When the processor executes the computer program, Implement the item label prediction method as described in the first aspect.

In a fourth aspect, an embodiment of the present application provides a storage medium, on which a computer program is stored, and when the program is executed by a processor, the method for predicting item tags as described in the first aspect is implemented.

Compared with the prior art, the beneficial effects of the present invention are: adopting the general language model BERT model in NLP, it can capture the character sequence information, context information, grammatical context information, etc. of the text information in the entire sentence, and solve the problem of multiple words question of justice. Moreover, the Bert model uses the encoder of the converter to perform operations in parallel, and can superimpose multiple layers. It has a strong ability to represent text information. The word vector output by it can very well represent the characteristics of text information, and then use it as a downstream natural Model parameters or model inputs for language processing tasks to improve the overall performance of the model. Followed by the CNN layer, convolution kernels of different sizes are used to capture the information of words of different lengths in the sentence. The information contained in Chinese words is often richer than that of words, so the word information is extracted from the entire sentence data and then classified. The classification results More ideally, to improve the accuracy, comprehensiveness and efficiency of investment project label classification.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the accompanying drawings that need to be used in the descriptions of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only of the present invention. For some embodiments, those of ordinary skill in the art can also obtain other drawings based on these drawings without paying creative efforts.

Fig. 1 is the flow chart of the item tag prediction method provided by Embodiment 1 of the present invention;

Fig. 2 is the structural block diagram of the item tag prediction system corresponding to the method of embodiment 1 provided by embodiment 2 of the present invention;

FIG. 3 is a schematic diagram of a hardware structure of an electronic device provided by Embodiment 3 of the present invention.

Explanation of reference signs:

10-characterization module, 11-serial unit, 12-removal unit, 13-stitching unit;

20 - mapping module;

30-semantic module, 31-transformation unit, 32-circulation unit, 33-deep processing unit, 34-splitting unit;

40-processing module, 41-extraction unit, 42-pooling unit;

50-transformation module;

60-taxonomic module, 61-definition unit, 62-taxonomic unit;

70-bus, 71-processor, 72-memory, 73-communication interface.

Detailed ways

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concept of example embodiments to those skilled in the art.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided in order to give a thorough understanding of embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, means, steps, etc. may be employed. In other instances, well-known methods, apparatus, implementations, or operations have not been shown or described in detail to avoid obscuring aspects of the present disclosure.

The block diagrams shown in the drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices entity.

The flow charts shown in the drawings are only exemplary illustrations, and do not necessarily include all contents and operations/steps, nor must they be performed in the order described. For example, some operations/steps can be decomposed, and some operations/steps can be combined or partly combined, so the actual order of execution may be changed according to the actual situation.

With the development and popularization of software technology, investment project management software has been deeply applied in the investment project industry. In the traditional investment project management mode, due to the lack of intelligence, refinement, and scalability of the project library classification, the project information Insufficient support. For example: when a new requirement for regional investment is put forward, it is necessary to manually create a new label, and simultaneously update the project's full-caliber database data to form a new theme analysis. At present, the classification of different items is judged manually, and actual operators need to analyze and process a large amount of data. Moreover, the processing process will be limited by personnel restrictions, resulting in the inability to respond to the rapid analysis requirements for item classification in a timely manner. This application is based on the data model to judge the project classification, reduce manpower input, improve the accuracy of analysis, and make the analysis results more scientific.

Example 1

Specifically, FIG. 1 is a schematic flowchart of a method for predicting an item label provided by this embodiment.

As shown in Figure 1, the item label prediction method of the present embodiment includes the following steps:

S101. Obtain a character sequence corresponding to a key text characterizing an investment project to be classified.

Specifically, according to the task type of natural language processing, the original text is used as the input data of natural language processing. Due to the high degree of dispersion of the original text, the data type cannot be directly called and processed in the process of natural language processing. The original data of the original text is converted into a character sequence, and then word segmentation is performed on the character sequence. In this embodiment, by selecting the project name, main construction content and text information of the industry field of a certain investment project as the key text of the investment project to be classified, it is because the text information of the project name, main construction content and industry field is because this The three items are the text fields that best reflect the construction content of the project. After completing the information extraction of the original data and removing the stop words through the encoder as the execution subject, the first n words of the text are spliced with the BERT established identifier [CLS], where [CLS] is placed in the first place and used to represent the The character sequence corresponding to the key text of the classified investment project.

Further, the specific steps of step S101 include:

S1011. Concatenate the project name, main construction content and industry field of the investment project to be classified to obtain the key text of the investment project to be classified.

Specifically, an investment project refers to an independent whole composed of activities, institutions and other aspects planned and implemented to accomplish a certain development goal (or a group of development goals) within a specified period. For the text of the investment project to be classified, usually the entire document of the investment project has certain format requirements, such as project name, main construction content, industry field, feasibility analysis, financial presupposition and other modules, in order to best reflect the investment project. Construction content, this embodiment selects the project name of the investment project, the main construction content and the text information of the industry field as the selection information, and splices the selection information to obtain the key text that best reflects the investment project to be classified.

S1012. Remove stop words in the key text to obtain a character group.

Specifically, stop words mean that in information retrieval, in order to save storage space and improve search efficiency, some words or words are automatically filtered out before or after processing natural language data (or text), and these words or words are called called stop words. These stop words are all manually input and non-automatically generated, and the generated stop words will form a stop word list. When using jieba for Chinese word segmentation, the main words that appear in the sentence will be divided, and some words have no practical meaning, which will increase the workload for subsequent keyword extraction, and the keywords that may be extracted are invalid . Therefore, in word segmentation processing, stop words will be introduced to remove and optimize word segmentation results. For stop words, you can manually add them to a txt file, and then import the file when needed, or you can use the already organized stop words list. Of course, on the basis of the existing stop vocabulary list, if there are still some words that are not needed, you can also improve the stop vocabulary list yourself.

S1013. Splice the first n words in the character group with identifiers, and place the identifiers at the first place, so as to form a character sequence corresponding to the key text.

Specifically, the identifier of the BERT model in this embodiment is specifically [CLS], and [CLS] means classification, which can be understood as being used for downstream classification tasks. For sentence pair classification tasks, the BERT model not only adds the [CLS] symbol and uses the corresponding output as the semantic representation of the text, but also divides the input two sentences with a [SEP] symbol, and attaches two sentences to the two sentences respectively. Different text vectors to differentiate. In this embodiment, the first n words in the obtained character group are spliced with the BERT identifier [CLS], and [CLS] is placed in the first place to form a character sequence corresponding to the key text, which is used to identify the semantics of the entire input.

S102. Convert the character sequence into several embedded representations by means of mapping, and superimpose the several embedded representations to obtain an item information word sequence; wherein, the embedded representations include character embedding, position embedding, and sentence type embedding.

Specifically, the contextual feature extraction layer uses BiGRU, CNN and BiGCN encoders to extract the multi-granularity feature representation and regional feature representation of sentences, and the multi-feature embedding layer BiGRU can fuse forward and backward information, so it can better extract features information. Input a character sequence, the task of the contextual feature extraction layer is to fully capture the semantic feature information of the sentence. This embodiment adds position embedding and sentence type embedding on the basis of character embedding.

Among them, character embedding: CNN has a strong local feature extraction ability, so the character feature representation _Hc of each word is extracted through CNN, and the specific formula is as follows:

_Hc = FCL(Maxpooling(Conv(c1, c2, . . . , cm)),

Among them, FCL() belongs to the full connection operation, Maxpooling() belongs to the maximum pooling operation, Conv() belongs to the convolution operation, and m is the length of the word.

Among them, position embedding: in a sequence, the position of a word can provide additional semantic information for the word itself. In this paper, the position information of each character is obtained based on the sine and cosine functions. The value range of the vector generated by the sine and cosine functions is between 0 and 1, which is conducive to the training and convergence of the model. Input a text sequence, and get the position embedding H _f through sine and cosine function transformation, the specific method is as follows:

Among them, f is the position where the character appears in the sequence, and d _p represents the dimension of the position vector generated by the sine-cosine function. The dimension and period of the sine-cosine function influence each other, so it can capture relative and absolute position information between words.

S103, process the project information word sequence through the Bert language model to output a word vector matrix.

Specifically, the Bert language model is a bidirectional language model in a complete sense, which can capture word sequence information, contextual relationship information, grammatical context information, etc. in the entire sentence to solve the problem of polysemy. Word vectors are used as high-quality feature inputs for downstream models. NLP models (such as LSTMs or CNNs) require input in the form of numerical vectors, meaning that features such as vocabulary and parts of speech need to be converted into numerical representations. These feature embeddings are produced by models such as Word2Vec or Fasttext. BERT offers an advantage over models like Word2Vec because, while under Word2Vec each word has a fixed representation independent of the context in which the word appears, BERT generates word representations that are dynamically informed by the words around it .

Further, the specific steps of step S103 include:

S1031. Convert the item information word sequence into unicode, and remove illegal characters and redundant spaces in the unicode through Unicode code points to obtain an information word string.

Specifically, unicode was created to solve the limitations of traditional character encoding schemes. It sets a unified and unique binary encoding for each character in each language to meet the needs of cross-language and cross-platform text conversion and processing. requirements. In the present embodiment, the character string or byte in the project information word sequence are converted into unicode purpose to facilitate operation, because follow-up operations will judge Chinese, English, special symbols, etc.; and remove illegal characters and characters in unicode by Unicode code points extra spaces.

S1032. Separate the Chinese characters in the information word string by spaces, and perform a loop strip() operation to obtain an initial word segmentation result.

Specifically, the strip() operation is used to remove the characters or character sequences specified at the beginning and end of the string. Only the characters at the beginning or the end can be deleted, but the characters in the middle cannot be deleted, such as spaces, newlines (\n), and restrictions. Tabulator (\t). In this embodiment, for the characters in the text, it is first judged whether it is a Chinese character, if yes, a space is added before and after it, otherwise it is output as it is. After this step, Chinese characters are separated by characters and separated by spaces, but English numbers, etc. remain as they are.

S1033, performing deep processing on the initial word segmentation result to obtain a target word segmentation result.

Specifically, first perform the strip() operation on the text to remove redundant blank characters on both sides, and then return an empty list if the remaining is an empty string, otherwise perform the split() operation to obtain the original word segmentation result orig_tokens; word segmentation The obtained list continues to be processed, and the diacritics contained in the corpus are removed and restored to character form, or word segmentation is performed according to punctuation marks.

S1034. Split the English in the target word segmentation result according to a preset splitting principle to obtain a word vector matrix. Wherein, the preset splitting principle is specifically: split English according to the subword vocabulary, the subword after each word is split as long as possible, adopt the greedy longest-first matching algorithm, and for each word, the pointer i =0, j=len are matched from back to front until the prefix [i:j] of the word is a subword in the subword vocabulary, then it is taken out, and then i=j, j=len are set, and the above-mentioned process is circulated.

Specifically, the target word segmentation results are further split by WordPiece Tokenizer. First of all, for Chinese, the word granularity is already the smallest inseparable granularity, and there is no way to perform subword. Subword basically processes English; Words are split according to the subword vocabulary, and the process of determining the word segmentation path can use greedy matching, for example, matching the longest string or code sequence as much as possible to obtain the word segmentation path; non-greedy matching can also be used, for example, matching the shortest possible character string or code sequence, find the word segmentation path of all consecutive words in the character or code, and determine the word segmentation fragment according to the word segmentation path, the word segmentation fragment exists in the form of a character set or a code set, and encode the word segmentation fragment , so that the word segmentation segment is vectorized. In this embodiment, a greedy longest-first matching algorithm is used.

S104, performing local feature extraction on the word vector matrix, and normalizing the extracted local features to obtain a pooling result.

Specifically, since the dimension of the word vector matrix is high and contains some noise, a multi-core convolutional neural network is introduced to optimize its representation. The convolutional neural network can perform convolution transformation on all features in the same area through the sliding window mechanism to effectively preserve the local features of words. Considering the training time and accuracy of the model, in practice, setting the convolution kernel as a mixed convolution kernel with a window of 3, 4, and 5 can ensure that the model has a low training complexity and has a good classification effect. The word vector matrix is used as the input of the convolutional layer, and multiple convolution kernels with a size of (2, 3, 4) are used for local feature extraction to obtain the corresponding feature extraction results, and then the local optimum is selected through the Max Pooling operation feature.

Further, the specific steps of step S104 include:

S1041. Use a convolutional neural network model to extract local features from the word vector matrix to obtain a feature extraction result.

Specifically, image information is stored in each pixel of the image. Define a convolution kernel (equivalent to a weight), which is used to extract certain features from the image. The convolution kernel is multiplied and added to the corresponding bits of the digital matrix to obtain the output result of the convolution layer. The value of the convolution kernel can be randomly generated by the function without previous learning experience, and then gradually trained and adjusted to extract the features of each small part of the picture.

S1042. Perform feature normalization on the feature extraction result through a maximum pooling operation, so as to select a local optimal feature to obtain a pooling result.

Specifically, the maximum pooling operation adopts the max pooling method, which is to take the maximum value of an area. Therefore, when the image undergoes minor changes such as translation, scaling, and rotation, it is still very likely to take the maximum value at the same position, which is the same as the response before the change, thus achieving affine invariance. The purpose of pooling in this embodiment is to reduce the number of training parameters, reduce the dimension of the feature vector output by the convolutional layer; reduce over-fitting phenomenon, retain only the most useful image information, and reduce the transmission of noise.

S105, transforming the spliced pooling result by using full connection to obtain an integrated feature.

Specifically, the full connection applies a dropout strategy to fix the activation probability of some neurons on the p value, where the p value ranges from 0 to 1; the dropout strategy can make the activation probability of some neurons fixed In terms of p value, the model will not be too dependent on some local features during the forward transmission process, so that the robustness of the model is better and the generalization ability is stronger. In this embodiment, the job of the convolutional layer and the pooling layer is to extract features and reduce the parameters brought by the original image. In order to generate the final output, a fully connected layer needs to be applied to generate a classifier equal to the number of classes required. The working principle of the fully connected layer is very similar to the previous neural network learning, and the tensor output by the pooling layer needs to be re-cut. Form some vectors, multiply the weight matrix, add the bias value, and then use the ReLU activation function to optimize the parameters with the gradient descent method.

S106. Obtain classification labels of the investment items to be classified by learning the integrated features through a softmax classifier.

Specifically, the softmax function is a commonly used multi-classifier in machine learning, especially in convolutional neural networks, where the last layer often uses a softmax classifier for multi-category classification tasks. The softmax function is the general form of the logistic function, which converts the classification problem into a probability problem, that is, solves and counts all possible probabilities, and then the one with the highest probability is considered as the category.

Further, the specific steps of step S106 include:

S1061, using a multi-class cross-entropy function as a loss function of the convolutional neural network model;

S1062. Calculate the integrated feature through a convolutional neural network model to output a corresponding classification label.

To sum up, the BERT model, the general language model in NLP, can capture the character sequence information, contextual relationship information, grammatical context information, etc. of the text information in the entire sentence, and solve the polysemy problem of a word. Moreover, the Bert model uses the encoder of the converter to perform operations in parallel, and can superimpose multiple layers. It has a strong ability to represent text information. The word vector output by it can very well represent the characteristics of text information, and then use it as a downstream natural Model parameters or model inputs for language processing tasks to improve the overall performance of the model. Followed by the CNN layer, convolution kernels of different sizes are used to capture the information of words of different lengths in the sentence. The information contained in Chinese words is often richer than that of words, so the word information is extracted from the entire sentence data and then classified. The classification results More ideally, to improve the accuracy, comprehensiveness and efficiency of investment project label classification.

Example 2

This embodiment provides a structural block diagram of a system corresponding to the method described in Embodiment 1. Fig. 2 is a structural block diagram of the item label prediction system according to the present embodiment, as shown in Fig. 2, the system includes:

The characterizing module 10 is configured to acquire character sequences corresponding to key texts representing investment items to be classified.

The mapping module 20 is used for converting the character sequence into several embedded representations through mapping, and superimposing the several embedded representations to obtain the project information word sequence; wherein, the embedded representations include character embedding, position embedding and sentence type embedding.

The semantic module 30 is configured to process the item information word sequence through the Bert language model and output a word vector matrix.

The processing module 40 is configured to perform local feature extraction on the word vector matrix, and normalize the extracted local features to obtain a pooling result.

The transformation module 50 is used to transform the spliced pooling results by using the full connection to obtain integrated features; specifically, the full connection applies a dropout strategy to fix the activation probability of some neurons on the p value, where , and the p-value ranges from 0 to 1.

The classification module 60 is configured to learn the integrated features through a softmax classifier to obtain the classification labels of the investment items to be classified.

Preferably, the characterization module 10 includes:

The concatenation unit 11 is used to concatenate the project names, main construction contents and industry fields of the investment projects to be classified to obtain the key text of the investment projects to be classified;

The removal unit 12 is used to remove the stop words in the key text to obtain the character group;

The concatenating unit 13 is configured to concatenate the first n words in the character group with identifiers, and place the identifiers at the first place, so as to form a character sequence corresponding to the key text.

Preferably, the semantic module 30 includes:

The conversion unit 31 is used to convert the item information word sequence into unicode, and remove illegal characters and redundant spaces in the unicode by the Unicode code point to obtain the information word string;

The loop unit 32 is used to separate the Chinese characters in the information word string by spaces, and perform a loop strip() operation to obtain the initial word segmentation result;

The deep processing unit 33 is used for performing deep processing on the initial word segmentation result to obtain the target word segmentation result;

The splitting unit 34 is used to split the English in the target word segmentation result according to a preset splitting principle to obtain a word vector matrix; wherein, the preset splitting principle is specifically: carry out English according to the subword vocabulary Splitting, the subword after splitting each word is as long as possible, using the greedy longest-first matching algorithm, for each word, the pointer i=0, j=len are matched from back to front until the prefix of the word [i: j] is a subword in the subword vocabulary, then it is taken out, and then i=j, j=len are set, and the above-mentioned process is circulated.

Preferably, the processing module 40 includes:

The extraction unit 41 is used to perform local feature extraction for the word vector matrix using a convolutional neural network model to obtain a feature extraction result;

The pooling unit 42 is configured to perform feature normalization on the feature extraction result through a maximum pooling operation, so as to select local optimal features to obtain a pooling result.

Preferably, the classification module 60 includes:

Definition unit 61, for adopting multiclass cross-entropy function as the loss function of convolutional neural network model;

The classification unit 62 is configured to calculate the integrated feature through a convolutional neural network model to output a corresponding classification label.

It should be noted that each of the above-mentioned modules may be a function module or a program module, and may be realized by software or by hardware. For the modules implemented by hardware, the above modules may be located in the same processor; or the above modules may be located in different processors in any combination.

Example 3

The item label prediction method described in conjunction with FIG. 1 can be implemented by electronic devices. FIG. 3 is a schematic diagram of a hardware structure of an electronic device according to this embodiment.

The electronic device may comprise a processor 71 and a memory 72 storing computer program instructions.

Specifically, the processor 71 may include a central processing unit (CPU), or an Application Specific Integrated Circuit (ASIC for short), or may be configured to implement one or more integrated circuits in the embodiments of the present application.

Wherein, the memory 72 may include a mass memory for data or instructions. For example without limitation, memory 72 may include a hard disk drive (Hard Disk Drive, referred to as HDD), a floppy disk drive, a solid state drive (SolidState Drive, referred to as SSD), flash memory, optical disc, magneto-optical disc, magnetic tape or universal serial bus (Universal Serial Bus, referred to as USB) drive or a combination of two or more of the above. Memory 72 may comprise removable or non-removable (or fixed) media, where appropriate. Memory 72 may be internal or external to the data processing arrangement, where appropriate. In a particular embodiment, memory 72 is a non-volatile (Non-Volatile) memory. In a specific embodiment, the memory 72 includes a read-only memory (Read-Only Memory, referred to as ROM) and a random access memory (Random Access Memory, referred to as RAM). In appropriate cases, the ROM can be mask programmed ROM, programmable ROM (Programmable Read-Only Memory, referred to as PROM), erasable PROM (Erasable Programmable Read-Only Memory, referred to as EPROM), electrically erasable In addition to PROM (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), electrically rewritable ROM (Electrically Alterable Read-Only Memory, referred to as EAROM) or flash memory (FLASH) or a combination of two or more of these. Where appropriate, the RAM can be a Static Random-Access Memory (SRAM for short) or a Dynamic Random-Access Memory (DRAM for short), where the DRAM can be a fast page Mode Dynamic Random Access Memory (Fast Page Mode Dynamic Random Access Memory, referred to as FPMDRAM), Extended Data Output Dynamic Random Access Memory (Extended Date Out Dynamic Random Access Memory, referred to as EDODRAM), Synchronous Dynamic Random Access Memory (Synchronous Dynamic Random-Access Memory, referred to as SDRAM) and so on.

The memory 72 may be used to store or cache various data files required for processing and/or communication, as well as possible computer program instructions executed by the processor 71 .

The processor 71 reads and executes the computer program instructions stored in the memory 72 to implement the item label prediction method of the first embodiment above.

In some of these embodiments, the electronic device may further include a communication interface 73 and a bus 70 . Wherein, as shown in FIG. 3 , the processor 71 , the memory 72 , and the communication interface 73 are connected through the bus 70 and complete mutual communication.

The communication interface 73 is used to realize the communication between various modules, devices, units and/or devices in the embodiment of the present application. The communication interface 73 can also implement data communication with other components such as external devices, image/data acquisition equipment, databases, external storage, and image/data processing workstations.

Bus 70 includes hardware, software, or both, and couples the components of the device to each other. The bus 70 includes but is not limited to at least one of the following: a data bus (Data Bus), an address bus (Address Bus), a control bus (ControlBus), an expansion bus (Expansion Bus), and a local bus (Local Bus). For example and without limitation, the bus 70 may include an Accelerated Graphics Port (AGP for short) or other graphics bus, an Enhanced Industry Standard Architecture (Extended Industry Standard Architecture, EISA for short) bus, a Front Side Bus (FrontSide Bus, Referred to as FSB), Hyper Transport (Hyper Transport, referred to as HT) interconnection, Industry Standard Architecture (Industry Standard Architecture, referred to as ISA) bus, wireless bandwidth (InfiniBand) interconnection, low pin count (Low Pin Count, referred to as LPC) bus, memory bus, Micro Channel Architecture (MCA) bus, Peripheral Component Interconnect (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology Attachment (Serial Advanced Technology Attachment, referred to as SATA) bus, Video Electronics Standards Association local (Video Electronics Standards Association Local Bus, referred to as VLB) bus or other suitable bus or a combination of two or more of these. Bus 70 may comprise one or more buses, where appropriate. Although the embodiments of this application describe and illustrate a particular bus, this application contemplates any suitable bus or interconnect.

The electronic device can obtain the item label prediction system, and execute the item label prediction method in Embodiment 1.

In addition, in combination with the method for predicting item tags in Embodiment 1 above, this embodiment of the present application may provide a storage medium for implementation. Computer program instructions are stored on the storage medium; when the computer program instructions are executed by a processor, the method for predicting item labels in the first embodiment above is implemented.

The technical features of the above-mentioned embodiments can be combined arbitrarily. To make the description concise, all possible combinations of the technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, should be considered as within the scope of this specification.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention should be included in the protection of the present invention. within range.

Claims (10)

1. A project label prediction method, characterized in that, comprising: Obtaining the character sequence corresponding to the key text characterizing the investment project to be classified; Converting the character sequence into several embedding representations by means of mapping, and superimposing the several embedding representations to obtain the item information word sequence; wherein, the embedding representations include character embedding, position embedding and sentence type embedding; Process the item information word sequence output word vector matrix by Bert language model; Performing local feature extraction for the word vector matrix, and normalizing the extracted local features to obtain a pooling result; Transforming the spliced pooling results by full connection to obtain integrated features; A softmax classifier is used to learn the integrated features to obtain the classification labels of the investment items to be classified. 2. The project label prediction method according to claim 1, wherein the step of obtaining the character sequence corresponding to the key text representing the investment project to be classified specifically comprises: Concatenate the project name, main construction content and industry field of the investment project to be classified to obtain the key text of the investment project to be classified; removing stop words in the key text to obtain character groups; The first n words in the character group are spliced with identifiers, and the identifiers are placed in the first place to form a character sequence corresponding to the key text. 3. The item label prediction method according to claim 1, wherein the step of processing the item information word sequence output word vector matrix by the Bert language model specifically comprises: Converting the item information word sequence into unicode, and removing illegal characters and redundant spaces in the unicode by Unicode code points, to obtain the information word string; The Chinese characters in the information word string are separated by spaces, and the loop strip() operation is performed to obtain the initial word segmentation result; Perform deep processing on the initial word segmentation results to obtain the target word segmentation results; The English in the target word segmentation result is split according to the preset splitting principle to obtain a word vector matrix. 4. The item label prediction method according to claim 3, wherein the preset splitting principle is specifically: Split the English language according to the subword vocabulary, the subword after each word is split as long as possible, adopt the greedy longest-first matching algorithm, for each word, the pointer i=0, j=len are matched from the back to the front, Until the prefix [i:j] of the word is a subword in the subword vocabulary, it is taken out, and then i=j, j=len are set, and the above-mentioned process is circulated. 5. The item label prediction method according to claim 1, wherein the step of performing local feature extraction for the word vector matrix, and normalizing the extracted local features to obtain a pooling result specifically includes: Using a convolutional neural network model to perform local feature extraction on the word vector matrix to obtain a feature extraction result; The feature extraction result is subjected to feature normalization through a maximum pooling operation, so as to select a local optimal feature to obtain a pooling result. 6. The item label prediction method according to claim 1, wherein the full connection applies a dropout strategy to fix the activation probability of some neurons on the p value, wherein the p value ranges from 0 to 1. 7. The project label prediction method according to claim 1, wherein the specific steps of learning the classification label of the investment project to be classified by the softmax classifier for the integration feature include: The multi-class cross-entropy function is used as the loss function of the convolutional neural network model; The integrated feature is calculated by a convolutional neural network model to output a corresponding classification label. 8. A project tag prediction system, comprising: A characterizing module, configured to obtain character sequences corresponding to key texts characterizing investment projects to be classified; The mapping module is used to convert the character sequence into several embedded representations by mapping, and superimpose several embedded representations to obtain the item information word sequence; wherein, the embedded representations include character embedding, position embedding and sentence type embedding; Semantic module, for processing described item information word sequence output word vector matrix by Bert language model; A processing module, configured to extract local features for the word vector matrix, and normalize the extracted local features to obtain a pooling result; A transform module, configured to transform the spliced pooling results using full connections to obtain integrated features; The classification module is used to learn the integrated features through a softmax classifier to obtain the classification labels of the investment items to be classified. 9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, characterized in that, when the processor executes the computer program, the computer program according to claim The item label prediction method described in any one of 1 to 7. 10. A storage medium, on which a computer program is stored, wherein when the program is executed by a processor, the item label prediction method according to any one of claims 1 to 7 is implemented.

Images