CN110032732A - A kind of text punctuate prediction technique, device, computer equipment and storage medium - Google Patents

Abstract

The invention discloses a text punctuation prediction method, device, computer equipment and storage medium, which are applied to the technical field of deep learning and are used to solve the problem of no punctuation in the lexical text obtained by speech recognition. The method provided by the invention includes: obtaining target text without punctuation; performing word segmentation processing on the target text to obtain each target word in the target text; performing vectorization processing on each target word to obtain each target vector; The order of words in the target text, each target vector is input into the network model in turn, and the output result sequence is obtained, and each value in the result sequence represents the punctuation corresponding to each target word; according to the preset numerical punctuation correspondence Each punctuation corresponding to each numerical value is determined respectively; for each punctuation in each punctuation, each punctuation is inserted into the target text at a position behind the target word corresponding to each punctuation, to obtain a punctuation-predicted lexical text.

Description

A text punctuation prediction method, device, computer equipment and storage medium

technical field

The present invention relates to the technical field of deep learning, and in particular, to a text punctuation prediction method, device, computer equipment and storage medium.

Background technique

With the rapid development of society and high-tech technology, natural language processing such as smart home control, automatic question answering, and voice assistants has received more and more attention. However, since spoken dialogue has no punctuation and cannot distinguish between sentence boundaries and canonical language structures, punctuation prediction is an extremely important natural language processing task. In the smart phone customer service scenario, for the user's speech, the original speech text with no punctuation and no segmentation is obtained through speech recognition, and there is no way to use it directly. Punctuation prediction to add punctuation to unpunctuated text.

Therefore, it is an urgent problem for those skilled in the art to find a method that can accurately predict the punctuation of technical texts.

SUMMARY OF THE INVENTION

Embodiments of the present invention provide a text punctuation prediction method, device, computer equipment, and storage medium, so as to solve the problem of no punctuation in the lexical text obtained by speech recognition.

A text punctuation prediction method, comprising:

Get the target text without punctuation;

Perform word segmentation processing on the target text to obtain each target word in the target text;

Perform vectorization processing on each of the target words to obtain each target vector corresponding to each of the target words;

According to the order of the target words in the target text, the target vectors are sequentially input to the network model to obtain a sequence of results output by the network model in sequence, and each value in the sequence of results represents the The punctuation corresponding to each target word, the network model is composed of a pre-trained LSTM network and a conditional random field;

Each punctuation corresponding to each numerical value is respectively determined according to the preset numerical punctuation corresponding relationship, and the numerical punctuation corresponding relationship records the one-to-one correspondence between the numerical value and the punctuation;

For each punctuation in the various punctuations, insert each punctuation into the target text at a position behind the target word corresponding to each punctuation to obtain a punctuation-predicted lexical text, and the latter position refers to the position in the target text that follows the target word and is immediately adjacent to the target word.

A text punctuation prediction device, comprising:

Text acquisition module, used to acquire target text without punctuation;

a word segmentation processing module, configured to perform word segmentation processing on the target text to obtain each target word in the target text;

A word vectorization module, configured to perform vectorization processing on each of the target words to obtain each target vector corresponding to each of the target words;

The vector input module is used for sequentially inputting the respective target vectors to the network model according to the order of the respective target words in the target text, to obtain a sequence of results output by the network model in sequence, in the sequence of results Each numerical value of represents the punctuation corresponding to each target word, and the network model is composed of a pre-trained LSTM network and a conditional random field;

a punctuation determination module, configured to respectively determine each punctuation corresponding to each numerical value according to a preset numerical punctuation correspondence, wherein the numerical punctuation correspondence records the one-to-one correspondence between the numerical value and the punctuation;

A punctuation insertion module is used for, for each punctuation in the various punctuations, inserting the each punctuation into the target text in the position behind the target word corresponding to each punctuation, to obtain the vocabulary after punctuation prediction text, and the latter position refers to a position in the target text that is located after the target word and is close to the target word.

A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the above-mentioned text punctuation prediction method when the processor executes the computer program.

A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, implements the steps of the above-mentioned text punctuation prediction method.

The above-mentioned text punctuation prediction method, device, computer equipment and storage medium, first, obtain a target text without punctuation; then, perform word segmentation processing on the target text to obtain each target word in the target text; Each target word is vectorized separately to obtain each target vector corresponding to each target word; and then, according to the order of each target word in the target text, the each target vector is sequentially Input into the network model, and obtain the result sequence output by the network model in turn, each numerical value in the result sequence represents the punctuation corresponding to each target word, and the network model is composed of a pre-trained LSTM network and conditions. A random field is formed; secondly, each punctuation corresponding to each numerical value is determined according to the preset numerical punctuation correspondence, and the numerical punctuation correspondence records the one-to-one correspondence between the numerical value and the punctuation; each punctuation, inserting the each punctuation into the target text in the back position of the target word corresponding to each punctuation, to obtain the punctuation prediction after the term text, the back position refers to the target text in the position following and immediately following the target word. It can be seen that the present invention can accurately predict the punctuation of the target text through the pre-trained LSTM network and the preset conditional random field, complete the addition of punctuation to the text without punctuation, and improve the efficiency of text punctuation prediction, so as to facilitate the subsequent natural language Handles direct use of text.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the following briefly introduces the drawings that are used in the description of the embodiments of the present invention. Obviously, the drawings in the following description are only some embodiments of the present invention. , for those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative labor.

1 is a schematic diagram of an application environment of a text punctuation prediction method in an embodiment of the present invention;

Fig. 2 is a flow chart of a text punctuation prediction method in an embodiment of the present invention;

3 is a schematic flowchart of step 103 of the text punctuation prediction method in an application scenario in an embodiment of the present invention;

4 is a schematic flowchart of a training network model in an application scenario of a text punctuation prediction method according to an embodiment of the present invention;

5 is a schematic flowchart of step 106 of the text punctuation prediction method in an application scenario in an embodiment of the present invention;

6 is a schematic structural diagram of a text punctuation prediction device in an application scenario according to an embodiment of the present invention;

7 is a schematic structural diagram of a text punctuation prediction device in another application scenario according to an embodiment of the present invention;

8 is a schematic structural diagram of a punctuation insertion module in an embodiment of the present invention;

FIG. 9 is a schematic diagram of a computer device according to an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The text punctuation prediction method provided by the present application can be applied in the application environment as shown in FIG. 1 , in which the client communicates with the server through the network. Wherein, the client can be but not limited to various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices. The server can be implemented as an independent server or a server cluster composed of multiple servers.

In one embodiment, as shown in FIG. 2 , a text punctuation prediction method is provided, and the method is applied to the server in FIG. 1 as an example for description, including the following steps:

101. Obtain target text without punctuation;

In this embodiment, the server may acquire the target text without punctuation according to actual needs or application scenarios. For example, the server can communicate with the client, the client provides questions to users in a certain place, the user inputs a voice question through the microphone of the client, the client uploads the voice question to the server, and the server converts the voice question into words. Then the text is obtained, which is generally the target text without punctuation. Alternatively, the server can also perform the task of punctuation recognition for a large number of discourse texts. A database collects a large number of discourse texts in advance, and then transmits multiple discourse texts to the server through the network. The server needs to punctuate these discourse texts respectively. prediction, so that these discourse texts are target texts without punctuation to be predicted respectively. It can be understood that the server can also obtain the target texts to be predicted by punctuation in various ways, which will not be repeated here.

It should be noted that the text mentioned in this embodiment generally refers to the vocabulary text, that is, the text content obtained from the words spoken by a person through sound-to-word conversion.

102. Perform word segmentation processing on the target text to obtain each target word in the target text;

It can be understood that, when predicting punctuation, it is necessary to accurately grasp the possible positions of punctuation, and the position of punctuation is closely related to each word in the target text, which requires the server to perform word segmentation on the target text to obtain all the describe each target word in the target text. For example, the target text is "Hello, I will reply you tomorrow", after word segmentation, you can get a total of 5 words "hello", "me", "tomorrow", "reply", "you", these 5 words The terms are the respective target terms.

In particular, when performing word segmentation processing on the target text, third-party software such as stuttering word segmentation can be used to implement word segmentation processing to obtain each target word.

In order to reduce the interference information in the target text and ensure the accuracy of subsequent word segmentation and input network model for recognition, further, before step 102, the method further includes: deleting the specified text in the target text, the specified text at least Include stop words. It is understandable that the stop words mentioned here may refer to single Chinese characters that are used with a particularly high frequency, such as Chinese characters without actual linguistic meaning such as "的" and "的". Before executing step 102, the server may delete the specified text in the target text. For example, assuming that the specified text includes stop words, and the target text includes the text "I came to work today", the server may first delete the "now" in the target text. " and other meaningless stop words are deleted, so as to get the deleted text "I'm coming to work today".

103. Perform vectorization processing on each of the target words to obtain each target vector corresponding to each of the target words;

After obtaining each target word, in order to facilitate the identification and learning of the subsequent network model, the server needs to perform vectorization processing on each target word, that is, the word is converted into a vector representation, so as to obtain the each target word Each target vector corresponding to the word. Specifically, the server may record each target word in the form of a one-dimensional matrix (one-dimensional vector).

For ease of understanding, in a specific application scenario, as shown in FIG. 3 , further, the step 103 may specifically include:

201. For each target word in each of the target words, search for whether each target word is recorded in the preset dictionary, if so, go to step 202, if not, go to step 203, and so on. The dictionary records the correspondence between words and one-dimensional vectors;

202. Obtain a one-dimensional vector corresponding to each of the target words;

203. Convert each target word into a first vector by loading the word vector of the first third-party platform;

204. Convert each target word into a second vector by loading the word vector of the second third-party platform;

205, splicing the first vector and the second vector, obtaining a one-dimensional vector as the one-dimensional vector corresponding to each target word;

206. Record the one-dimensional vector obtained by splicing and the target word corresponding thereto in the dictionary.

For the above step 201, when the server converts each target word into a vector, it can convert these target words one by one, or it can convert multiple target words at the same time in a multi-threaded manner, and each thread can convert the target words at the same time. A target word for vector transformation. Specifically, during the vector transformation process for each target word, first, the server may retrieve whether the target word is recorded in a preset dictionary. It should be noted here that, in order to facilitate the conversion of words into vectors, the server may be provided with a dictionary in advance, and the dictionary records the one-to-one correspondence between words and one-dimensional vectors. For example, you can set "hello" to correspond to "vector 1", "me" to correspond to "vector 2", "tomorrow" to correspond to "vector 3", "reply" to correspond to "vector 4", " "you" corresponds to "No. 5 vector"..., perfect the dictionary by exhausting all the words as much as possible, so that when each target word in the target text needs to be transformed, the server can use a preset dictionary to convert the target Each target word in the text is converted into each one-dimensional vector.

Therefore, if the server detects that the target word is recorded in the dictionary, it means that the one-dimensional vector corresponding to the target word is also recorded in the dictionary; otherwise, the one-dimensional vector corresponding to the target word is not recorded.

For the above step 202, it can be understood that if it is detected and found that each target word is recorded in the preset dictionary, it means that a one-dimensional vector corresponding to each target word is recorded in the dictionary. Therefore, The server can obtain a one-dimensional vector corresponding to each target word from the dictionary.

For the above step 203, it can be understood that if it is detected that each target word is not recorded in the preset dictionary, it means that the one-dimensional vector corresponding to each target word is not recorded in the dictionary. This is because it is often difficult to exhaust all the words when the server presets the dictionary. Even if it costs a lot of money to exhaust all the words and record them in the dictionary, due to the rapid increase of the current amount of social information every day, new words will be generated almost every day, such as Internet terms, so the default dictionary may not include some words. Faced with this situation, in this embodiment, the vector transformation of the target word can be realized while adding new words to the dictionary to improve the dictionary. Specifically, the server firstly converts each target word into a first vector by loading the word vector of the first third-party platform. It can be known that, because third-party platforms are often updated in a timely manner, the word vectors loaded on them generally cover all the words that may currently appear, so the target word can be converted into the first vector.

For the above step 204, in order to increase the accuracy of vector conversion and reduce the error rate, this embodiment also converts each target word into a second vector by loading the word vector of the second third-party platform. It can be seen that the second third-party platform and the first third-party platform are two different platforms, and the word vectors loaded on each are also different.

For the above step 205, after obtaining the first vector and the second vector, the server may obtain a one-dimensional vector from the first vector and the second vector as the one-dimensional vector corresponding to each target word. Specifically, the first vector and the second vector corresponding to the same word may be spliced together one after the other, that is, the tail of the first vector is immediately followed by the head of the second vector, thereby obtaining a new one-dimensional vector. It can be seen that since the first vector and the second vector come from two word vectors on different platforms, the two are different. This embodiment integrates the transformation rules of the two platforms, which can reduce the error of vector transformation as a whole. Notice It is also ensured that each one-dimensional vector has a sufficient length to improve the accuracy of subsequent use.

For the above-mentioned step 206, it can be understood that the one-dimensional vector obtained by splicing is a new one-dimensional vector relative to the preset dictionary. Therefore, in order to improve the dictionary, it is convenient to improve the wording in the subsequent use of the dictionary. The server can record the one-dimensional vector obtained by splicing and the target word corresponding to it in the dictionary.

104. According to the order of the target words in the target text, the target vectors are sequentially input into the network model to obtain a sequence of results output by the network model in turn, and each value in the sequence of results is respectively Characterizing the punctuation corresponding to each target word, and the network model is composed of a pre-trained LSTM network and a conditional random field;

After obtaining each target vector corresponding to each target word, the server may sequentially input each target vector into the pre-trained network model according to the order of each target word in the target text, and obtain the A sequence of results sequentially output by the network model, wherein each numerical value in the sequence of results respectively represents the punctuation corresponding to each of the target words. For example, assuming that there are 5 target vectors corresponding to the target text, which are vectors 1-5 respectively, then when step 104 is executed, first vector 1 is input into the network model, and then vector 2 is input into the network model , followed by vector 3, vector 4 and vector 5; at the same time, it can be seen that shortly after vector 1 is input to the network model, the network model will output the value corresponding to vector 1, and then output the value corresponding to vector 2. The value corresponding to the number vector, and the value corresponding to the number 3 vector, the number corresponding to the number 4 vector, and the number corresponding to the number 5 vector are output. Therefore, the 5 values output by the network model in turn constitute the result sequence.

It should be noted that the server pre-sets the corresponding relationship between each numerical value and the punctuation, which can be set according to actual needs. For example, in an application scenario, the corresponding relationship between numerical values and punctuation can be set as shown in Table 1 below:

Table I

punctuation space period comma question mark Numerical value 0 1 2 3

It can be seen that the types of the above punctuation can be increased or decreased according to the needs of the actual situation, and which value corresponds to which punctuation can be set as required, as long as the network model is trained and used using the same set of correspondences Can.

In this embodiment, the network model consists of two parts, the first half is the LSTM network, and the second half is the conditional random field. It is understandable that in the punctuation prediction application scenario of unpunctuated text, the LSTM network is good at solving the long-order dependency problem, suitable for processing and predicting important events with relatively long intervals and delays in time series, and can well understand unpunctuated text. However, the LSTM network lacks the ability to model the output category information. Therefore, this method abandons the method of adding a fully connected layer after the LSTM network, and connects it after the LSTM network. Conditional random field (CRF, conditional random field algorithm), which can make up for this defect of the LSTM network well, make the combination of the two complement each other, and improve the accuracy of punctuation prediction for unpunctuated text.

For ease of understanding, the training process of the network model will be described in detail below. As shown in Figure 4, further, the network model can be pre-trained through the following steps:

301. Collect multiple punctuation texts;

302. Separate the punctuation and text in each of the collected vocabulary texts to obtain each sample text and each punctuation set corresponding to each of the sample texts;

303. For each punctuation set, determine the first numerical value corresponding to each punctuation in the each punctuation set according to the preset numerical punctuation corresponding relationship, and form each punctuation set with each of the first numerical values. The corresponding standard sequence, the numerical punctuation correspondence records the one-to-one correspondence between numerical values and punctuation;

304. Perform word segmentation processing on the sample texts to obtain each sample word in each of the sample texts;

305. Perform vectorization processing on each sample word in each of the sample texts to obtain each sample vector corresponding to each of the sample words;

306. For each sample text in each of the sample texts, according to the order of each sample word in each sample text, input each sample vector into the LSTM network in the network model in turn, and obtain the output of the LSTM network in turn. Each intermediate vector of ;

307. Input each of the intermediate vectors into the conditional random field in the network model, respectively, to obtain a sample sequence output by the conditional random field, where each value in the sample sequence represents the correspondence of each sample word respectively. punctuation;

308. Taking the output sample sequence as the adjustment target, adjust the parameters of the LSTM network and the weight coefficient of the conditional random field to minimize the obtained sample sequence and the standard sequence corresponding to each sample text error between;

309. If the error between the sample sequence and the standard sequence corresponding to each sample text satisfies a preset training termination condition, determine that the network model has been trained.

For the above step 301, in this embodiment, the staff can collect a large number of lexical texts in different application scenarios, for example, can collect lexical texts when users ask questions, collect lexical texts when users complain, and collect lexical texts when users chat text, etc. When collecting the discourse texts, the server can collect a large amount of original discourse texts through channels such as professional knowledge bases and network databases. It should be noted that these vernacular texts need to be punctuated. If the collected original vernacular texts do not have punctuation, punctuation can be added to them manually.

For the above step 302, during the training, the input is the vocabulary text without punctuation, so the server can separate the punctuation and the text in each collected vocabulary text, and obtain each sample text and the corresponding sample text. collection of punctuation. For example, a collected phrasal text is "what products do you have?", after separating this phrasal text, the sample text "what products do you have" and a set of punctuation points "?" (there are four spaces in front of the question mark).

For the above step 303, it can be understood that, in order to facilitate the processing of the subsequent steps, after the punctuation set is separated from the vocabulary text in step 302, the server can also convert these punctuation sets into a sequence consisting of numerical values, that is, a standard sequence . Specifically, each punctuation in each punctuation set is converted into a first numerical value according to the above-mentioned numerical punctuation correspondence, and then the first numerical values are arranged to obtain a standard sequence. For example, for example, the above punctuation set "?", referring to the corresponding relationship shown in Table 1 above, the standard sequence can be obtained as "00003".

For the above-mentioned step 304, similar to the above-mentioned step 102, before the network model training is performed, the word segmentation processing also needs to be performed on these sample texts. Therefore, the server may perform word segmentation processing on the sample texts to obtain each sample word in each of the sample texts. For example, the sample text is "what products do you have", after word segmentation, you can get 4 sample words including "you", "you", "what", and "products".

In particular, when performing word segmentation processing on the sample text, third-party software such as stuttering word segmentation can be used to implement word segmentation processing to obtain each sample word.

In order to reduce the interference information in the sample text and ensure the accuracy of subsequent word segmentation and input network model training, further, before step 304, the method further includes: deleting the specified text in the sample text, the specified text at least Include stop words. It is understandable that the stop words mentioned here may refer to single Chinese characters that are used with a particularly high frequency, such as Chinese characters without actual linguistic meaning such as "的" and "的". Before executing step 304, the server may delete the specified text in the sample text. For example, assuming that the specified text includes stop words, the sample text includes the text "I'm coming to work today", and the server may first delete the "now" in the sample text. " and other meaningless stop words are deleted, so as to get the deleted text "I'm coming to work today".

For the above step 305, in the same way as the above step 103, after each sample word is obtained, in order to facilitate the identification and learning of the subsequent network model, the server needs to perform vectorization processing on the each sample word, that is, convert the word into vector, so as to obtain each target vector corresponding to each sample word. Specifically, the server may record each sample word in the form of a one-dimensional matrix (one-dimensional vector).

For the above step 306, it can be understood that when training the network model, training is performed separately for each sample text in each of the sample texts. The server may sequentially input each sample vector into the LSTM network in the network model for training according to the order of each sample word in each sample text, and obtain each intermediate vector sequentially output by the LSTM network. For example, assuming that there are 4 sample vectors of a certain sample text, which are No. 1-4 vectors respectively, then when step 306 is executed, the No. 1 vector is first input to the LSTM network, and then the No. 2 vector is input to the LSTM network , followed by the No. 3 vector and No. 4 vector; at the same time, it can be seen that not long after the No. 1 vector is input to the LSTM network, the LSTM network will output the intermediate vector corresponding to the No. 1 vector, and then output the No. 2 vector corresponding to the , and output the intermediate vector corresponding to the No. 3 vector and the intermediate vector corresponding to the No. 4 vector. It can be understood that, based on the feature that the LSTM network can perform short-term memory on the content of the text, the intermediate vector output by the LSTM network contains more text information than the input sample vector, which is the basis for the present invention to perform punctuation prediction on text without punctuation. .

Regarding the LSTM network, it can overcome the shortcomings of the traditional RNN (Recurrent Neural Network) that cannot handle long-distance dependencies. LSTM has three gates, namely forget gate, input gate and output gate. Among them, the first forget gate represents the information discarded from the previous cell state, and the value ranges from 0 to 1. The smaller the value, the more information to be discarded. The next input gate represents how much new information is added to the cell state. The final output gate will get the corresponding output according to the current cell state and new information, and update the cell state. For the specific network structure of LSTM, you can refer to the existing materials, which will not be repeated here.

For the above step 307, after obtaining each intermediate vector output by the LSTM network, the server may input each intermediate vector into the conditional random field in the network model, and obtain the sample sequence output by the conditional random field, wherein, Each numerical value in the sample sequence respectively represents the punctuation corresponding to each sample word.

It should be noted that CRF is Conditional Random Fields, which is a conditional probability distribution model of another set of output random variables under the condition of a given set of input random variables. It is a discriminant probability undirected graph. The model, since it is a discriminant, models a conditional probability distribution. Therefore, in this embodiment, the CRF can select a sequence with the highest probability as the sample sequence from various possible output sequences according to each intermediate vector given by the LSTM network. It can be seen that a CRF is usually composed of multiple feature functions, and each feature function corresponds to a different weight coefficient. When training the CRF, the training of the CRF is completed by determining these weight systems.

For the above step 308, it can be understood that the process of training the network model in this embodiment is the process of training the LSTM network and the conditional random field, and the parameters of the LSTM network and the weight coefficients of the conditional random field need to be adjusted. For example, suppose that for a sample text "what products do you have", after the sample vectors corresponding to the four sample words in the sample text are input into the LSTM network + conditional random field in turn, the final sample sequence output by the conditional random field is [ 00104], and the standard sequence corresponding to the sample text is [00003], the server can detect and know that there is an error between the two, for this reason, the server can adjust the parameters of the LSTM network and the weight coefficient of the conditional random field as much as possible. Make the output of the network model close to [00003].

When performing step 308 to adjust the parameters of the LSTM network and the weight coefficient of the conditional random field, the adjustment may also be performed by using an existing back-propagation algorithm, which will not be described further.

For the above step 309, the server can determine whether the error between the sample sequence and the standard sequence corresponding to each sample text satisfies the preset training termination condition, and if so, indicates the parameters and weights in the network model If the coefficient has been adjusted in place, it can be determined that the network model has been trained; otherwise, if it is not satisfied, it means that the network model needs to continue training. Wherein, the training termination condition may be preset according to the actual use situation. Specifically, the training termination condition may be set as: if the error between the sample sequence and the standard sequence corresponding to each sample text is less than If the error value is specified, it is considered to meet the preset training termination condition. Alternatively, it can also be set as: the above steps 306-308 are performed using the vocabulary text in the verification set, and if the error between the sample sequence output by the network model and the standard sequence is within a certain range, it is considered that it satisfies the preset training. Termination condition. The collection of discourse texts in the verification set is similar to the above step 301. Specifically, after a large amount of discourse texts can be collected by performing the above step 301, a certain proportion of the collected discourse texts can be divided into a training set, and the remaining discourse texts can be divided into training sets. The text is divided into a validation set. For example, 80% of the collected discourse texts can be randomly divided as the samples of the training set for the subsequent training of the network model, and the other 20% can be divided into the subsequent verification of whether the training of the network model is completed, that is, whether the preset training termination conditions are met. A sample of the validation set.

105. Determine each punctuation corresponding to each numerical value according to a preset numerical punctuation correspondence, where the numerical punctuation correspondence records the one-to-one correspondence between the numerical value and the punctuation;

After obtaining the result sequence output by the network model, the server may determine each punctuation corresponding to each numerical value according to the preset numerical punctuation corresponding relationship. For example, if each target vector corresponding to "Hello, I will reply you tomorrow" is input into the network model, the result sequence is [20001], then according to the correspondence in Table 1 above, the five punctuation points corresponding to the result sequence can be obtained respectively. is ",", space, space, space, ".".

106. For each punctuation in the various punctuations, insert the each punctuation into the target text at the position behind the target word corresponding to each punctuation to obtain the punctuation prediction after the term text, the The latter position refers to a position in the target text that is located after the target word and is immediately adjacent to the target word.

It can be understood that, after each punctuation is determined, the server inserts the punctuation into the corresponding position of the target text to obtain the punctuation-predicted phrasal text, and completes the addition of punctuation to the target text. Following the above example, after getting the 5 punctuations of ",.", add them to the target text "Hello, I will reply you tomorrow", and get the text of speech "Hello, I will reply you tomorrow."

For ease of understanding, as shown in FIG. 5 , further, the foregoing step 106 may specifically include:

401. Determine the first punctuation in the result sequence as the current punctuation;

402. Determine the first target word in the target text as the current word;

403, inserting the current punctuation into the position before the current word and the next word in the target text, and the next word refers to the next word of the current word in the target text;

404. If the current punctuation is not the last punctuation of the result sequence, the next punctuation of the current punctuation in the result sequence is determined as the new current punctuation, and the next punctuation of the current word in the target text is determined. The word is determined to be the new current word, and then returns to step 403;

405. If the current punctuation is the last punctuation of the result sequence, determine that the target text is the punctuation-predicted vocabulary text.

For the above step 401, following the above example, the result sequence is [20001], the first punctuation is ",", and "," is determined as the current punctuation.

For the above step 402, the target text is "Hello, I will reply you tomorrow", and the first target word is "Hello", so that "Hello" is determined as the current word.

For the above step 403, "," is inserted after "Hello", so that the target text is updated to "Hello, I will reply you tomorrow". In this case, the next word is "me" after "hello".

For the above step 404, the server determines that "," is not the last punctuation of the result sequence, so it can determine "" (space) as the new current punctuation, determine "I" as the new current word, and return to execute Step 403. It can be seen that when step 403 is executed, "" is inserted after "I", so that the target text is updated to "Hello, I will reply you tomorrow". Then, the server continues to judge that "" is not the last node of the result sequence, so "" (the second space) can be determined as the new current punctuation, and "tomorrow" can be determined as the new current word, so as to analogy. Until the current punctuation is ".", the server determines that "." is the last punctuation of the result sequence, so step 405 is executed.

For the above step 405, when the current punctuation is ".", it means that all punctuations in the result sequence have been added to the target text, and the target text is updated to "Hello, I will reply you tomorrow.", it can be seen that, The target text has completed punctuation prediction and addition, so the server can determine that the target text is the punctuation-predicted phrasal text.

In the embodiment of the present invention, first, a target text without punctuation is obtained; then, word segmentation is performed on the target text to obtain each target word in the target text; then, each target word is vectorized processing, to obtain each target vector corresponding to each target word; then, according to the order of each target word in the target text, input each target vector into the network model in turn to obtain the The result sequence output by the network model in turn, each numerical value in the result sequence represents the punctuation corresponding to each target word, and the network model is composed of a pre-trained LSTM network and a conditional random field; secondly, according to The preset numerical punctuation corresponding relationship determines each punctuation corresponding to each numerical value, and the numerical punctuation corresponding relationship records the one-to-one correspondence between the numerical value and the punctuation; finally, for each punctuation in the various punctuations, the Each punctuation is inserted into the back position of the target word corresponding to each punctuation in the target text, to obtain a punctuation-predicted lexical text, where the back position refers to the position behind the target word in the target text , and next to the target word. It can be seen that the present invention can accurately predict the punctuation of the target text through the pre-trained LSTM network and the preset conditional random field, complete the addition of punctuation to the text without punctuation, and improve the efficiency of text punctuation prediction, so as to facilitate the subsequent natural language Handles direct use of text.

It should be understood that the size of the sequence numbers of the steps in the above embodiments does not mean the sequence of execution, and the execution sequence of each process should be determined by its functions and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

In one embodiment, a text punctuation prediction apparatus is provided, and the text punctuation prediction apparatus is in one-to-one correspondence with the text punctuation prediction method in the above-mentioned embodiment. As shown in FIG. 6 , the text punctuation prediction device includes a text acquisition module 501 , a word segmentation processing module 502 , a word vectorization module 503 , a vector input module 504 , a punctuation determination module 505 and a punctuation insertion module 506 . The detailed description of each functional module is as follows:

A text acquisition module 501, configured to acquire target text without punctuation;

A word segmentation processing module 502, configured to perform word segmentation processing on the target text to obtain each target word in the target text;

A word vectorization module 503, configured to perform vectorization processing on the respective target words to obtain respective target vectors corresponding to the respective target words;

The vector input module 504 is configured to sequentially input the respective target vectors into the network model according to the order of the respective target words in the target text, and obtain a sequence of results output by the network model in turn, the sequence of results. Each numerical value in represents the punctuation corresponding to each target word, and the network model is composed of a pre-trained LSTM network and a conditional random field;

The punctuation determination module 505 is configured to determine each punctuation corresponding to each numerical value according to the preset numerical punctuation correspondence, which records the one-to-one correspondence between the numerical value and the punctuation;

The punctuation insertion module 506 is configured to, for each punctuation in the various punctuations, insert the each punctuation into the back position of the target word corresponding to the each punctuation in the target text, and obtain the words after punctuation prediction technical text, the latter position refers to a position in the target text that is located after the target word and is close to the target word.

As shown in Figure 7, further, the network model can be pre-trained by the following modules:

A vocabulary text collection module 507, configured to collect a plurality of punctuated vocabulary texts;

The punctuation text separation module 508 is used to separate the punctuation and text in each collected vocabulary text, and obtain each sample text and each punctuation set corresponding to each sample text;

The first numerical value determination module 509 is configured to, for each punctuation set, determine the first numerical value corresponding to each punctuation in the each punctuation set according to the preset numerical punctuation corresponding relationship, and form each of the first numerical values With the standard sequence corresponding to each punctuation set, the numerical punctuation correspondence records the one-to-one correspondence between numerical values and punctuation;

The sample word segmentation processing module 510 is configured to perform word segmentation processing on the sample texts to obtain each sample word in each of the sample texts;

The sample vectorization module 511 is configured to perform vectorization processing on each sample word in each of the sample texts to obtain each sample vector corresponding to each of the sample words;

The sample vector input module 512 is used to input each sample vector into the LSTM network in the network model in turn according to the order of each sample word in each sample text for each sample text in each of the sample texts, to obtain Each intermediate vector output by the LSTM network in turn;

The random field module 513 is configured to respectively input each of the intermediate vectors into the conditional random field in the network model to obtain a sample sequence output by the conditional random field, and each value in the sample sequence represents the Punctuation corresponding to each sample word;

The parameter coefficient adjustment module 514 is used to adjust the parameters of the LSTM network and the weight coefficient of the conditional random field with the output sample sequence as the adjustment target, so as to minimize the difference between the obtained sample sequence and each of the The error between the standard sequences corresponding to the sample text;

The training completion determination module 515 is configured to determine that the network model has been trained if the error between the sample sequence and the standard sequence corresponding to each sample text satisfies a preset training termination condition.

As shown in FIG. 8, further, the punctuation insertion module 506 may include:

The current punctuation determination unit 5061 is used to determine the first punctuation in the result sequence as the current punctuation;

The current word determination unit 5062 is used to determine the first target word in the target text as the current word;

The inserting unit 5063 is configured to insert the current punctuation into the position before the current word and the next word in the target text, where the next word refers to the lower part of the current word in the target text. a word;

A new punctuation determining unit 5064 is configured to determine the next punctuation of the current punctuation in the result sequence as the new current punctuation if the current punctuation is not the last punctuation of the result sequence, and set the target text in the The next word of the current word is determined as the new current word, and then returns to the step of inserting the current punctuation into the position before the current word and the next word in the target text;

The prediction completion determining unit 5065 is configured to determine that the target text is the term text after punctuation prediction if the current punctuation is the last punctuation of the result sequence.

Further, the word vectorization module may include:

A word retrieval unit, used for each target word in each of the target words, to retrieve whether each target word is recorded in a preset dictionary, and the dictionary records the word and the one-dimensional vector Correspondence between;

a vector obtaining unit, configured to obtain a one-dimensional vector corresponding to each target word if each of the target words is recorded in the preset dictionary;

The first word conversion unit is used to convert each target word into the first word vector by loading the word vector of the first third-party platform if the preset dictionary does not record each of the target words. vector;

A second word conversion unit, configured to convert each of the target words into a second vector by loading the word vector of the second third-party platform;

A vector splicing unit, for splicing the first vector and the second vector to obtain a one-dimensional vector as the one-dimensional vector corresponding to each target word;

A word recording unit, configured to record the one-dimensional vector obtained by splicing and the target word corresponding to it in the dictionary.

Further, the text punctuation prediction device may also include:

A specified text deletion module, configured to delete specified text in the target text, where the specified text includes at least stop words.

For the specific limitation of the text punctuation prediction apparatus, reference may be made to the above limitation on the text punctuation prediction method, which will not be repeated here. Each module in the above-mentioned text punctuation prediction apparatus may be implemented in whole or in part by software, hardware and combinations thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

In one embodiment, a computer device is provided, and the computer device may be a server, and its internal structure diagram may be as shown in FIG. 9 . The computer device includes a processor, memory, a network interface, and a database connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium, an internal memory. The nonvolatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the execution of the operating system and computer programs in the non-volatile storage medium. The database of the computer device is used to store the data involved in the text punctuation prediction method. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer program, when executed by a processor, implements a text punctuation prediction method.

In one embodiment, a computer device is provided, which includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the method for predicting text punctuation in the above embodiment is implemented. steps, such as step 101 to step 106 shown in FIG. 2 . Alternatively, when the processor executes the computer program, the functions of each module/unit of the text punctuation prediction apparatus in the above-mentioned embodiment are implemented, for example, the functions of modules 501 to 506 shown in FIG. 6 . To avoid repetition, details are not repeated here.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the text punctuation prediction method in the above-mentioned embodiment are implemented, for example, step 101 shown in FIG. 2 . Go to step 106 . Alternatively, when the computer program is executed by the processor, the functions of each module/unit of the text punctuation prediction apparatus in the above-mentioned embodiments, such as the functions of modules 501 to 506 shown in FIG. 6 , are implemented. To avoid repetition, details are not repeated here.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage In the medium, when the computer program is executed, it may include the processes of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database or other medium used in the various embodiments provided in this application may include non-volatile and/or volatile memory. Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Road (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. Module completion, that is, dividing the internal structure of the device into different functional units or modules to complete all or part of the functions described above.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it is still possible to implement the foregoing implementations. The technical solutions described in the examples are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the within the protection scope of the present invention.

Claims (10)

1. a kind of text punctuate prediction technique characterized by comprising

Obtain the target text without punctuate；

Word segmentation processing is carried out to the target text, obtains each target words in the target text；

Vectorization processing is carried out respectively to each target words, obtain the corresponding each target of each target words to Amount；

According to order of each target words in the target text, each object vector is sequentially input to net Network model, obtains the result sequence that the network model is sequentially output, and each numerical value in the result sequence characterizes respectively The corresponding punctuate of each target words, the network model is by preparatory trained LSTM network and condition random field group At；

The corresponding each punctuate of each numerical value, the numerical value punctuate pair are determined respectively according to preset numerical value punctuate corresponding relationship Answer the one-to-one relationship of relation record numerical value and punctuate；

For each punctuate in each punctuate, by each punctuate be inserted into the target text with it is described each Punctuate corresponds to the back location of target words, obtains art text after punctuate is predicted, the back location refers to the target It is located at behind the target words and abuts the position of the target words in text.

2. text punctuate prediction technique according to claim 1, which is characterized in that the network model passes through following steps It trains in advance:

Collect art text if multiple band punctuates；

Punctuate in each words art text being collected into is separated with text, obtain each sample text and with each sample The corresponding each punctuate set of text；

For each punctuate set, determined respectively according to preset numerical value punctuate corresponding relationship each in each punctuate set Corresponding first numerical value of a punctuate, and standard sequence corresponding with each punctuate set is formed with each first numerical value Column, the numerical value punctuate corresponding relationship have recorded the one-to-one relationship of numerical value and punctuate；

Word segmentation processing is carried out to the sample text respectively, obtains each sample words in each sample text；

Vectorization processing is carried out to each sample words in each sample text respectively, is obtained and each sample word The corresponding each sample vector of word；

For each sample text in each sample text, according to time of each sample words in each sample text Each sample vector is sequentially input into the network model LSTM network by sequence, obtains what the LSTM network was sequentially output Each intermediate vector；

Each intermediate vector is input in the network model conditional random field respectively, obtains the condition random field The sample sequence of output, each numerical value in the sample sequence characterize the corresponding punctuate of each sample words respectively；

Using the sample sequence of output as adjustment target, adjust the LSTM network parameter and the condition random field Weight coefficient, to minimize the mistake between obtained sample sequence standard sequence corresponding with each sample text Difference；

If the error between sample sequence standard sequence corresponding with each sample text meets preset training eventually Only condition, it is determined that the network model has trained.

3. text punctuate prediction technique according to claim 1, which is characterized in that described in each punctuate Each punctuate is inserted into the target text position behind target words corresponding with each punctuate by each punctuate It sets, obtaining art text after punctuate is predicted includes:

First punctuate in the result sequence is determined as current punctuate；

First aim words in the target text is determined as current words；

The current punctuate is inserted into current words and the position before next words, next word in the target text Word refers to next words of current words described in the target text；

If the current punctuate is not the last one punctuate of the result sequence, by punctuate current in the result sequence Next punctuate is determined as new current punctuate, and next words of words current in the target text is determined as new Current words, return again to execute it is described by the current punctuate be inserted into the target text current words and next words it The step of preceding position；

If the current punctuate is the last one punctuate of the result sequence, it is determined that the target text is after punctuate is predicted If art text.

4. text punctuate prediction technique according to claim 1, which is characterized in that described to each target words point Not carry out vectorization processing, obtaining the corresponding each object vector of each target words includes:

For each target words in each target words, retrieves and whether recorded each mesh in preset dictionary Marking-up word, the dictionary have recorded the corresponding relationship between words and one-dimensional vector；

If record has each target words in preset dictionary, obtain it is corresponding with each target words it is one-dimensional to Amount；

If in preset dictionary without record have each target words, by load the first third-party platform word to Amount, converts primary vector for each target words；

By loading the term vector of the second third-party platform, secondary vector is converted by each target words；

Splice the primary vector and secondary vector, it is corresponding one-dimensional as each target words to obtain an one-dimensional vector Vector；

The obtained one-dimensional vector will be spliced and corresponding target words is recorded to the dictionary.

5. text punctuate prediction technique according to any one of claim 1 to 4, which is characterized in that the target Text carries out word segmentation processing, before obtaining each target words in the target text, further includes:

The specified text in the target text is deleted, the specified text includes at least stop words.

6. a kind of text punctuate prediction meanss characterized by comprising

Text obtains module, for obtaining the target text without punctuate；

Word segmentation processing module obtains each target in the target text for carrying out word segmentation processing to the target text Words；

Words vectorization module obtains each target for carrying out vectorization processing respectively to each target words The corresponding each object vector of words；

Vector input module, for the order according to each target words in the target text, by each mesh Mark vector is sequentially input to network model, obtains the result sequence that the network model is sequentially output, in the result sequence Each numerical value characterizes the corresponding punctuate of each target words respectively, and the network model is by preparatory trained LSTM net Network and condition random field composition；

Punctuate determining module, for determining the corresponding each mark of each numerical value respectively according to preset numerical value punctuate corresponding relationship Point, the numerical value punctuate corresponding relationship have recorded the one-to-one relationship of numerical value and punctuate；

Punctuate is inserted into module, for for each punctuate in each punctuate, each punctuate to be inserted into the mesh The back location for marking target words corresponding with each punctuate in text, obtains art text after punctuate is predicted, after described Face position refers to the position for being located at behind the target words and abutting the target words in the target text.

7. text punctuate prediction meanss according to claim 6, which is characterized in that the network model passes through with lower module It trains in advance:

Talk about art text collection module, the art text for collecting multiple band punctuates；

Punctuate text separation module obtains each sample for separating the punctuate in each words art text being collected into text This text and each punctuate set corresponding with each sample text；

First numerical value determining module is determined for being directed to each punctuate set according to preset numerical value punctuate corresponding relationship respectively Corresponding first numerical value of each punctuate in each punctuate set out, and with each first numerical value form with it is described each The corresponding standard sequence of punctuate set, the numerical value punctuate corresponding relationship have recorded the one-to-one relationship of numerical value and punctuate；

Sample word segmentation processing module obtains each sample text for carrying out word segmentation processing respectively to the sample text In each sample words；

Sample vector module, for carrying out vectorization processing respectively to each sample words in each sample text, Obtain each sample vector corresponding with each sample words；

Sample vector input module, for being directed to each sample text in each sample text, according to each sample words Each sample vector is sequentially input into the network model LSTM network, obtains institute by the order in each sample text State each intermediate vector that LSTM network is sequentially output；

Random field module is obtained for each intermediate vector to be input in the network model conditional random field respectively The sample sequence exported to the condition random field, each numerical value in the sample sequence characterize each sample respectively The corresponding punctuate of words；

Parameter coefficient adjustment module, for adjusting the LSTM network using the sample sequence of output as adjustment target The weight coefficient of parameter and the condition random field, to minimize the obtained sample sequence and each sample text pair The error between standard sequence answered；

Determining module is completed in training, if between sample sequence standard sequence corresponding with each sample text Error meets preset trained termination condition, it is determined that the network model has trained.

8. text punctuate prediction meanss according to claim 6 or 7, which is characterized in that the punctuate is inserted into module and includes:

Current punctuate determination unit, for first punctuate in the result sequence to be determined as current punctuate；

Current words determination unit, for the first aim words in the target text to be determined as current words；

It is inserted into unit, for the current punctuate to be inserted into current words and the position before next words in the target text It sets, next words refers to next words of current words described in the target text；

New punctuate determination unit will be described if for the current punctuate not being the last one punctuate of the result sequence As a result next punctuate of current punctuate is determined as new current punctuate in sequence, and by words current in the target text Next words is determined as new current words, returns again to and executes step 403；

Determination unit is completed in prediction, if for the current punctuate being the last one punctuate of the result sequence, it is determined that institute Stating target text is art text after punctuate is predicted.

9. a kind of computer equipment, including memory, processor and storage are in the memory and can be in the processor The computer program of upper operation, which is characterized in that the processor realized when executing the computer program as claim 1 to Text punctuate prediction technique described in any one of 5.

10. a kind of computer readable storage medium, the computer-readable recording medium storage has computer program, and feature exists In realization text punctuate prediction side as described in any one of claims 1 to 5 when the computer program is executed by processor Method.