CN110008469A - A Multi-level Named Entity Recognition Method - Google Patents

Abstract

The present invention proposes a kind of multi-level name entity recognition method, comprising: S1 pre-processes data text, obtains vocabulary C；Term vector S2 good using pre-training, in conjunction with the image information features sequence of text, the vector of obtained text is indicated；S3 encodes the vector expression of the text, the Text eigenvector sequence after being encoded；S4 is decoded the Text eigenvector sequence with CRF model, marks out the entity in the Text eigenvector sequence；S5 is using the information of the information above of the entity of mark, hereinafter information and the entity as the candidate sequence of subsequent identification process；The Text eigenvector sequence and the candidate sequence are input to the reasoning element based on attention mechanism by S6, and attention force vector is calculated；Text eigenvector sequence inputting described in the attention vector sum into CRF model, is marked out the entity in sequence by S7.

Description

A Multi-level Named Entity Recognition Method

technical field

The invention relates to the field of natural language processing, in particular to a multi-level named entity recognition method.

Background technique

As a cross direction between the computer field and the artificial intelligence field, natural language processing continues to develop with the rapid development of the artificial intelligence field. Named Entity Recognition (NER) is a basic task of natural language processing. Its purpose is to identify entities with specific meanings in the text and classify them. The types of these entities mainly include person names, institution names, place and some other proper nouns. With the generation of massive big data in the Internet, the task of named entity recognition has also received increasing attention from academia and industry. It has been widely used in machine translation, intelligent question answering, information retrieval and other natural language processing tasks.

At present, other methods of named entities include traditional rule-based methods, dictionary-based traditional methods, and statistical-based traditional methods. The more representative methods are statistical-based Hidden Markov Model (HMM) and Conditional Random Field Model. (CRF), with the advent of the boom of deep learning, many neural network-based methods have emerged, such as the use of long short-term memory network (LSTM) for named entity recognition, and the combination of traditional methods and neural network methods. , with good results.

Rule-based and dictionary-based methods are very dependent on the construction of dictionaries and rules, so they are only suitable for small-scale domain-limited corpora, which are a bit stretched for large-scale corpora, and have great limitations when dealing with new vocabulary; based on Statistical methods rely on manual feature extraction, which will consume a lot of manpower and time. At present, many methods based on neural networks can solve the shortcomings of traditional methods to a certain extent, but for rare words in texts or words with unclear semantics, the recall and accuracy of such methods still need to be improved.

SUMMARY OF THE INVENTION

In view of the above-mentioned shortcomings of the prior art, the purpose of the present invention is to provide a multi-level named entity recognition method. The present invention effectively solves the problem in practical application named entity recognition by using the inference unit and the storage unit to perform multiple recognitions. For uncommon words and words with unclear semantics, the accuracy rate is not high, which improves the recall rate and accuracy rate of text information.

In order to achieve the above object and other related objects, the present invention provides a multi-level named entity recognition method, comprising the following steps:

S1 preprocesses the data text to obtain the vocabulary C;

S2 uses the pre-trained word vector, combined with the character feature information sequence of the text, to obtain the vector representation of the text;

S3 encodes the vector representation of the text to obtain an encoded text feature vector sequence;

S4 decodes the text feature vector sequence with the CRF model, and marks the entities in the text feature vector sequence;

S5 takes the preceding information, the following information and the information of the entity at the marked place as the candidate sequence of the subsequent identification process;

S6 inputs the text feature vector sequence and the candidate sequence into the inference unit based on the attention mechanism, and calculates the attention vector;

S7 inputs the attention vector and the text feature vector sequence into the CRF model, and marks the entities in the sequence.

S8 repeats steps S5-S7 until no new entity is generated in step S7.

Optionally, the data text is preprocessed to obtain the character feature information sequence of the text, which specifically includes:

First, with a period as a sign, a long text is divided into sentences;

tokenize all said sentences;

Then remove duplicate words to construct vocabulary C.

Optionally, the use of the pre-trained word vector, combined with the character feature information sequence of the text, obtains the vector representation of the text, specifically including:

The corpus is pre-trained, and a word vector can be obtained after pre-training;

Represent the input text in the form of one-hot as a sequence of character feature information;

Obtain the word vector representation of each phrase through the vocabulary C, the character feature information sequence and the pre-trained word vector;

The word embedding is performed on the sentence after word segmentation, and the vector representation X of the text is obtained.

Optionally, BiLSTM is used to encode the vector representation of the text to obtain an encoded text feature vector sequence.

Optionally, the text feature vector sequence is decoded with a CRF model, and the entities in the text feature vector sequence are marked, specifically including:

Input the text feature vector sequence H={h ₁ ,h ₂ ,h ₃ ,...,h _t } into the CRF model, and the predicted label sequence L={l ₁ ,l ₂ ,l ₃ ,... ,l _n }, l represents the label of each word, the marking result contains "BIE" or "S" is the marked entity e, the entity set is represented as E=(e ₁ ,e ₂ ,e ₃ ,…,e _m ), where m represents the number of entities.

Optionally, the predicted label sequence is calculated and obtained by the following method:

Let the label sequence be Y={y ₁ , y ₂ , y ₃ ,...,y _n }, the sequence Y represents the set of all possible labels in the BIOES labeling system, and the score function of the result of label y on the x label is:

Among them, t _j (y _i-1 ,y _i ,x,i) represents the feature function in the CRF model, indicating that given x, the previous label node y _i-1 is transferred to the current label node y In the case of _i , the value is 0 or 1, which is called the transition feature, and s _k (y _i ,x,i) indicates whether the current label node y _i is marked on x, and the value is also 0 or 1, which is called the state Features, λ _j and μ _k represent the weights of t _j and s _k respectively, and j, k represent the number of feature functions;

By indexing and normalizing the score(y,x), the conditional probability p(y|x) that labels x with y can be obtained. The specific formula for calculating p(y|x) is as follows:

Z(x) is the normalization factor, and the calculation formula is:

where y′=(y′ ₁ , y′ ₂ , y′ ₃ ,…,y′ _n ), indicating the possible label sequence;

The solution is solved by the Viterbi algorithm, and the model takes the y' with the highest probability as the labeling result, denoted as l, that is, Then the predicted label sequence is L={l ₁ ,l ₂ ,l ₃ ,...,l _n }.

Optionally, for each entity e _i (i=1,2,3,...,m) in the entity set E, combine the forward LSTM hidden layer output and the output of the backward LSTM hidden layer Represent each entity in the form of V′=[v ₁ , v ₂ , v ₃ , v ₄ ], where v ₁ represents the above information of the entity, and v ₂ represents the entity itself obtained by forward LSTM information, v ₃ represents the information of the entity itself obtained by the backward LSTM, and the latter information of the v ₄ entity, the entity sequence in the storage unit is expressed as V={V′ ₁ ,V′ ₂ ,V′ ₃ ,..., V'_m' }, m' represents the number of entities that have been stored in the storage unit, and the entity sequence in the storage unit is represented as a candidate sequence.

Optionally, the candidate sequence V and the text feature vector sequence H in the storage unit are input into the inference unit, and the degree of influence of each entity information V on the text feature vector sequence H is obtained by the inference unit, and the attention vector S is obtained;

For the text feature vector sequence H={h ₁ ,h ₂ ,h ₃ ,…,h _t } and the candidate sequence V={V′ ₁ ,V′ ₂ ,V′ ₃ ,…,V′ _m′ }, respectively calculate The dot product of hi at each moment and all _V'j ( _j =1, 2, 3,...,m') to get the attention score σ, where i=1,2,3,...,t;

At any time t, the formula for calculating the attention score of the candidate sequence V to h _t is as follows:

Use the softmax function to convert the attention score into a probability distribution as the weight α of the post-order weighted summation:

α ^t =softmax(σ ^t )

The weighted summation of the candidate sequence V is carried out to obtain any time t, and the attention vector s of the candidate sequence V to h _t is obtained. The formula is as follows:

After calculating the results at all times, the attention vector sequence S={s ₁ , s ₂ , s ₃ ,..., s _t } of the candidate sequence V to the text feature vector sequence H is obtained.

Optionally, the method further includes: calculating the similarity between the entity in the step S7 and the entity in the step S5, and when the similarity is less than the similarity threshold, the entity is regarded as a new entity.

Optionally, the similarity is calculated by the method of cosine similarity, and the specific formula is:

As mentioned above, a multi-level named entity recognition method of the present invention has the following beneficial effects:

1. Compared with the method of performing named entity recognition only once, this method uses multiple recognitions, and improves the recall rate of the named entity recognition task through multiple recognitions;

2. The existing method is more suitable for short text data, but for longer long text data, the efficiency will decrease. The present invention designs a storage unit to store important entity information and its context information, and process long text data in this way. , while designing a candidate unit to reduce the space overhead of the storage unit;

3. The reasoning unit is designed, and the reasoning unit is used for entity recognition. For some rare words in the text and some words with unclear semantics, the recognition effect can be improved, and the accuracy and recall rate of the system can be improved.

Description of drawings

In order to further illustrate the content described in the present invention, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings. It should be understood that these drawings are presented by way of example only and should not be considered as limiting the scope of the present invention.

Figure 1 is a flowchart of a named entity recognition system;

2 is a schematic diagram of a storage unit structure and a schematic diagram of a candidate unit structure;

Fig. 3 is a schematic diagram of the structure of an inference unit;

4 is a schematic diagram of the system structure;

Figure 5 is a schematic diagram of the CRF structure;

Figure 6 is a schematic diagram of the structure of the LSTM unit.

Detailed ways

The embodiments of the present invention are described below through specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments may be combined with each other under the condition of no conflict.

It should be noted that the drawings provided in the following embodiments are only used to illustrate the basic concept of the present invention in a schematic way, so the drawings only show the components related to the present invention rather than the number, shape and number of components in actual implementation. For dimension drawing, the type, quantity and proportion of each component can be changed at will in actual implementation, and the component layout may also be more complicated.

The invention provides a multi-level named entity recognition method, which can gradually recognize the unrecognized entities in the text in the first several named entity recognition processes by means of multiple recognition. The core problem to be solved by the present invention includes the following two points:

1. For the traditional named entity recognition system, the entire system will only be recognized once, so there will still be some words in the recognition result that cannot be recognized or recognized incorrectly, resulting in low recall and accuracy of named entity recognition. This situation is more serious when recognizing some uncommon words, such as rare names of people, places, etc., and recognizing words with unclear semantics in some texts;

2. For the problem that many current named entity recognition systems cannot solve the problem of long texts, the present invention uses a storage unit to store the relevant information of the entity to solve the problem.

The core idea of the present invention is to save the context information of the identified entities and the information of the entity itself, because the information composed of the context information of the entity and the information of the entity itself can be regarded as some sentence information. way to help the system to identify unrecognized entities in the text, the specific method will be explained in the following example.

The embodiment of the present invention is shown in FIG. 1, which includes the general idea of the present invention, wherein the labeling system adopted in the present invention is the BIOES labeling system, that is, “B” represents the beginning character of a multi-character entity, and “I” represents a multi-character entity. The middle character of a character entity, "O" represents other non-entity words, "E" represents the end of a multi-character entity, "S" represents a word alone as an entity, and "PER" represents a person's name, "ORG" represents an organization name, "LOC" means place name.

As shown in FIG. 1, this embodiment provides a multi-level named entity recognition method, and the method includes the following steps:

S1 preprocesses the data text to obtain the vocabulary C;

S2 uses the pre-trained word vector, combined with the character feature information sequence of the text, to obtain the vector representation of the text;

S3 encodes the vector representation of the text to obtain an encoded text feature vector sequence;

S4 decodes the text feature vector sequence with the CRF model, and marks the entities in the text feature vector sequence;

S5 takes the preceding information, the following information and the information of the entity at the marked place as the candidate sequence of the subsequent identification process;

S6 inputs the text feature vector sequence and the candidate sequence into the inference unit based on the attention mechanism, and calculates the attention vector;

S7 inputs the attention vector and the text feature vector sequence into the CRF model, and marks the entities in the sequence.

S8 repeats steps S5-S7 until no new entity is generated in step S7.

In step S1, the data text is preprocessed, which is as follows: first, using a period as a symbol, a long text is divided into sentences, the processed sentences are stored in units of actions, and the open source tool jieba is used to segment all sentences. Divide the sentence into the form of words, and then remove the repeated words to get the vocabulary. Let the vocabulary be C.

In step S2, the vector representation of the text is obtained by using the pre-trained word vector and combining the character feature information sequence of the text. Specifically, first use jieba word segmentation to segment the Chinese Wikipedia corpus, and use the open source tool word2vec to pre-train the Chinese Wikipedia corpus after word segmentation. After pre-training, a word vector can be obtained, and the dimension of the pre-trained word vector is d. . For the input text, first represent it in the form of one-hot as a sequence of character feature information, and then through the vocabulary C and the pre-trained word vector, the word vector representation of each phrase can be obtained, let the word vector be x , by performing word embedding on the sentence after word segmentation, the vector representation X of the text can be obtained, where X={x ₁ ,x ₂ ,x ₃ ,...,x _n }, X∈R ^d×n , n is the sentence after word segmentation number of words.

In step S3, step S1 and step S2 obtain the text vector representation X through the processing work of the first two steps, and input X as the input sequence into the bidirectional LSTM model. Through LSTM, text feature information can be extracted, and the feature information of a word in a certain period of time sometimes includes not only the influence of the previous words on it, but also the influence of the following words on it, so the use of two-way LSTM can The text feature vector sequence is fully extracted from two directions. The vector representation X of the text is encoded as a text feature vector sequence H by bidirectional LSTM, where H={h ₁ ,h ₂ ,h ₃ ,...,h _t }, is the output at each moment of the hidden layer of the bidirectional LSTM, Represents the output of the forward LSTM hidden layer at each moment and the output of the backward LSTM hidden layer Do stitching.

The LSTM model in this step is shown in Figure 6, and its specific formula is described as follows:

First decide which information LSTM needs to choose to forget and which information to retain, f _t represents the output of the forget gate

f _t =σ(W _f ·[h _t-1 ,x _t ]+b _f )

Then determine the information that LSTM needs to update, i _t represents the output of the memory gate, Represents the state value of the temporary memory cell in the LSTM

i _t =σ(W _i ·[h _t-1 ,x _t ]+b _i )

Then update the state of the LSTM cell, C _t , C _t-1 represent the current cell state in the LSTM, and the cell state at the previous moment, respectively

Finally output the hidden layer information, o _t represents the value of the output gate

o _t =σ(W _o [h _t-1 ,x _t ]+b _o )

h _t =o _t *tanh(C _t )

Among them, x _t is the input at the current moment, h _t-1 is the output of the hidden layer at the previous moment, h _t is the output of the hidden layer at the current moment, W is the weight matrix corresponding to each function, and b is the corresponding value of each function. Bias, σ is the sigmoid function, and tanh is the hyperbolic tangent function.

In step S4, the text feature vector sequence H={h ₁ , h ₂ , h ₃ ,...,h _t } is input into the CRF model shown in Figure 5, and the predicted label sequence L={ l ₁ ,l ₂ ,l ₃ ,...,l _n }, l represents the label of each word, the labeling result contains "BIE" or "S" is the marked entity e, the entity set is expressed as E=(e ₁ , e ₂ , e ₃ ,...,e _m ), where m represents the number of entities.

The specific calculation method of the predicted label sequence L in this step is as follows:

Let the label sequence be Y={y ₁ , y ₂ , y ₃ ,...,y _n }, the sequence Y represents the set of all possible labels in the BIOES labeling system, and the score function of the result of label y on the x label is:

Among them, t _j (y _i-1 ,y _i ,x,i) represents the feature function in the CRF model, indicating that given x, the previous label node y _i-1 is transferred to the current label node y In the case of _i , the value is 0 or 1, which is called the transition feature; s _k (y _i ,x,i) indicates whether the current label node y _i is marked on x, and the value is also 0 or 1, which is called the state Features; λ _j and μ _k represent the weights of t _j and s _k respectively, and j, k represent the number of feature functions.

By indexing and normalizing the score(y,x), the conditional probability p(y|x) of the label y on the x tag can be obtained. The specific formula for calculating p(y|x) is as follows:

Z(x) is the normalization factor, and its calculation formula is:

Among them, y′=(y′ ₁ , y′ ₂ , y′ ₃ ,..., y′ _n ), which represents the possible label sequence.

The solution is solved by the Viterbi algorithm, and the model takes the y' with the highest probability as the labeling result, denoted as l, that is, Then the predicted label sequence is L={l ₁ ,l ₂ ,l ₃ ,...,l _n }.

In step S5, step S4 recognizes some entities, but there may still be some unrecognized unrecognized words or words with unclear semantics in the text, so by storing the identified entity information, use entity information Its own information and its context information help the system to identify these rare words.

The operation of step S5 is as follows:

According to grammatical rules, a noun should be preceded or followed by a verb or a preposition, and the sentence pattern composed of a verb or a preposition and a noun contains rich textual feature information. Because the output of the hidden layer of LSTM includes the information of each moment of the text, for each entity e _i (i=1, 2, 3, ..., m) in E, combine the output of the forward LSTM hidden layer Store the information of e _i at the previous moment as v ₁ , store the information of e _i itself as v ₂ , and combine the output of the backward LSTM hidden layer Store the information at the moment after e _i as v ₃ , and store the information of e _i itself as v ₄ , then each entity is represented in the form of V′=[v ₁ , v ₂ , v ₃ , v ₄ ], which is stored in the storage unit. As shown in Figure 2, v ₁ represents the above information of the entity, v ₂ represents the information of the entity itself obtained by the forward LSTM, v ₃ represents the information of the entity itself obtained by the backward LSTM, and the back of the v ₄ entity If the text information, the entity sequence in the storage unit is represented as V={V′ ₁ , V′ ₂ , V′ ₃ ,..., V′ _m′ }, where m′ represents the number of entities that have been stored in the storage unit, then the storage The entity sequences in the cell are denoted as candidate sequences V.

Step S6: Input the candidate sequence V and the text feature vector sequence H in the storage unit into the inference unit, obtain the degree of influence of each candidate sequence V on the text feature vector sequence H through the inference unit, and obtain the attention vector S.

A schematic diagram of the reasoning unit in this step is shown in Figure 3, and an attention vector is obtained by calculating the attention degree of each entity in the storage unit to each feature vector by using the method of attention mechanism.

For the text feature vector sequence H={h ₁ ,h ₂ ,h ₃ ,…,h _t } and the candidate sequence V={V′ ₁ ,V′ ₂ ,V′ ₃ ,…,V′ _m′ }, respectively calculate The dot product of hi ( _i =1,2,3,...,t) at each moment and all V'j ( _j =1,2,3,...,m') yields the attention score σ.

The formula for calculating the attention score of V to h _t at any time t is as follows:

Then use the softmax function to convert the attention score into a probability distribution as the weight α of the post-order weighted summation:

α ^t =softmax(σ ^t )

Finally, the weighted summation of V is performed to obtain the attention vector s of V to h _t at any time t. The formula is as follows:

After calculating the results at all times, the V-to-H attention vector sequence is obtained as S={s ₁ , s ₂ , s ₃ ,...,s _t }.

In step S7, after splicing the text feature vector sequence H and the attention vector S, the splicing process is the same as the splicing process of the hidden layer of the bidirectional LSTM. The spliced vector is expressed as [H:S], and the spliced After the vector is input into the CRF model, a new labeling result is obtained. The CRF model here and the CRF model in step S4 share parameters.

In an embodiment, the method based on the multi-level named entity recognition further includes: obtaining corresponding new entities through the new labeling in step S7, and using the method in step S5 to represent them as V″=[v ₁ , v ₂ , v ₃ , v ₄ ], prepare to store V″=[v ₁ , v ₂ , v ₃ , v ₄ ] into the storage unit. If the new entity information is stored in the storage unit without screening, it may lead to an excessive amount of data in the later storage unit and waste of storage space. Therefore, before storing the new entity information in the storage unit, store it first. To the candidate unit, the structure is shown in Figure 2. By calculating the similarity between these new words in the candidate unit and the new words of the storage unit, a threshold β is set. When the similarity is less than the threshold, the new entity can be added. into the storage unit.

The similarity between words in this step is calculated by the method of cosine similarity, and the specific formula is:

In one embodiment, the multi-level named entity identification method further includes step 9: repeating the above steps S5 to S8 until no new entity is generated in step S7, the named entity identification process ends.

The above named entity process is described below through an example, as shown in Figure 4 below:

Step 1: The input sentence is "When Tom meets Jerry when Li meets Cai", first preprocess the sentence, use jieba Chinese word segmentation tool to segment the sentence, the result is "when/tom/encounter/jie" Rui / Shi / Li Mou / Encounter / Cai Mou", where "/" represents the delimiter of the word segmentation, and the word vector representation of the sentence can be obtained by using the pre-trained word vector.

Step 2: Input the word vector representation obtained in Step 1 into the bidirectional LSTM, use the bidirectional LSTM as the encoder to encode the word vector, and the output of the hidden layer of the LSTM is the encoded text feature vector sequence we need.

Step 3: Input the text feature vector in Step 2 into the CRF model of the next layer, use the CRF model as a decoder to decode the text feature vector, and perform annotation at the same time to obtain the labeling result as follows:

when tom meet Jerry Time Lee meet Cai O S-PER O S-PER O O O O

From the above results, it can be seen that the entity names "Tom (S-PER)" and "Jerry (S-PER)" are correctly labeled and recognized, while the other two words "Li" and "Cai" are not Not marked, because these two terms are relatively uncommon, so they have not been marked.

By observing the above example sentence "When Tom met Jerry when Li met Cai", it can be seen that although the latter two words "Li" and "Cai" are relatively uncommon words, but the two words The context information and the identified entities "Tom" and "Jerry" are very acquainted, and the word "encounter" appears in both "Tom meets Jerry" and "Li meets Cai", that is to say , "Tom meets Jerry" and "Li meets Cai" are similar sentence patterns, they are entities with very similar context semantics, so by storing the context information of "Tom" and "Jerry", Use their information to help the system identify the two entities "Li" and "Chai".

Step 4: Take the entities "Tom" and "Jerry" identified in Step 3 as a candidate sequence according to the format of entity above information, forward entity information, backward entity information, and entity's post information. The sequence is stored in the storage unit.

Step 5: Input the text feature vector sequence obtained in step 2 and the candidate sequence stored in the storage unit into the inference unit. After the inference unit performs calculations, the information and context information of the entities "Tom" and "Jerry" can be obtained. The degree of influence of each word in the sentence, and get a set of attention vectors.

Step 6: Input the attention vector and text feature vector sequence in Step 5 into the CRF model for decoding. The CRF model labels the two entities "Li" and "Cai" by referring to the reference vector.

Step 7: Store "Li" and "Cai" in the candidate unit, set the threshold to β, and find out that these two words are the same as "Li" and "" by calculating the similarity between them and each word in the storage unit. Tom", "Cai" and "Jerry" are more similar than the set threshold β, so these two words are not stored in the storage unit.

Step 8: After repeating the above steps 4 to 7, it is found that no new entities are generated, so the named entity recognition ends, and the final marked entity is:

when tom meet Jerry Time Lee meet Cai O S-PER O S-PER O S-PER O S-PER

After the entities identified by the above process are stored in the storage unit, as long as information similar to the entity in the storage unit appears in the subsequent text, that is, text similar to the sentence pattern in the storage unit appears, these entities can be stored. Identify it, which solves the problem of low accuracy and recall in long texts.

In this system, optionally, in order to reduce the cost of calculation, the number of cycles for identifying named entity recognition can be set in advance, instead of using "no new entity generation" as the end mark of the named entity system, which can reduce the amount of calculation. However, it will lead to a decrease in recall rate and accuracy rate. In actual application, it is necessary to weigh the pros and cons.

The above-mentioned embodiments merely illustrate the principles and effects of the present invention, but are not intended to limit the present invention. Anyone skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical idea disclosed in the present invention should still be covered by the claims of the present invention.

Claims (10)

1. a kind of multi-level name entity recognition method, which is characterized in that it includes following that this names entity recognition method at many levels Step:

S1 pre-processes data text, obtains vocabulary C；

Term vector S2 good using pre-training, in conjunction with the image information features sequence of text, the vector of obtained text is indicated；

S3 encodes the vector expression of the text, the Text eigenvector sequence after being encoded；

S4 is decoded the Text eigenvector sequence with CRF model, marks out in the Text eigenvector sequence Entity；

S5 is using the information of the information above of the entity of mark, hereinafter information and the entity as the time of subsequent identification process Select sequence；

The Text eigenvector sequence and the candidate sequence are input to the reasoning element based on attention mechanism by S6, Attention force vector is calculated；

Text eigenvector sequence inputting described in the attention vector sum into CRF model, is marked out the reality in sequence by S7 Body；

S8 repeats step S5~S7 until step S7 does not generate novel entities.

2. according to claim 1 a kind of based on multi-level name entity recognition method, which is characterized in that described to data Text is pre-processed, and is obtained the image information features sequence of text, is specifically included:

It is first mark with fullstop, is sentence one section of long text segmentation；

All sentences are segmented；

Then remove dittograph building vocabulary C.

3. according to claim 1 a kind of based on multi-level name entity recognition method, which is characterized in that described using pre- Trained term vector, in conjunction with the image information features sequence of text, the vector of obtained text is indicated, is specifically included:

Pre-training is carried out to corpus, by the available a term vector of pre-training；

The form for being one-hot by the text representation of input, as image information features sequence；

By the vocabulary C, the term vector of the image information features sequence and pre-training, obtain the word of each phrase to Amount indicates；

Word insertion is carried out to the sentence after participle, the vector for obtaining text indicates X.

4. according to claim 3 a kind of based on multi-level name entity recognition method, which is characterized in that use BiLSTM The vector expression of the text is encoded, the Text eigenvector sequence after being encoded.

5. according to claim 4 a kind of based on multi-level name entity recognition method, which is characterized in that the text Characteristic vector sequence is decoded with CRF model, is marked out the entity in the Text eigenvector sequence, is specifically included:

Text eigenvector sequence H={ h₁,h₂,h₃,…,h_tIt is input to CRF model, pass through being calculated for CRF model Prediction label sequence L={ l₁,l₂,l₃,…,l_n, what l was indicated is the label of each word, contains " BIE " in annotation results Or " S " is all the obtained entity e of mark, entity sets are expressed as E=(e₁,e₂,e₃,…,e_m), m presentation-entity number.

6. according to claim 5 a kind of based on multi-level name entity recognition method, which is characterized in that the pre- mark Label sequence calculates acquisition by the following method:

If flag sequence is Y={ y₁,y₂,y₃,…,y_n, sequence Y indicates the collection of all possible label in BIOES mark system It closes, the scoring function of this result of x marked with tag y are as follows:

Wherein, t_j(y_i-1,y_i, x, i) and characteristic function in CRF model is represented, it indicates in the case where given x, a upper label Node y_i-1It is transferred to current label node y_iThe case where, value is 0 or 1, referred to as transfer characteristic, s_k(y_i, x, i) and indicate current mark Sign node y_iWhether mark on x, value is also 0 or 1, referred to as state feature, λ_jAnd μ_kThe t respectively indicated_jAnd s_kWeight, The number of j, k expression characteristic function；

Indexation and standardization are carried out to score (y, x), it will be able to obtain stamping the conditional probability p (y | x) of y label, meter for x Calculation p (y | x) specific formula is as follows:

Z (x) is standardizing factor, calculation formula are as follows:

Wherein y '=(y '₁,y′₂,y′₃,…,y′_n), indicate possible annotated sequence；

It is solved by viterbi algorithm, model takes so that the y ' of maximum probability is to be denoted as l, i.e., as annotation resultsThen prediction label sequence is L={ l₁,l₂,l₃,…,l_n}。

7. according to claim 6 a kind of based on multi-level name entity recognition method, which is characterized in that for entity set Close each entity e in E_i, i=1,2,3 ..., m is exported in conjunction with preceding to LSTM hidden layerAnd LSTM hidden layer is defeated backward OutEach entity is expressed as V '=[v₁,v₂,v₃,v₄] form, wherein v₁That indicate is the information above of entity, v₂Table That show is the preceding entity self-information obtained to LSTM, v₃What is indicated be after the entity self-information that is obtained to LSTM, v₄Entity Information hereinafter, then the entity sequence in storage unit is expressed as V={ V₁′,V₂′,V₃′,…,V′_m′, m ' expression has been deposited into The entity number of storage unit, then the entity sequence in storage unit is expressed as candidate sequence.

8. according to claim 7 a kind of based on multi-level name entity recognition method, which is characterized in that storage unit In candidate sequence V and Text eigenvector sequence H be input in reasoning element, each entity information is obtained by reasoning element V is to the influence degree of Text eigenvector sequence H, and gain attention force vector S；

For Text eigenvector sequence H={ h₁,h₂,h₃,…,h_tAnd candidate sequence V={ V₁′,V₂′,V₃′,…,V′_m′, point The h at each moment is not calculated_iWith all V_j' dot product, the power that gains attention score σ, wherein i=1,2,3 ..., t, j=1,2, 3,...,m'；

Any time t, candidate sequence V is to h_tAttention score calculation formula it is as follows:

Attention score is converted into probability distribution as the weight α of postorder weighted sum:

α^t=softmax (σ^t)

Summation is weighted to candidate sequence V, obtains any time t, candidate sequence V is to h_tNotice that force vector s, formula are as follows:

After the result for calculating all moment, candidate sequence V is obtained to the attention sequence vector S of Text eigenvector sequence H ={ s₁,s₂,s₃,…,s_t}。

9. according to claim 8 a kind of based on multi-level name entity recognition method, which is characterized in that this method is also wrapped It includes: calculating the similarity of the entity in the entity and step S5 in the step S7, when the similarity is less than similarity threshold When, using the entity as new entity.

10. according to claim 9 a kind of based on multi-level name entity recognition method, which is characterized in that described similar Degree is calculated using the method for cosine similarity, specific formula are as follows: