CN101520775A - Chinese syntax parsing method with merged semantic information - Google Patents

Abstract

The invention discloses a Chinese syntax parsing method with merged semantic information, belonging to the technical field of natural language processing. The method comprises the following steps: step 1), extracting different hierarchical semantic classes of words according to the hyponymy of the knowledge network to obtain indexes from the words to the semantic classes; step 2), using a word in a syntactic tree as a key assignment and query the knowledge network to obtain a semantic class of the word and add the semantic class to a certain layer of the syntactic tree; step 3), using the syntactic tree after being processed in the step 2) as training data to train grammar so as to obtain a grammar model; step 4), utilizing the grammar model after being trained in the step 3) to decode a sentence to be analyzed. Compared with the prior art, the invention adopts the semantic information to disambiguate parsing so that the parsing effect is remarkably improved.

Description

A Method of Chinese Syntax Analysis Incorporating Semantic Information

technical field

The invention belongs to the technical field of natural language processing, and specifically relates to a Chinese syntax analysis method incorporating semantic information, which introduces semantic knowledge into the syntax analysis to help improve the performance of syntax analysis.

Background technique

Syntactic analysis is a very important technology in natural language processing. It analyzes how words are combined to form meaningful phrases and sentences to reveal deep language laws. The results of syntactic analysis will directly affect the understanding of natural language. In actual natural language processing applications, a high-performance syntactic analyzer is conducive to improving the performance of high-level application systems such as information extraction, information retrieval, machine translation, and automatic question answering.

The process of syntactic analysis is to deduce the grammatical structure of a sentence according to a certain algorithm given a set of grammatical models, which is usually represented by a tree structure. For example, for a sentence, "more than half of Dalian's foreign trade exports come from 'foreign-funded' enterprises.", the result of syntactic analysis can be represented by the structure tree in Figure 1(a). In this tree structure, the bottom-level leaf nodes are words, called terminators; the upper-level non-leaf nodes are called non-terminal symbols, and the bottom-level non-leaf nodes represent parts of speech, called pre-terminals. Due to the general ambiguity in natural language, multiple different grammatical structures may be analyzed for the same sentence, so it is necessary to use effective information and algorithms to resolve the existing ambiguity and find the most reasonable grammatical structure. The problem to be solved by a method of syntactic analysis.

Using the method of statistical writing can learn the biased information of vocabulary and structure from the training corpus, so as to deal with the ambiguity of syntactic structure to a certain extent. The emergence of some artificially labeled grammatical structure treebank resources (such as the Penn Treebank constructed by the University of Pennsylvania in the United States) has created conditions for proposing statistical-based syntactic analysis methods and greatly promoted the development of such technologies. Probabilistic Context-Free Grammar (PCFG: Probabilistic Context-Free Grammar) is the most researched method in statistical syntax analysis, which describes the sentence structure through a series of context-free grammatical rules, and assigns a certain probability to each rule. The advantage of this approach is that it is simple in form and can be processed in polynomial time.

A problem of the PCFG model comes from the assumption of conditional independence. Under this assumption, it is considered that the expansion of any non-terminal (that is, each node above the word node in the syntax tree) is mutually related to the expansion of other non-terminals. independent. However, through the study of the statistical distribution of non-terminals in each position in the tree bank, it is found that sometimes the expansion of a node is related to its position in the tree, but this point is ignored in simple PCFG modeling. In order to solve this problem, it is necessary to improve the basic PCFG model. There are usually two ways: introducing lexicalization information and extending non-terminal symbols. The latter is often called a non-lexicalization method. The most representative work on the introduction of lexical information is the syntactic analysis method driven by the head word. For example, Michael Collins introduced vocabulary, distance and other information for each non-terminal in the grammatical rules in his doctoral thesis to improve the grammar. The discriminative, non-lexical syntactic analysis methods mainly include manual refinement of some non-terminal symbols, and automatic refinement of tags through unsupervised learning methods to cover more language phenomena. The representative work is UC Work by Dan Klein et al. at Berkeley. However, these two methods also have their own shortcomings: the introduction of lexical information in the lexicalization method brings certain data sparse problems, and the automatic refinement of tags in the non-lexicalization method has problems such as whether the description of language phenomena is accurate.

Contents of the invention

The purpose of the present invention is to provide a Chinese syntax analysis method that incorporates semantic information, uses semantic information to help improve the performance of syntax analysis, and can also obtain semantic information with syntax constraints from the results of syntax analysis.

There have been theoretical studies showing that semantic information can help syntactic disambiguation. Semantic concepts involve the meaning, structure, and way of speaking of words. Related research can be divided into two parts: the study of the semantics (meaning) of individual words and how the meanings of individual words are combined to form the meaning of sentences. The main task of semantic analysis is to generate lexical semantic unit representations of language texts and the dependencies between them. Although syntactic analysis and semantic analysis are two different levels of language analysis, they are mutually restrictive. The word order of Chinese has strong constraints on semantics, and there are complex semantic relationships between syntactic components. In many cases, syntactic structure analysis of grammatical forms cannot explain the internal laws of sentences. Therefore, the introduction of semantics into Chinese syntactic analysis will help resolve structural ambiguity.

The premise of using semantic information is that there is a set of pre-defined semantic specifications, and the most direct way is to use existing semantic resources. The semantic resource used in our method is HowNet. HowNet is a commonsense knowledge base based on the concepts represented by English and Chinese bilinguals and the characteristics of concepts, and the basic content is to reveal the relationship between concepts and the characteristics of concepts. From it, we can get the concepts or conceptual attributes of different levels of a word as our semantic class, for example, we can get the semantic class "entity|entity=>thing|everything=>physical|substance=>inanimate| No creature=>artifact|artificial object=>implement|apparatus=>vehicle|vehicle=>LandVehicle|car", which from left to right represents the different levels of "car" in HowNet from coarse to fine Semantic class. For example, "entity|entity" is the thickest semantic class, which covers the widest range; and "LandVehicle|car" is the thinnest semantic class, which expresses the smallest meaning and is closest to "car" .

The invention solves the problem of lack of semantic information in the PCFG model by investigating the relationship between syntactic analysis and semantic analysis, and integrates semantic information into the non-lexical syntactic analysis process, and further refines the part-of-speech layer through semantic tags. By introducing semantic information, it helps syntactic analysis to resolve ambiguity, thereby improving the performance of syntactic analysis to a certain extent.

Therefore, the basic idea of the present invention is that syntax and semantics are two different levels of language analysis, they play a role together in the process of language analysis and influence each other, and semantic information is very helpful for dissolving structural ambiguity. By incorporating semantic information into the non-lexical syntactic analysis method, the performance of the syntactic analyzer is significantly improved, and the obtained analysis results include not only the syntactic modification relationship, but also the semantic category of each word.

The starting point of the present invention is to obtain a high-performance syntax analyzer, and use semantic analysis as an auxiliary means to improve the performance of syntax analysis. The basic model of syntactic analysis adopts the non-lexicalized PCFG model, which automatically refines tags through unsupervised learning methods to improve the descriptive ability of grammar, and its performance has surpassed that of lexicalized syntactic analyzers. On this basis, this method uses HowNet as a semantic dictionary to provide a certain level of semantic categories for some words in the syntax tree bank, and attach the semantic categories to the pre-terminals of the syntax tree (that is, the upper layer of the vocabulary layer) level, and trained with the labeled tree bank to obtain a grammar model containing semantic information. In the decoding part, the syntactic analysis result with semantic markup can be obtained without any special processing. Through experiments, it is found that this method effectively improves the performance of syntactic analysis.

The technical scheme of the present invention will be introduced in detail in three parts below.

1. The way semantic information is integrated into syntactic analysis

Take HowNet as the semantic dictionary, and the sememe (defined as the smallest unit of meaning) defined in it as the semantic category. Sememes have a certain hyponymy relationship in HowNet, as shown in Figure 2, according to this hyponymy relationship, different levels of semantic categories are extracted, and the semantic categories are obtained by querying the words in the syntax tree as key values. and attach semantic classes to preterminals. In order to ensure the consistency of the semantic system and alleviate the problem of data sparsity, it is necessary to ensure that the semantic classes obtained by all word queries are at the same layer in HowNet.

For words with multiple semantic categories, there is a problem of word sense disambiguation. Our strategy here is to take the first semantic category; Semantic classes are annotated. For words that do not exist in HowNet, no semantic information is added.

Figure 1 shows an example of annotation semantics. Attached Figure 1(a) is the sentence in the tree bank before labeling; Figure 1(b) is the sentence after semantic labeling. It can be seen that the strategy of introducing semantics is to attach the semantic category of a word to its corresponding on the preterminal.

For the non-terminals above the part-of-speech level, they cannot be directly obtained from HowNet. The easiest way to add them is to use a method similar to extracting the central word, using the semantic information of the pre-terminal as the central word and extracting it to the upper node. However, considering that there are many semantic categories of words, more non-terminals may be generated when appending to upper-level nodes, which will cause very serious data sparsity problems when the amount of data is insufficient. Therefore, for the upper-level non-terminals, the unsupervised automatic splitting and merging method is still used for automatic subdivision without introducing semantics.

After such processing, the upper-level pre-terminals corresponding to most of the words in the tree bank are marked with a certain layer of semantic classes in HowNet, and the tree bank is used for syntactic analysis model training, and the semantic information can be obtained. grammar model. Using this grammar to decode, the syntactic analysis result with semantic marks can be obtained, and the syntactic analysis result is more accurate.

2. Syntactic analysis model training

The basic syntactic analysis model adopted in the present invention is a non-lexical syntactic analysis model, that is, the non-terminal node label is refined in an unsupervised manner to improve the description ability of the grammar. The model is briefly described below.

In recent years, non-lexicalized PCFG syntactic analysis methods have made great progress, and the performance of the best models has reached the highest level of current syntactic analysis. The model is based on the basic PCFG framework and automatically refines the non-terminal symbol mark through unsupervised learning to enhance the descriptive ability of the grammar. The training part of the model mainly includes two processes of splitting and fusion. The splitting process splits each non-terminal symbol into two and refines the tokens, thereby expanding the complexity of the grammar and expanding the coverage of the language phenomena that appear in the treebank; the fusion process is to ensure that the tokens in the splitting step Which splits are necessary, this is measured by examining the impact of whether a certain marker is split or not on the likelihood of the entire treebank, that is, if the two split sub-markers are merged, the likelihood of the entire treebank decreases is not obvious, the split of this tag is unnecessary, and the sub-tags are merged.

Using this non-lexical syntactic analysis method based on automatic splitting can firstly ensure a high-performance baseline system, and at the same time, this model is easy to incorporate semantic information. In addition, adding semantic information through an external semantic dictionary is beneficial to constrain the automatic splitting of syntactic tokens; on the other hand, the subsequent automatic splitting can ensure that the added semantic classes will not affect the division of syntactic functions.

3. Syntactic analysis decoding process

For a new sentence to be analyzed, its syntactic structure can be analyzed according to the grammar model obtained during the training process. The basic method is to use the grammatical rules in the grammatical model to derive the most probable syntax tree from bottom to top in the way of line graph analysis, but the search space of this simplest analysis method is very huge. In order to improve efficiency, a coarse-to-fine analysis strategy is adopted, that is, first, a series of candidate results are obtained by decoding with a simple grammar model, and then a more refined grammar model is used to decode these candidate results, so that Many impossible results are cut before the subsequent fine decoding, thereby reducing the search space and improving efficiency.

Positive effect of the present invention:

Compared with the prior art, the present invention uses semantic information to help syntactic analysis disambiguation, effectively improves the performance of syntactic analysis, and significantly improves the efficiency and accuracy of syntactic analysis; and can use this syntactic analyzer that fuses semantic information Semantic information of some words is obtained.

Description of drawings

Fig. 1 syntax tree and syntax tree after adding semantic information;

(a) is the sentence in the tree bank before labeling; (b) is the sentence after semantic labeling;

Figure 2 Example of sememe tree fragments in the semantic resource HowNet;

Fig. 3 is a flow chart of the method of the present invention.

Detailed ways

The specific implementation manner of the present invention will be described in detail below in conjunction with the accompanying drawings, and the flow chart of the method of the present invention is shown in FIG. 3 .

1. Build a word-semantic index

According to the hyponym relationship between sememes defined in HowNet, the semantic classes of different layers from coarse to fine are extracted, and correspond to each word, so as to construct the index from word to semantic class. The words here are accompanied by part-of-speech information.

2. Add semantic class information to the original tree bank

For the original tree bank, the semantic class information is obtained by using word and part of speech as the key value, and then the semantic class information is attached to the part of speech (pre-terminal) level to realize the refinement of the part of speech layer tag. Such parts of speech contain semantic information.

Some words may have multiple different semantic classes, and two strategies are used for this situation: select the first of multiple semantics, or use manual labeling to select according to the context.

3. Training grammar model

The tree bank with added semantic class information is used as training data. The non-lexical syntactic analysis model introduced above is used for grammar training. During the training process, non-terminal symbols are refined by automatic splitting and merging. On the other hand, in order to investigate whether it is necessary to perform this refinement process on the pre-terminals with added semantic information, we conducted experiments to verify that the effect of automatic subdivision while adding coarse-grained semantic information is better than that without Segmentation, and the effect of this approach is also better than directly adding more differentiated fine-grained semantics without automatic refinement. The following effect analysis section will introduce it in detail.

4. Perform syntactic analysis on the sentence to be analyzed

With the grammatical model trained above, for a sentence to be analyzed (which has been processed by word segmentation), the non-lexical parser introduced above can be used to decode according to the grammatical model, and the result of syntactic analysis is obtained, and the sentence is also included semantic annotation results.

Effectiveness analysis:

In order to verify the effectiveness of the present invention, we designed a series of experiments, some of which are described below.

Experimental corpus:

The training and testing corpus uses UPenn Chinese Tree Bank 2.0, which contains a total of 325 news corpora, which are divided in a standard way: use 1-25 as the development set, a total of 350 sentences; 26-270 as training A total of 3172 sentences; 271-300 as a test set, a total of 348 sentences.

Semantic dictionary using HowNet.

Baseline system:

The baseline system adopts the non-lexical syntactic analysis model introduced above, and adopts an unsupervised method to automatically split and refine non-terminal tokens. Each iteration splits the original token into two, and determines the parameters corresponding to the new token through the EM algorithm, and then The split tokens are merged according to the likelihood contribution.

Evaluation procedure:

The evaluation program adopts EVALB, a currently widely used syntax analysis evaluation tool. The tool uses bracket tag matching as the evaluation criterion, focusing on precision, recall and F-value.

Experimental results and analysis:

The results of the baseline system tested on the CTB standard dataset are shown in Table 1:

Table 1: Baseline System Performance

Among them, S&M represents the number of split-merge process loops. For example, S&M-1 represents one split-iteration; S&M-2 represents two split-iterations, that is, a split-iteration is performed on the basis of the grammar obtained by one split-iteration . Len indicates the length of the sentence, that is, the number of words contained in the sentence. Len<=40 means that the test is only performed on sentences whose length is less than 40; All means that the test is performed on all sentences. LR means the recall rate, LP means the precision rate, and F1 means the F value.

In order to alleviate the problem of data sparsity to a certain extent, we select the topmost semantic class in HowNet and automatically refine all tags. The experimental results using the same data set are shown in Table 2.

Table 2 Adding coarse-grained semantic class tag analysis performance

From the table above, it can be found that starting from the fourth iteration of split-merging, the parsing performance by adding semantic information classes exceeds the baseline system. In the sixth iteration, over-training occurred due to too fine splitting, and the F value decreased to a certain extent, which showed the same trend in the baseline system and the improved system. But the results of adding semantic classes are still better than the baseline system. Comparing with the results of the fifth iteration, the F value has increased from 80.26% to 81.63%, an absolute increase of 1.37 points, which is quite significant in the study of syntactic analysis.

In addition, the newly released 5.0 version of the Penn Chinese tree bank (comprising 18782 sentences in total) is used for training, and the syntactic analysis performance of the present invention can reach the highest F value of 86.39%. The comparison trend before and after adding semantic information is similar to the results obtained on the Penn Chinese Treebank 2.0 listed above, so I won’t go into details here.

The present invention is based on a non-lexical syntactic analyzer, integrates semantic information into it, uses semantic information to help syntactic analysis to disambiguate, significantly improves the performance of the syntactic analyzer, and can obtain Semantic information of some words.

Claims (8)

1. A Chinese syntax analysis method in combination with semantic information, the steps are: 1) Extract the semantic categories of different levels of words according to the hypernymy relationship of HowNet, and obtain the index from words to semantic categories; 2) Use the word in the syntax tree as the key value to query HowNet to obtain the semantic class of the word, and add the semantic class to a certain layer of the syntax tree; 3) using the syntax tree processed in step 2) as training data, and performing grammar training to obtain a grammar model; 4) Utilize the grammatical model trained in step 3) to decode the sentence to be analyzed. 2. The method according to claim 1, characterized in that said certain layer is a pre-terminator layer. 3. The method according to claim 2, characterized in that said words contain part-of-speech information. 4. The method according to claim 3, wherein the word and part of speech are used as key values to query HowNet to obtain the semantic category of the word. 5. The method according to claim 1 or 4, characterized in that the query is performed on the semantic classes of the same layer in HowNet, so that the semantic classes obtained by all word queries are in the same layer in HowNet. 6. The method according to claim 1, characterized in that the grammar training is carried out using a non-lexicalized syntactic analysis model. 7. The method according to claim 6, characterized in that the grammar training method is as follows: the pre-terminals are subdivided by means of automatic splitting and merging. 8. The method according to claim 1, wherein if there are a plurality of different semantic classes in the word, then select the first semantic class in the plurality of semantic classes as the semantic class of the word, or adopt manual labeling according to Context selection.

Images