JPH02190972A - Example retrieving system - Google Patents

Abstract

PURPOSE:To improve the efficiency of extraction and analysis of various data for natural language research by retrieving a data storage means by a defined keyword obtained by adding information for grammatically restricting an object to be retrieved to a keyword. CONSTITUTION:A keyword input part 10 input one or more keywords, specifies grammatical information for defining the category of words constituting a sentence example executed in each keyword and forms a defined keyword by editing the specified grammatical information. The defined keyword is converted into a partial item to be matched, the converted partial item is matched with a sentence information list of a text data base 13 and the text data base 13 is retrieved to obtain the example of the corresponding sentence. A retrieved result output part 12 displays the example obtained by retrieval by a KWIC (context filing word) format in accordance with an output mode specified by a user. Thus, the example can be flexibly retrieved by using a structured retrieval keyword.

Description

DETAILED DESCRIPTION OF THE INVENTION (a) Field of Industrial Application This invention relates to an example retrieval system useful for language research or natural language processing research.

(b) Conventional technology Conventionally, in order to refer to language examples and analyze linguistic phenomena, example retrieval systems have used KWIC (keyford in C), which is called "headword with context".
ontext) is often used. This KWIC is a language database that provides examples of word usage.
It is said to be a useful language analysis tool for language research or natural language processing research. The structure of KWIC is a table in which headwords are sorted by the context in which they are used, and many KWICs have been developed. Most of the usual KWICs are classified using one word as a keyword. On the other hand, in addition to the KWIC sorted by word, there are KWICs with a slightly different format, developed by the Electronic Technology Research Institute, called the New Collection of Japanese Parts of Speech, Right Order Edition (1979) and Left Order Edition (1979).
) is the “Japanese part-of-speech collection”. This collection of Japanese part-of-speech strings is a collection of word strings with part-of-speech sorted by part-of-speech, and a part-of-speech string of a predetermined length, an arrangement of words corresponding to the part-of-speech string, and context.

(c) Problems to be solved by the invention However, as the name suggests, conventional KWICs have a mechanism for searching using the notation of a single word as a keyword, or are often printed in deaf form. It was not possible to obtain useful information that met the above objectives. The linguistic phenomena that natural language processing IE researchers and linguists really want to investigate are not examples of individual words, but examples that include the co-occurrence of words with certain constraints or complex patterns. In the standardized KWrC, it is impossible to search for examples using such complex patterns. Therefore, in conventional example search systems, it is necessary to refer to many examples of a word or to understand the actual context of the word to be searched. It was not possible to conduct a wide survey to see if it was used in Under these circumstances, a flexible example retrieval system has been desired. The information that linguists want to search for when conducting language research is (1) whether a certain word is used; and (2)
(3) Which page, in which line, and in what context a certain word is used in which work? (4) Which work does a certain word appear in? On which page, in which line, in what context, and in what part of speech information sequence? (5) Give a word with a certain meaning with a context. (6) Give a conjunctive modification with a context.
7) Give the subject with context (8) Give an example of the "noun of ~" construction (9) Give the word that refers to a certain word (10)
Items include producing sentences that use personal pronouns in conversations written by upper-class women in the early Meiji period.

The present invention has been made in consideration of the above circumstances, and provides an example search system that allows example searches to be performed flexibly using structured search keywords.

(d) Means and effects for solving the problem In this invention, the term "example" means an example of how to use it, and includes how it is used in a sentence or in a phrase.

These sentences and phrases can be applied to Japanese or foreign languages such as English and German.

The sentence numbers and the like attached to the text database are, for example, information for classifying sentences into categories that can be included in addition to the sentence numbers and sentence numbers.

The defined keyword refers to a structured keyword explained in the embodiment, and is an ordered list whose elements are word information, which is a subset of grammatical features for defining the word cloud expression (single!M).

The KWIC format refers to a format in which automatic searches include not only headwords but also the context in which the headwords are used.

This invention stores a large number of sentence examples in notation with sources, sentence numbers, and sentence information lists attached, and stores the sentence information list in terms of part-of-speech classification, notation, part-of-speech M class, etc. of the words that make up the sentence examples. A text database represented as a collection of grammatical information consisting of information such as conjugation types and meanings, input of at least 1 or 2 keywords, and definition of the range of words constituting examples of sentence usage for each keyword. A keyword input section that specifies grammatical information and creates defined keywords by editing the specified grammatical information, and a keyword input section that converts defined keywords entered manually from the keyword input section into matching subterms. , consists of a search section that searches for examples of sentences corresponding to defined keywords by matching with the sentence information list in the text database, and a search result output section that outputs examples of the searched sentences in KWIC format. This is an example retrieval system characterized by:

Further, in the second invention, the keyword input unit creates a keyword with a definition by specifying at least one part of speech, or by specifying one or more parts of speech and specifying grammatical information specific to each specified part of speech. In the third invention, the keyword input unit creates a keyword with a definition by specifying the part of speech and, when the specified part of speech is a predicate with conjugation, specifying information regarding the conjugation of the predicate. In the invention, the text database has semantic information consisting of a thesaurus that connects noun concepts in a superior/subordinate relationship to form a system, and the keyword input section creates keywords with definitions by specifying at least 61 or more noun concepts. Then, the search unit searches the text database for examples connected by concepts based on defined keywords. Further, in the fifth invention, the keyword input unit creates a keyword with a definition by specifying the auxiliary term 11 and the type of the specified auxiliary term; A keyword with a definition is created by specifying the specification and the semantic information of the specified particle.

Further, the example search system of the present invention has the following features.

(1)) Examples of Japanese sentences can be searched using a pattern consisting of one word as a search keyword. Furthermore, various grammatical constraints can be imposed on each word of the search keyword. A search keyword string with such characteristics is called a structured keyword. As grammatical constraints, it is possible to provide not only morphological information such as "parts of speech, subdivisions of parts of speech, conjugation information, headings, and occurrences," but also thesaurus concept names that express hierarchical relationships between concepts as semantic constraints. be.

(2) The searched examples are sorted in the order specified by the user and displayed in KWIC format. Additionally, more detailed grammatical information can be provided for specific usage examples within the search results.

(3) It has a structured keyword input function that is convenient for users.

(4) A wild card that can be matched with any word string can be used as part of the structured keyword.

(e) Examples The present invention will be described in detail below based on examples shown in the drawings. Note that this invention is not limited by this.

FIG. 1 is a block diagram of the example retrieval system of the present invention.

In the figure, the example search system mainly includes a keyword input section 10 as an input means, a search section 11, a search result output section 12, and a text database I3 as a data storage means. Keyword input section 10
Is a structured keyword word count or each unit for searching! Manually check my grammar background etc. The search unit 11 searches the given structured keywords as CI L (Complex Index).
terminateLanguage: A functional language suitable for problems in the field of artificial intelligence.
og" (extended language) and searches the text database 13. The search result output section +2 displays the examples obtained by the search in Kwtc Sugi style according to the output mode specified by the user. The text database 13 is a database that stores text (with additional information) to be searched by the search section 11. Note that I4 indicates a file section, and 15 indicates a printer output section.

The details of each part described above will be explained below.

Text Database (a) Words Expressed as a Feature Set The text database 13 is a database in which texts to be searched by the search unit II are stored. In order to realize the features of this invention described above,
The database contains not only information about the original text, but also
It is necessary to add "parts of speech, subclassification of parts of speech, conjugation information, semantic information," etc. In addition, “word breaks, sources,
Information such as "sentence number" is also required.In order to easily describe such information, this system treats each word in text data as a set of "feature name/feature value" pairs, and stores it as a set of various grammatical information. Expresses one word. This is based on the framework of unified grammar theory, which considers grammatical ranges as feature sets. The information of each sentence is expressed as a list of sets. The grammatical features that make up each word differ depending on the part of speech of that word.

(b) Part-of-speech classification table I shows the part-of-speech classifications adopted in this system and the classification features of each part of speech.

(Leaving space below) Table 1 Classification and features of parts of speech As shown in Table 1, 14 types of parts of speech have been introduced. This classification of parts of speech is based on general school grammar. However, the last “auxiliary side” is generally an auxiliary movement or one
, is a general term for what are called auxiliary adjectives. Currently, eight types of grammatical features are assigned to each of these parts of speech. However, each part of speech does not have feature values for all eight 4 degrees, but only features marked with "O" in Table 1. The role and value of each grammatical feature are as follows.

Part of speech: I4 [! One of the part-of-speech names of l.

Form 2: The form in which the word is actually used in the sentence (occurring form).

Heading: Final form of conjugated word. If there is more than one notation, use the representative form.

Often the same as the notation.

Type: Has different values depending on the part of speech. In the case of nouns, they can be either "proper nouns, slang nouns, numerals, proper nouns, or formal nouns." In the case of prefixes and suffixes, either "general" or "numerical" depending on the word they connect. In the case of particles, either "case particle, final particle, adverbial particle, parallel particle! 111, conjunctive particle, conjunction particle, or corpuscular particle".

Conjugation type: One of "godan, -dan, kihen, suhen, go, aru, and do".

Add only to conjugated verbs with conjugation lines 2 and 5.

Conjugated form: “foreign, conjunctive, terminating, adjunct, presupposition, command’.

Meaning: A list whose elements are concept names in the thesaurus for this system shown in Figure 2. It can take multiple values, so make it a list.

(e) Implementation of sentence information Each sentence in the text database has the following format and is expressed as one unit clause in CIL.

5entence (source, sentence number, sentence information list)
The "sentence information list" is a list whose elements are each word expressed as a feature set (subterms of CIL). The reason why feature sets and combinations are expressed as CIL subterms is that subterms are easy to express data structures such as frames, and CI
This is because extended unification of L can be used.

By using this format, for example, the 13th sentence in the source ``protecting nature'', ``Let's take the environment of the forest as an example,'' can be expressed as shown in Figure 3.

Keyword Input Unit The keyword input unit IO is for inputting structured keywords for searching, and is usually configured with a keyboard or mouse, and inputs the number of words of the structured keyword and the grammatical features of each word.

(a) Structured keywords for structured keyword search are defined as follows.

Definition 1: A structured keyword is an ordered list whose elements are disjunction information.

Definition 2: Word cloud information is a subset of grammatical features for defining words. However, grammatical features are expressed as a "feature name/T, gender value" pair.

By introducing such structured keywords, conventional KW that used only expressions or parts of speech as keywords can be improved.
Flexible searches that were not possible with fG become possible. Below are some examples of structured keywords.

Example 1: [(article iA/noun, type/common noun, meaning/abstract concept, (part of speech/particle, heading/wo)] Example 2: [(article/adverb), *, (article rA/verb, conjugation Example 1 is a structured keyword to search for examples in which the particle "wo" is used next to a common noun that has the semantic feature of abstract concept. Example 2 is This is an example of using the wildcard "*", which is a structured keyword to search for examples in which the conjunctive form of a five-stage conjugated verb appears after a certain adverb. "*" represents multiple word strings, and The part can mean anything. Therefore, for example, if you say “Tokyo*”, it means “Tokyo”, such as Tokyo, Tokyo Head Office, Chiyoda Ward, Tokyo, etc.
All characters starting with .

(b) Structured keyword input procedure + 1tlf + structured keywords in the following order, i.e. (1)
They are input in the order of source (2) part of speech (3) notation/heading 4) type (5) conjugation form/conjugation line/conjugation form (6) meaning.

For feature values, "notation/heading" is directly input, but other features are entered using a menu selection method in a dedicated window display. The features (4) to (6) prevent unnecessary input requests from being made depending on the part of speech values. FIG. 4 shows an example of a screen during inputting such a structured keyword, and 4a to 4e each show a window display.

Search unit The search unit 11 is composed of a CPLI (central processing unit),
The manually generated structured keywords are converted into subterms that can be pattern matched, and the text database I3 is searched. And save the search results.

Search Result Output Unit The search result output unit 12 displays the searched examples in KWfC format according to the mode specified by the user. The user can then specify the following modes.

(1) Grammar features to display (default is notation only) (2
) Whether or not to sort (ascending order, descending order, appearance order) (3) Sorting priority (which feature of which keyword) (4) Output light of search results (CFt'r, file, editor buffer) (5 ) Detailed grammar feature display specification (detailed information display of specific search sentence) FIG. 5 is an example of the result of executing a search using the structured keyword explained in the structured keyword manual procedure.

It is possible to scroll this screen in the vertical and horizontal directions.

In addition, the text database in this example! 3 currently adds morphological information through manual operations, but as with ICO'V's DLIALS, which has already been produced as a prototype, various types of information are automatically added to sentences with only notation through morphological analysis, syntactic analysis, semantic analysis, etc. A text database constructed by a system that adds linguistic information may also be used. Therefore, as a matter of course, the example sentence database generated by this type of system is also included in the text database I3 in this embodiment.

(The following is a blank space.) Next, the operation of this embodiment will be explained according to the flow charts shown in FIGS. 6 to 12.

FIG. 6 is a diagram showing the overall operation in example search.

In the same figure, when the number of words N of a structured keyword is input from the keyword input section IO (step 20), it is determined whether or not the end of processing has been specified (step 21). If the end of processing has not been specified, from the keyword input section IO Receive human power for structured keywords (step 22),
Next, the search unit Il searches the text database 13 for information corresponding to the structured keyword K (step 23). When the search is completed, the search results are transferred to the search result output unit 12, and the search result output unit 12 displays the transferred search results on the screen according to the set output mode 1 (step 24). In addition, step 2
When the end of processing is specified in step 1, end processing such as canceling the searched information and closing the window is performed (step 25), and the example search processing is ended by 1°.

Next, detailed flows of each of the above-mentioned processes will be explained.

Structured Keyword Manual Processing FIGS. 7 and 8 show detailed processing of the "structured keyword manual processing" shown in step 22. In Figure 7,
First, it is determined whether or not the number N of words entered in step 20 has been entered manually (step 3α); if no, the words are continued and manually entered step 3I); then it is determined whether the number N of words has been entered 4- (Step 32).

If No here, step 33) returns to step 31 to manually determine whether or not a wild card is specified. However, the manpower required to specify a wild card is not counted in the number of words N. If Yes in step 3o and step 32, it is determined whether to enter a new keyword instead of other keyword candidates (step 34); if Yes, the process returns to step 30; if No, the search process (step 34) is performed; 23).

FIG. 8 shows details of the "word manual" processing shown in step 31. First, enter the part of speech (step 40),
Next, input the notation and heading of the word (step 41).
), then the grammatical features, that is, the type, conjugation type, conjugation line, conjugation form, and meaning of the input part of speech, are manually determined (step 42).
). However, the feature value differs depending on the type of part of speech. These inputs are performed manually according to manual instruction guidance displayed on the screen, and once the part of speech and grammatical features are input, the manually entered information is edited and the words are displayed on the screen (step 43). Next, it is determined whether to input another word candidate (step 44), and if YES, the process returns to step 40, and if NO, the process moves to step 32.

Search Process FIG. 9 shows details of the "search" process shown in step 23. First, the search takes in one sentence '■'' to be searched (step 50), and when the target sentences are found (step 51), the number of words M included in that one sentence is extracted (step 52), and then The word pointer '1' i associated with the sentence 'r' is set to 1 (step 53), and the word pointer Kj associated with the keyword K is set (step 54). Next, the matching information area is cleared (Step 55), and it is determined whether the value of the word pointer related to the keyword is less than or equal to the number of words N of the structured keyword (Step 56). If Yes, the value of the word pointer related to the sentence is then cleared. It is determined whether the value is less than or equal to the number of words M included in the sentence (step 57), and if Yes, the element pointed to by the word pointer in sentence T and the word pointer in keyword K is determined here. If the elements match (step 58), the value i of the word pointer regarding the sentence and the value j of the word pointer regarding the keyword are stored (save) (step 5
9). Next, in order to continue the search, I is added to each of the values of i and j (step 60), and the process returns to step 56. If the elements do not match in step 58, it is determined whether a wild card is set between the keywords (step 61), and if Yes, only the value i of the word pointer in the sentence T is calculated by IJII (step 61).
2) Return to step 56. This is because wildcards are not counted as keywords. Then, in step 56, when the value jh of the word pointer for the keyword K<the number of words N of the structured keyword is exceeded, the matched information is stored in the storage area of the search result output unit I2 (step 63), and the information of the search target is In order to search for any other element matching the keyword in the sentence, the value j of the word pointer for the keyword K is set to 1 and the processes of steps 55 to 62 described above are repeated.

Note that the mapping information is the word pointer i in the sentence.
, the value j of the word pointer in the keyword, the source, the sentence number, etc.

Search result output FIGS. 1O to 12 show details of the "search result output process" shown in step 24.

In FIG. 10, "search result output" mainly includes output mode setting (step 70), window display (step 71), command input (step 72), determination of whether there is an instruction to end output processing (step 73), and command Processing is performed in the order of execution (step 74). In Figure 1t,
The "output mode setting" process shown in step 70 will be explained. First, the grammatical features to be output are specified, and the range of information (notation, part-of-speech classification, etc.) required is specified (step 80). Next, the key to be centered (the keys ``rata'' and ``rau'' arranged in a row as shown in FIG. 5) is designated (step 81). Next, it is determined whether or not to sort (step 82), and if yes, the sorting direction is designated as ascending or descending (step 83). Next, the order of the keys to be sorted, that is, the number of keys to be set as the center of sorting is specified (step 84), and the grammatical features of the keys to be sorted are specified (step 85).

This grammatical feature is specified, for example, when it is desired to display key output by part of speech classification rather than notation. The output mode is set by the process described above.

Next, the "window display" process shown in step 71 will be explained with reference to FIG. 12. First, it is determined whether or not the output mode is set to sort.
If so, perform sorting (step 91) and generate display data according to the set output mode (step 92).
), the generated results are displayed on the screen (step 93). Window display is executed by the above processing.

Next, in the example search method that performs such processing, the input form of the structured keyword and an example of the extraction of 返cloud as output will be compared and shown below.

In order to simplify the explanation, we will use the text database 13.
Assume that the only memorized word moxibustion is the sentence ``Let's take the forest environment as an example.'' Also, an example of extraction is
Suppose that we output the notation of the sentence.

(1) Extraction based on morphological information When the keyword is [(part of speech/verb), (part of speech/particle)], the extraction example is "Let's take the forest environment as an example." According to this search method, it is possible to set a definition for each keyword, and if the part of speech is a verb, it is also possible to define the conjugation type, conjugation line, and conjugation form, which are verb-specific information, and if the part of speech is a noun, it is possible to define the conjugation type, conjugation line, and conjugation form. Of course, the information is no longer needed,
Instead, superordinate concept information (thesaurus), which is noun-specific information, can be set.

(2) Extraction based on conjugation information of predicates When the keyword is [(part of speech/verb, conjugation type/5 stages)], the extraction examples are "Let's take the environment called a forest as an example" or "Let's take the environment called a forest as an example". There are verbs, adjectives,
Independent words with conjugations, such as adjective verbs, are called pragmatics, but here, auxiliary verbs with conjugations are also treated as pragmatics. There are generally six conjugations of predicates: conjugation, conjunctive, terminating, adjunctive, presupposition, and imperative. A search is performed by defining this as one element of a structured keyword. Also, according to this search method, structured keywords can be written as
If you set ``Administrative form'', the continuous form ``Move'' and the final form ``Move'' will be set.
``move'' can be distinguished, and only examples related to the adjunctive form of ``move'' can be searched.

(3) Extraction based on semantic information When the keyword is [(meaning/abstract concept)], the extraction example is "Let's take the forest environment as an example."

According to this search method, if the meaning of a structured keyword is set to animal and [(animal) runs], then (animal)
It is possible to search all the sub-concepts of . Also,[
If you set ``runs'', many examples will be found, but ``(
“The train runs” by setting “Inanimate object) runs”
You can perform a search that excludes (organisms), such as.

(4) Extracted keywords based on particle type tit information are [[article rA/noun], (part of speech/particle, ll
l1 case particle)], the extraction examples are "Let's take the environment called a forest as an example", "Let's take the environment called a forest as an example"
It is. According to this search method, it is possible to distinguish between ゛ and '', which are used as case particles, and `` and '', which are used as parallel particles, in a search.

(5) Extraction based on semantic information of particles However, in this case, it is assumed that the following two examples are the words stored in the database I3.

l! A11 [Wind (), de (part of speech / particle, heading / 1 notation / de, fX class / case particle, meaning / [cause ko), company ()
wo (), rest ()] Example 2 [Paper (), wo (), scissors (), de (part of speech/particle, heading/one notation/one type/case particle, meaning/[tool])
, cut()]. Note that information in parentheses is omitted.

The keywords are [(part of speech/noun), (part of speech/particle, meaning/
tool)], the extraction example is "cutting paper with scissors."

There are several meanings for each particle, but examples of the classification of meanings for the case particle ``are'': studying at school (place), carrying luggage by car (means, method), nailing with a hammer. hitting (tool), resting in the wind (cause), making bread with flour (ingredients), completing it in three days (@en, degree), speaking in a whisper (sama t)
s), hired by the company (subject), etc. Therefore, according to this search method, particles with the same notation (and particles of the same type) can be searched by distinguishing them according to the role they are used in (referred to as meaning in this embodiment).

Note that the keywords shown here are just examples, and there are various variations.

(f) Effects of the Invention According to the present invention, data storage means is searched using defined keywords, which are added with information that grammatically restricts the search target. Extraction and analysis can be performed efficiently.

[Brief explanation of the drawing]

Fig. m1 is a configuration diagram of the example retrieval system according to the present invention, Fig. 2 is an explanatory diagram of a thesaurus used in the example retrieval process of the embodiment, and Fig. 3 shows an example of sentence information stored in the text database of the embodiment. 4 is an explanatory diagram showing the structured keyword input state of the embodiment, FIG. 5 is an explanatory diagram showing the search results of the embodiment, and FIGS. 6 to 12 are flowcharts showing the operation of this embodiment. It is. IO: Keyword input section, It: Search section, 12: Search result output section, 13: Text database. Figure 5 Figure 8

Claims (1)

[Claims] 1. A large number of usage examples are stored, classified into grammatical information such as part-of-speech classification, notation, part-of-speech type, conjugation type, meaning, etc. for each word constituting the usage example, and A text database with sources, sentence numbers, etc. added as desired, at least one piece of grammatical information in a predetermined order, and when grammatical information is specified, a definition can be created by editing the grammatical information. The keyword input section that creates keywords and the defined keyword input from the keyword input section are converted into matching subterms, and by matching the converted subterms with the sentence information in the text database, the defined keywords are created. An example search system comprising: a search section that searches for examples corresponding to a keyword; and a search result output section that outputs the searched examples in KWIC format. 2. In the usage example retrieval system according to claim 1, the keyword input section specifies at least one part of speech, or by specifying one or more parts of speech and specifying information such as conjugation form specific to each specified part of speech. 2. The example search system according to claim 1, wherein keywords with definitions are created. 3. In the example search system according to claim 2, the keyword input section specifies the part of speech and, when the specified part of speech is a predicate with conjugation, specifies information regarding the conjugation of the predicate, thereby generating the keyword with a definition. 3. The example retrieval system according to claim 2, wherein the example retrieval system creates an example retrieval system. 4. In the example retrieval system according to claim 2, the text database has semantic information consisting of a thesaurus that forms a system that connects concepts of nouns in a superior/subordinate relationship, and the keyword input section includes at least one or more nouns. 3. The example search system according to claim 2, wherein a defined keyword is created by specifying a concept, and the search unit searches for examples connected by the concept based on the defined keyword. 5. The example search system according to claim 2, wherein the keyword input section creates a keyword with a definition by specifying a particle and a type of the specified particle. Example search system. 6. The example search system according to claim 2, wherein the keyword input section creates a defined keyword by specifying a particle and semantic information of the specified particle. example search system.