JPH0486948A - Method for preparing kana-added data base utilizing dictionary by fields - Google Patents

Abstract

PURPOSE:To prepare the data base of biographies and technical terms, etc., by automatically executing separate descriptions and the addition of Japanese syllabary (KANA) by using a natural language processing, which is one kind of AI, for preparing the KANA-added data base. CONSTITUTION:First of all, a preprocessing is executed to a stored data base. First, data are extracted and code conversion is executed. The serial number (ID) of a dictionary 106 by fields is set to the code-converted data, afterwards, a natural language processing input file is prepared, and the above-mentioned operation is repeated to all the data. The dictionary 106 by fields is a table paring Chinese characters (KANJI) and the readings. For each field to be processed, the field is registered on a computer in advance and managed by the ID starting from '1'. When using the dictionary 106, the ID of the dictionary by fields is set to the data attribute of a natural language processing input data record, and for preparing the natural language processing input file, the natural language processing input file record is prepared for each extracted data.

Description

[Detailed Description of the Invention] Purpose of the Invention: (Industrial Application Field) This invention provides a CD-RO that uses a natural language processing system to create a kana-furi database using field-specific dictionaries.
This relates to a method for creating a database such as M.

(Prior art) Recently, document data accumulated for printed matter has been used as a secondary
D-ROMs and databases are increasingly being created. The work of extracting keywords for database searches and writing kana characters has traditionally been done manually by experts. Especially people's names.

In special fields such as addresses and medicine, it was extremely difficult to read, and it was almost impossible for non-specialists to do the work.

(Problem to be Solved by the Invention) Conventionally, the task of extracting keywords for database searches has been to have experts select important words from documents and then add readings to them. For this reason, the task of extracting keywords from the database requires a great deal of effort, and the task itself is inefficient. It has been extremely difficult to create keywords, especially in special fields such as people's names.

This invention was made due to the above-mentioned circumstances,
An object of the present invention is to provide a method for automatically creating a kana-furi database using field-specific dictionaries while utilizing natural language processing technology in the field of AI (artificial intelligence).

Structure of the Invention: (Means for Solving the Problems) The present invention relates to a method for creating a kana-furi database using a field-specific dictionary. This is achieved by referring to a dictionary, creating a natural language processing output file using natural language processing, and creating a kana-furi database through post-processing.

(Operation) In this invention, natural language processing, which is a type of AI, is used to create a kana-furi database.
It automatically performs part-of-speech (part-of-speech decomposition) and kana translation.

Nouns, using the computer's built-in dictionary and AI techniques.
It decomposes the document into elements such as particles and verbs, adds readings to the kanji in the divided documents, and extracts keywords. Machines now handle work that was traditionally done by hand, so all that is left is the same checks as before.The created kana-furi database can be processed and used as an index for CD-ROMs and online databases, and can also be used as an index for CD-ROMs and online databases. You can also format it as a full ruby tree using the scroll function.

(Example) First, a natural language processing system used in the present invention will be explained.

FIG. 7 shows an example of the hardware configuration of a natural language processing system, in which a host machine 10 has a built-in CPU II and a built-in memory 12, and a magnetic disk device 14. Cassette magnetic tape device 15
is connected. The host machine IO further includes a magnetic tape device 20. A laser printer 21 and a console terminal 23 are connected, and a confirmation/correction terminal 22 is also connected via an interface 16 of R5-=232C.

Figure 8 shows the software configuration of the natural language processing system, in which the input data from the magnetic tape is input to the input processing 10.
The information that has been taken in as 1 and processed by the host machine 1O is output processed 120 and becomes output data on the magnetic tape. That is, the human processing 101 copies the natural language processing system human data magnetic tape onto a disk file as the human data 102, checks the kanji code, etc., and then converts it into a record for Japanese processing. Also,
The output processing 120 outputs the processing result file on the disk to a natural language processing output magnetic tape as processing result data 121 . The driver 103 classifies/analyzes the human data 102, controls the Japanese language processing system 110, obtains the results of separation, kana writing, and keyword extraction, and edits/presses the processing results in the natural language processing system output data format. .

The Japanese language processing system 110 performs morphological analysis via the basic dictionary access routine 112, extracts the pronunciations of all words recognized by language processing, and outputs them as kana-jiri output sentences. Noun string extraction is the result of word recognition through language processing, and when the part of speech corresponds to the following (a) or (b), it is extracted as a noun.

(a) Common nouns, deformed nouns, verbal nouns, transposed nouns 1
Regarding temporal words, number words, proper nouns, one pronoun, and formal noun (b) affixes, when the preceding and following parts of speech correspond to the following, the corresponding word is extracted as a noun.

■For prefixes, backward part of speech: common noun, s-inflected noun, morphological noun, transposed noun, 1 hour, numeral, proper noun, 1 pronoun, formal noun. ■For suffix, forward part of speech/common noun, s-inflected noun, morphological noun. Type nouns, transposed nouns 1 hour, numerals, proper nouns 1 pronoun, formal nouns.In addition to inputting the Japanese sentences and the keyword analysis table obtained from the above, statistical analysis was performed.

The input Japanese text is analyzed in cooperation with the access file routine 111 using methods such as syntax analysis and knowledge processing, and keyword extraction and narrowing 1 importance evaluation are performed.

The terminal communication processing 123 communicates with the confirmation/correction terminal 22 and converts data for terminal output. Then, the modified data from the terminal is converted into an output file format and written. Further, the list pressure processing 122 edits the processing result data requested to be output from the terminal into printer output data, and outputs the printer output data to the laser printer 21 .

By the way, the natural language processing functions that the host machine IO can handle are: A. IA Rise 1/Wakigaki BJIL Rise 2 Kana Furi I (Kana Furi for each dividing line) c. (Kana translation, total ruby translation) O3 processing type 4: There are four types: keyword extraction and kana translation to the keyword, and each of the above functions can be switched and processed for each record of the manual file.

Next, each function (processing types 1 to 4) will be explained.

A1 Separation (Processing Type 1) A Japanese sentence (a sentence containing Kanji and Kana) is manually separated and part-of-speech information is added to each noun, verb, and adjective. The output information includes separation using sura nshu “/” and part-of-speech information (nouns.

(verbs, adjectives, unknown words). The output format of processing type 1 is as shown in FIG.

B. Kana-Furi I (Processing type 2: Kana-Furi for each separated word): A Japanese sentence (including Kanji and Kana) is manually separated, and the separated words are then transformed into Kana-Furi. Readings are written in katakana, and part-of-speech information is added to nouns, verbs, and adjectives. The output information is separated by slashes and contains nine parts of speech information (nouns, verbs, adjectives).

(Unknown word) This is the result of kana translation to the 9-minute written word element. The output format of the process f12 is as shown in FIG. 1O.

C, Kana Furi II (processing type 3): This process f! ! 3 has a function of kana-furi and total ruby-furi (kana-furi for each kanji (column)) using the field-specific dictionary 106. Kana-furi using the field-specific dictionary 106 can be used for item data such as people's names, place names, and various technical terms.
Kana-furi is performed using a dictionary dedicated to the item. The kana-furi method searches for field-specific dictionaries i 1 Q 5 using the item data as KEY, and when a match is found, moves the kana registered in the field-specific dictionary 106 . If kana is not obtained, the Japanese language processing system is called and the basic dictionary 115 is used to determine the kana.

The manual format of data is "item data" for single item data, and "item data 1"/"item data 2"/... when processing multiple item data with a code.
.../"Item data N", each item data is separated by a slash. The input information is the reading (katakana) for the input item data and the authority dictionary identification of the kana data (identification of which dictionary the kana was assigned based on).The output format of processing type 3 is i
As shown in Figure 11, ■ If the reading is acquired in the field-specific dictionary 106, ■ If the reading is acquired in the basic dictionary Its, ■ The reading is registered in both the field-specific dictionary 106 and the basic dictionary 115. If not, it is divided into identification codes (for example, AA, AB, A (:)).

The data to be processed in kana-furi using the field-specific dictionary 106 includes item data such as person names, place names, and various technical terms (
The data to be processed with full ruby processing is Japanese kanji and kana mixed sentences. Total ruby swing (
The function (Kana-Furi for each Kanji (column)) is to input the meaning of a Japanese sentence (a combination of Kanji and Kana) and perform Kana-Furi for all kanji. In the kana-furi method, kana (ruby) is cast for kanji (rows) (other than JIS non-kanji) in the human original text, and ruby is cast in the form of ``group ruby''.

The output format is as shown in Figure 12.

D. Extracting keywords and adding kana to keywords (4): Free keywords are extracted from the human-generated Japanese meaning using the language processing function of the Japanese language processing system, and pronunciations are added to the extracted keywords.

The output information is the extracted keyword structure (Katakana) and the keyword analysis results, and the output format is as shown in FIG. Note that the analysis information is an area where analysis information obtained in the process of keyword recognition by the Japanese language processing system is set.

The function of the confirmation/correction terminal 22 is to receive the human input original text data and processing result data 121 in the processing result file from the host machine 10 via the terminal communication processing 123, display it on the display of the terminal device, and send it to the post machine. The purpose is to facilitate checking and correction of processing results by outputting to 10 laser printers 21. Terminal 2
By using the keyboard from step 2, specify the job name of the processing result file to be confirmed/corrected, display the human original data and processing result data 121 for each code on the display of the terminal device, and perform the confirmation/correction work. Let's do it.

The display format varies from (A) to the following depending on the processing type.
It looks like (D).

(A) In the case of processing type 1 (separation), the input original text and the processed human original text are compared on the screen.

(B) In the case of IA Rise 2 (Kana translation in units of dividing lines), the human original text and the result of Kana translation in units of separating lines of the input original text that has been processed are output on the screen.

(C) In the case of processing type 3 (total ruby writing), the kana writing results for all kanji in the input original text are output on the screen with different display colors.

(D) In the case of processing type 4 (keyword extraction), the input original text, the keyword extracted from the input original text, and the kana translation result are output on the screen.

Next, the processing result data is corrected by keyboard operations, and the basic correction functions will be listed and explained below.

Correction is possible only in the case of processing type 3 and processing type 4. In the case of processing type 3 (total ruby swing), it is possible to modify the kana swing results, and in the case of processing type 4 (keyword extraction), it is possible to modify the kana swing results, insert or delete keywords, and change the ranking. .

If the terminal 22 modifies the processing result data 121,
The modified data can be transferred to the host machine by keystroke operation.
Send to O. The host machine 1o updates the record of the processing result file based on the corrected data.

On the other hand, a process result file or a process result record is output by pressing the P key (print key) by an operator 6 who outputs the specified process result file or record to the laser printer 21 of the host machine IO by keystroke operation from the terminal 22. When a unit printout request is made, the record taken out from the host machine IO is outputted to the printer in accordance with the format of each processing pole.

The above is an overview of the natural language processing system, and the present invention uses the above natural language processing system to automatically create a base for kana-speech data such as a person's name. In this embodiment, a case will be described in which a kana-furi database of a person's name is created using the field dictionary 105 as a person's name.

Figure %x shows the processing flow of the present invention, in which preprocessing is first performed on a database stored in a magnetic storage medium, etc. (step 510), and the details of the preprocessing are shown in Figure 2. First, data is extracted (step 5ll), and code conversion of the extracted data is performed (step 512). Then, the ID of the field-specific dictionary 106 is set for the code-converted data (step 513), and then a natural language processing input file is created (step 514), and the above 28 operations are repeated for all the data.

The data is extracted from the database by using this process to extract the name and name, and the coat conversion data is J
IS code and CTS (Computer TypeS
e'tting) code.

Since the code system of a natural language processing system is a system-specific code, it is necessary to perform code conversion of data. In the field-specific dictionary ID set, the field-specific dictionary 106 is a table in which kanji and their readings are paired. Each field to be processed is registered in the computer in advance and managed using a serial number (ID) starting from 1.

When using the field-specific dictionary 106, the field-specific dictionary 1 is added to the data attribute of the natural language processing human file data record.
Set to 0. Furthermore, natural language processing input file creation involves creating a natural language processing human file record for each extracted data.

The data preprocessed as described above is subjected to natural language processing in the next step S1, which will be detailed later in 6.
In the case of kana furi using the field-specific dictionary 106, as shown in FIG. 3, pattern matching is first performed with reference to the field-specific dictionary 108 (step 51^), and if matching is achieved, the pronunciation is output. Other than that, basic dictionary 1
Do the normal parting/kana-furi referring to 15 (
Step 5IB). The pressure format is as shown in FIG. , 4th
As shown in FIG. 5, the input original text data as shown in the figure is subjected to separation (part-of-speech decomposition) and kana translation. Immediately before the separated data is the word's part of speech identification 1.
A 0 is added to the code so that the part of speech of a word can be determined. Next, post-processing is performed on the natural language processing output file that has undergone natural language processing (step 520).
The details of the post-processing are shown in FIG. 6. First, code conversion is performed (step 521). Since the processing results of the natural language processing system are output as system-specific codes, the CTS of the kana-jiri processing result data is Coat conversion to code is performed (step 521), and then a database is created (step 522), that is, the code-converted data is output to a database format file record and registered in the database. Next, step S2) outputs the contents of the kana-furi file as a list.
The keyword data that has undergone zero proofreading, in which the surname and name in kana characters are corrected after adding red characters, etc., is made into a database of personal names in kana characters. The field-specific dictionary 106 is corrected for the data for which kana-furi was not correctly performed, and the accuracy of the next natural language processing is improved. Processing is performed according to the authority dictionary identification of the processing result, but if the reading has been obtained from the field-specific dictionary 106, the data in the field-specific dictionary is corrected, and for other data, it is checked to see if the correct reading has been assigned. After checking and correcting the information, it is registered in the field-specific dictionary 106 to improve the accuracy of natural language processing from the next time onwards. In addition, as field-specific dictionaries, it is possible to register medical terminology dictionaries, economic terminology dictionaries, chemical technical terminology dictionaries, etc.

By using field-specific dictionaries for first and last names, it is possible to perform kana-furi as shown in Table 1 below.Table 1 The basic dictionary 115 is the most basic for natural M-word processing (partition/kana-furi). In the dictionary, system dictionary g
131 and a user dictionary 132. By modifying the user dictionary 132, the accuracy of natural language processing can be improved.

In this invention, a general-purpose file (hereinafter referred to as N1. file) is used as a general-purpose human output file for CTS natural language processing.
However, in the NL file, as shown in Figure 14, the NL zein file, IIIL out file and N
It consists of three types of L information files, and the format is the same. The entire format consists of a header record and a data record, and the header record includes record identification, sequence number, file identification, job name, manuscript name, CTS system name, etc. Further, the data record includes record identification, sequence number, data number, IA type, data, etc.

As shown in FIG. 15, the input routine 5100 reads the NL zein file along with parameters to create a natural language processing input file and NL information file, the details of which are shown in FIG. 16. .

A natural language processing input file is created by reading the NL zein file, deleting functions by specifying parameters, and performing code conversion (external-system specific code). The position information and code conversion information of the deleted function are stored in an information file, and the job name etc. are output as a list after the processing is completed. In the parameter check (step 5101), the instruction of the execution of function deletion and code conversion information is analyzed. In the header record creation (step 5102), the natural language processing input file and the NL Create a header record K of the information file. The process of reading data with the same data number 21 (step 5203) uses all valid data of records having the same data number as a processing unit.

Therefore, valid data is extracted from data records having the same data NO among the NL zein file data records.

In data processing (step 5104), functions of the data extracted from the NL zein file are deleted and code conversion is performed. Information on the deleted functions and code conversion information are output to the NL information file, and processed data is output to the natural language processing human file. Further, in creating a data record (step 5105), the data after processing (deleting functions, code conversion) of the same data number is output to the natural language processing human file, and the processing information is output to the NL information file.

On the other hand, as shown in FIG. 17, the output routine 5200 in FIG. 14 reads the natural language processing output file and the NL information file together with parameters to create an NL out file as post-processing of the natural language processing. What are the details? ; It is as shown in Figure 18. In other words, by integrating the natural language processing output file and the NL information file,
A NL out file is created by returning the function and converting the code (system specific code - external) by specifying the parameter. After processing is completed, job names, etc. are output as a list. Parameter check (step 5201
), the presence or absence of function return execution and the instruction of code conversion information are analyzed. In the creation of a header record (step 5203), a header record of the NL out file is created from the contents of the header record of the NL information file and the natural language processing output file. Same data No.
The reading of the data (step 5204) is the same data NO.
The processing unit is all valid data of the record with . Although the natural language processing output file data record includes human input original text data and processing result data, only the processing result data is valid data. Therefore, the processing result data is extracted from the data record having the same data number in the natural language processing pressure cuff file record. In addition, data processing (
In step 52o5), function restoration and code conversion are performed on the data extracted from the natural language processing output file. The processed data is output to the NL out file.

The present invention can be used to support the construction of databases such as CD-ROMs, and can extract search keywords and add pronunciations to the extracted keywords. In addition, it can be used for printing work, such as creating printed materials with full ruby using the kana-Furi function, automatic kana-Furi of items such as address 1 name of the list, etc.
It can be used as a support system for index creation.

Effect of the invention.

As described above, according to the method of creating a kana-furi database using the field-specific dictionary of the present invention, it is possible to automatically create a database of personal names and terminology without requiring any specialized knowledge or skills.

[Brief explanation of the drawing]

FIG. 1 is a flowchart showing an example of the operation of this invention, and FIG.
Figure 3 is a flowchart showing an example of preprocessing operation, Figure 3 is a flowchart showing the operation of natural language processing, Figure 4 is a diagram showing an example of an original text subjected to natural language processing, and Figure 5 is a diagram showing an example of splitting kana. , Fig. 6 is a flowchart showing an example of post-processing operation, Fig. 7 is a block diagram showing an example of the hardware configuration of the natural language processing system, Fig. 8 is a diagram showing an example of its software configuration, and Fig. 9 is a diagram showing an example of the hardware configuration of the natural language processing system. A diagram showing the output format of writing,
Figure 1O is a diagram showing the output format of kana furi in dividing line units, Figure il1 is a diagram showing the output format of kana furi using a field-specific dictionary, Figure 12 is a diagram showing the output format of total ruhi furi, Fig. 13 is a diagram showing the output format of keyword extraction and kana translation to the keyword, Fig. 14 is a flowchart showing an example of the configuration of a general-purpose file used in this invention, Fig. 15 is a diagram showing the human output of the input routine, FIG. 16 is a flowchart showing the details of the human power routine, FIG. 17 is a flowchart showing the human output of the output routine, and FIG. 18 is a flowchart showing the details of the output routine. 10...Host machine, 11...CPII, 1
2...Memory, 14...Magnetic disk device, 15.
...Cassette magnetic tape device, 20...Magnetic tape device, 21...Laser printer, 22...Verification/correction terminal, 23...Console terminal. Engraving of drawings (no changes to the content) Requester's agent: Yu Angata, Sanbo 3 Ω Author: t9 Zu Hina Sojuku Zu Kyo Zume Zu Capital Zu certain map procedural amendment (method) November 20, 1990 Director General of the Patent Office Toshi Uematsu Kuchi = 1, case description 1' 1990 Patent Application No. 20
2973 No. 2, Relation to the case of method for creating a kana-furi database using a field-based dictionary of invention names Patent applicant (289) Inu Nippon Printing Co., Ltd. 4, attorney 5, date of amendment order October 15, 1990 day (all shipping date: October 30, 1990)

Claims (1)

[Claims] 1. A database is pre-processed, a natural language processing output file is created by natural language processing by referring to a field-specific dictionary and a basic dictionary, and a kana-furi database is created by post-processing. A method for creating a kana-furi database using a field-specific dictionary. 2. The method for creating a kana-furi database using a field-specific dictionary according to claim 1, wherein the field-specific dictionary is corrected when proofreading the keyword data. 3. Creation of a kana-furi database using the field-specific dictionary according to claim 1, wherein the preprocessing is a repetition of data extraction, code conversion, ID set of the field-specific dictionary, and creation of a natural language processing input file. Method. 4. Claim 1, wherein the post-processing performs code conversion and creation of a database format file on the natural language processing output file, and repeats the above operations.
How to create a kana-furi database using the field-specific dictionary described in .