JP2001101184A - Method and device for generating structurized document and storage medium with structurized document generation program stored therein - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make an attribute decision reference in the case of structuring a nonstructured document variable according to a theme and to retrieve each different theme selected by a retriever at retrieval. SOLUTION: A basis attribute set is set in each theme, when a semi- structured document is inputted, pattern matching of the character string of the document is performed to a preliminarily registered pattern, and dictionary matching of the character string of the document is performed to an attribute dictionary in which a word and a plurality of attribute names are associated and described. An attribute candidate for the character string of the semi- structurized document is extracted, and attributes that are possible to appear in the semi-structured document are acquired by referring to attributes in each theme. An attribute candidate having a high priority is adopted as an attribute among the attribute candidates by referring to an attribute relation rule showing whether the attributes are in a word order relation or in an exclusion relation with each other and adding priorities, and the attribute candidate is outputted as a structurized document.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for generating a structured document and a storage medium storing a structured document generating program.
The present invention relates to a structured document generation method and apparatus for structuring an unstructured document based on a theme, and a storage medium storing a structured document generation program.

[0002]

2. Description of the Related Art Structuring is to assign attributes to information (character strings of characteristics) in a plain document and clarify the relationship between the attributes. There are semantic attributes and logical attributes in the attributes, but the semantic attributes are handled here. In the case of a semantic attribute, the attribute given to a certain character string differs depending on the theme (viewpoint).

For example, the following two sentences are all sentences relating to "camera". How to take a picture with “camera” Significantly lower price for “camera” In these two sentences, the word “camera” is used differently. Is a "camera as a means" for taking a picture, and a statement is a "camera as a product" to be reduced. Such usage of words for each context is called an "attribute". By the way, the attribute is decided
Not necessarily the author. It can be a reader or some kind of system. Conventional full-text search (eg "goo" http: //www.g
oo.ne.jp), the above “attributes” are taken into account.
Since all results that match the input keyword are returned, many useless search results are included. However, if the information is classified by attributes, the user can obtain only the information that he really wants. Here, giving an attribute to information (a word or the like) in the text is called “structuring”.

The conventional document structuring specifies a single attribute to be given to a character string by fixing a theme (viewpoint). For example, when a character string “apple” appears in a document, various attributes such as “fruit”, “produce”, and “snack” may be given depending on the theme (viewpoint) of the document. However, by describing only "fruit" in the attribute dictionary, the character string "apple" always has the attribute "fruit".

[0005] In addition, since the attribute of the logical attribute is specified, a specific attribute is assigned using a logical relation rule or an attribute value extraction rule such as a dictionary. FIG. 9 shows an example of a conventional document structuring apparatus. The document structuring apparatus shown in FIG. 1 includes a semi-structured document input unit 11, an attribute value extraction unit 12, an extraction rule database 13, and a structured document output unit 14.

In the document structuring device, when a document is input in the semi-structured document input unit 11, the attribute value extracting unit 12 refers to the extraction rule database 13 and assigns a specific attribute to a certain character string. . The structured document output unit 14 outputs a structured document in which a character string to which an attribute value has been added is integrated. Details are disclosed in JP-A-9-69101.

[0007]

However, in the above-described conventional document structuring apparatus, the attribute determination criterion cannot be changed according to the theme, and the search by the theme selected by the searcher at the time of the search is flexibly performed. There is a problem that you can not. SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and has a structure in which an attribute determination criterion when structuring an unstructured document is variable according to a theme, and a search can be performed for each theme selected by a searcher during a search. An object of the present invention is to provide a document generation method and apparatus, and a storage medium storing a structured document generation program.

[0008]

FIG. 1 is a diagram for explaining the principle of the present invention. The present invention (Claim 1) provides a structured document generation method for structuring an unstructured document based on a theme for the purpose of searching documents by theme. A basic attribute set is set for each theme (step 1), and when a semi-structured document is input (step 2),
The character string of the semi-structured document is subjected to pattern matching with a pattern registered in advance for the character string of the document, and dictionary matching with a word and an attribute dictionary in which a plurality of attribute names are correspondingly described. Are extracted (step 3), and with respect to the extracted attribute candidates, attributes which may appear in the semi-structured document are obtained by referring to the attributes for each theme, and the semi-structured document is obtained. A priority is given according to the co-occurrence relationship or the exclusive relationship with reference to an attribute relationship rule indicating whether the attributes that may appear in the co-occurrence relationship or the exclusive relationship are present (step 4) Among the attribute candidates, the one with the highest priority is adopted as the attribute (step 5), and the character string of the semi-structured document input based on the adopted attribute is tagged (step 6). ), Structured And outputs as written (Step 7).

According to the present invention (claim 2), the priority of the attribute candidate is compared with a predetermined threshold, and if the priority is lower than the threshold, the attribute candidate is deleted. The present invention (claim 3)
Is a structured document generating apparatus for structuring a non-structured document based on a theme for the purpose of searching documents by theme, and a theme specifying means 29 for specifying in advance a theme which is a viewpoint of search; An attribute set storage unit 26 storing a basic attribute set for each theme, a pattern storage unit 23 storing a pattern including a symbol, a character string, and a part of speech, and a word and a plurality of attribute names are associated with each other. A dictionary storage means 24 stored and stored; an attribute relation rule storage means 27 storing attribute relation rules indicating whether a certain attribute and another attribute are co-occurring or exclusive;
A semi-structured document input unit 21 for inputting a semi-structured document;
When a semi-structured document is input from the semi-structured document input means 21, pattern matching is performed on the character string of the document by referring to the pattern storage means 23, and further, the dictionary is referred to by referring to the dictionary storage means 23. Attribute candidate extracting means 22 that performs matching and extracts attribute candidates for the character string of the semi-structured sentence, and the attribute candidates extracted by the attribute candidate extracting means 22,
The attribute that may appear in the semi-structured document is acquired, and the attributes that may appear in the semi-structured document have a co-occurrence relationship with reference to the attribute relation rule storage unit 27. Attribute cost calculating means 25 for assigning a priority according to whether the attribute is present or in an exclusive relationship, and adopting an attribute candidate having the higher priority as an attribute; and a semi-structure input based on the adopted attribute. And structured document output means 28 for tagging a character string of the structured document and outputting the result as a structured document.

The present invention (Claim 4) Attribute cost calculation means 2
In step 5, means for comparing the priority of the attribute candidate with a predetermined threshold and deleting the attribute candidate when the priority is lower than the threshold is included. The present invention (claim 5) is a storage medium storing a structured document generation program for structuring an unstructured document based on a theme for the purpose of searching documents by theme, comprising: And a semi-structured document input process for inputting, when a semi-structured document is input, a symbol registered in advance for a character string of the document,
Pattern matching is performed with reference to a pattern including a character string and a part of speech, and dictionary matching is performed with reference to a dictionary in which a word and a plurality of attribute names are registered in advance and the semi-structured document is The attribute candidate extraction process for extracting attribute candidates for the character string and the attribute candidates extracted in the attribute candidate extraction process are referred to a basic attribute set registered in advance for each theme, and the semi-structured document is referred to. Attributes that may appear in the document, and an attribute relationship rule that indicates whether the attributes that may appear in the semi-structured document are co-occurring or exclusive. Referring to the attribute cost calculation process of assigning a priority according to the co-occurrence relationship or the exclusive relationship of the attribute candidates and employing the attribute candidate having the higher priority as the attribute, To tag on a string of semi-structured document input based on the attribute, and a structured document outputting process of outputting a structured document.

In the present invention (claim 6), in the attribute cost calculation process, the priority of the attribute candidate is compared with a predetermined threshold, and if the priority is lower than the threshold, the attribute candidate is deleted. Process. As described above, according to the present invention, by setting a basic attribute set and a basic attribute relation rule for each preset theme, it is possible to make the attribute determination standard variable.

Furthermore, a plurality of attributes are first assigned to information (character strings) in a document and attribute candidates are set, and attributes can be specified by referring to co-occurrence and exclusion relation rules of basic attributes. In addition, a search for each theme selected by the searcher at the time of search is enabled.

[0013]

FIG. 3 shows a configuration of a document structuring apparatus according to the present invention. The document structuring apparatus shown in FIG. 1 includes a semi-structured document input unit 21, an attribute candidate extraction unit 22, a pattern database 23, a dictionary database 24, an attribute cost calculation unit 25, an attribute set database 26, an attribute relation rule database 27, a structure It comprises a structured document output unit 28 and a theme designation unit 29.

The semi-structured document input unit 21 inputs a semi-structured document and transfers it to the attribute candidate extracting unit 22. The attribute candidate extraction unit 22 extracts a character string of the input semi-structured document from the pattern database 23 or the dictionary database 2.
4 for performing pattern matching and dictionary matching, and includes a pattern matching processing unit 221 and a dictionary matching processing unit 222.

The pattern matching processing section 221 performs pattern matching with reference to the pattern database 23. The pattern database 23 describes patterns such as symbols, character strings, parts of speech, and the like, and a plurality of attribute names. The dictionary match processing unit 222 stores the dictionary database 24
And performs dictionary matching processing. The dictionary database 24 describes words and a plurality of attribute names in association with each other. By performing matching with these databases 23 and 24 in the attribute candidate extraction unit 22, all attribute candidates are extracted from the document data.

The attribute cost calculator 25 calculates the cost of each attribute candidate. In the cost calculation, an attribute set for each theme is acquired by referring to the attribute set database 26, and then, the attribute relations in the attribute set acquired from the attribute set database 26 are acquired by referring to the attribute relation rule database 27. Examine co-occurrence / exclusivity and calculate weight. The attribute set database 26 describes a set of theme names and attributes corresponding to the theme names. The attribute relation rule database 27 indicates by weight whether a certain attribute and another certain attribute have a co-occurrence relationship or an exclusive relationship.

The attribute cost calculation unit 25 preferentially employs, from among a plurality of attribute candidates for a certain character string, one having a higher weight as an attribute. If there is only one attribute candidate and the weight is below the threshold, the attribute is deleted. The structured document output unit 28 includes an attribute cost calculation unit 25
Tag the character string with the attribute specified in and output.

The theme designation section 29 allows the user to input the determined theme to the attribute cost calculation section 25. FIG.
5 is a flowchart showing the operation of the document structuring device of the present invention. Step 101) First, as preprocessing, a plurality of search themes are set, and a database and rules are prepared in advance. Attributes are set in the attribute set database 26 for each theme as a database. A pattern for extracting symbol and numeral attributes is created and set in the pattern database 23. Information for extracting a character string attribute is set in the dictionary database 24. Attribute relationship rules describing attribute relationships are stored in an attribute relationship rule database 27.
Set to.

Step 102) Semi-structured document input unit 2
First, a semi-structured document is input and transferred to the attribute candidate extracting unit 22. Step 103) The pattern matching processing unit 221 of the attribute candidate extraction unit 22 refers to the pattern database 23 and performs a pattern matching process on a character string of the input semi-structured document.

Step 104) The dictionary match processing section 222 of the attribute candidate extraction section 23 performs a dictionary match process on the character string with reference to the dictionary database 24.
Extract attribute candidates. Note that the processing order of step 103 and step 104 may be reversed.

Step 105) The attribute candidates (stored in the temporary spool) extracted by the attribute candidate extraction unit 22 are sent to the attribute cost calculation unit 25. The theme determined by the user is input from the theme designation unit 29 to the attribute cost calculation unit 25, and the attribute cost calculation unit 25
The priority is given with reference to the attribute set database 26 and the attribute relation rule database 27.

Step 106) Attribute cost calculator 25
Determines whether the priority assigned to the attribute candidate is a candidate that is equal to or less than a predetermined threshold. If the priority is equal to or less than the threshold, the process proceeds to step 107;
08. Step 107) If the priority of the attribute candidate is equal to or less than the low predetermined threshold, the attribute candidate is deleted.

Step 108) If the priority of the attribute candidate is lower than the predetermined threshold, the attribute of the attribute candidate is determined for each theme, and a tag is added to the original document (semi-structured document). The document is output from the structured document output unit 28 as a structured document.

[0024]

Embodiments of the present invention will be described below with reference to the drawings. In the following description, description will be made based on the above-described processing of FIG. FIG. 5 shows an example of an input semi-structured document according to an embodiment of the present invention. First, the semi-structured document shown in FIG. 5 is input from the semi-structured document input unit 21 (step 102).

The attribute candidate extraction unit 22 performs attribute candidate extraction processing on the input semi-structured document. When the pattern matching processing unit 221 refers to the pattern database 23 and finds that the pattern “XX-XXXX-XXXX” corresponds to the “phone number” and “FAX number” attributes, the character string “03- 333X-000X ”and the pattern match, so“ 03-333X-000 ”
In "X", the "telephone number" and "FAX number" attributes are registered as candidates (step 103).

The dictionary match processing unit 222 performs a dictionary match process with reference to the dictionary database 24. For example,
If the word “hamburger” is described in the dictionary database 24 as having the semantic attributes “menu” and “product”, the “menu” and “product” are added to the character string “hamburger” in FIG. Are acquired (step 104).

The attribute cost calculator 25 calculates each attribute candidate. The attribute set for each theme is acquired with reference to the attribute set database 26. This example is shown in FIG. For example, the user wants to structure the semi-structured document shown in FIG. 5 from the viewpoint of “theme: restaurant advertisement”, and inputs the theme from the theme specifying unit 29. As a corresponding attribute set, “shop name, menu, address,...” Or the like can be obtained (step 105).

Here, the attribute candidates that are not included in the attribute set among the attribute candidates extracted in steps 103 and 104 are deleted (step 107). Next, the attribute cost calculation unit 25 checks the co-occurrence and the exclusion of the relation between the attributes in the attribute set obtained by referring to the attribute relation rule database 27, and performs the weight calculation. This example is shown in FIG. In the example of FIG. 7, the combination of attributes (attribute values) is described on the left, and the weighting cost is described on the right. As for the cost, plus is the co-occurrence rule and minus is the exclusive rule. Here, FIG.
In the example, in the attribute candidates "menu" and "product name" for the character string "hamburger", the "menu" attribute is given to "hamburger" because the total weight of "menu" is heavier (step 105). .

In the structured document output unit 22, the specified attribute is tagged with a character string and output. For example, as shown in FIG. 8, it is output as an XML document. Although the above embodiment has been described based on the configuration shown in FIG. 3, the present invention is not limited to this example, and the semi-structured document input unit, attribute candidate extraction unit, attribute cost calculation unit, and structuring The document output unit is constructed as a program and stored in a disk device connected to a computer used as a structured document generation device, a floppy disk, a portable storage medium such as a CD-ROM, and the present invention is implemented. The present invention can be easily realized by installing the software at that time.

It should be noted that the present invention is not limited to the above-described embodiment, and various modifications and applications are possible within the scope of the claims.

[0031]

As described above, according to the present invention, the attribute determination criterion for structuring an unstructured document can be changed according to the theme, and a search can be performed for each theme selected by the searcher at the time of search. it can.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining the principle of the present invention.

FIG. 2 is a principle configuration diagram of the present invention.

FIG. 3 is a configuration diagram of a document structuring apparatus of the present invention.

FIG. 4 is a flowchart showing the operation of the document structuring apparatus of the present invention.

FIG. 5 is an example of an input semi-structured document according to an embodiment of the present invention.

FIG. 6 is an example of a theme-specific attribute set according to an embodiment of the present invention.

FIG. 7 is an example of an attribute relation rule according to an embodiment of the present invention.

FIG. 8 is an example of a structured document output according to an embodiment of the present invention.

FIG. 9 is a configuration diagram of a conventional document structuring apparatus.

[Description of Signs] 21 semi-structured document input unit, semi-structured document input unit 22 attribute candidate extraction unit, attribute candidate extraction unit 23 pattern storage unit, pattern database 24 dictionary storage unit, dictionary database 25 attribute cost calculation unit, attribute Cost calculation unit 26 Attribute set storage unit, attribute set database 27 Attribute relation rule storage unit, Attribute relation rule database 28 Structured document output unit, Structured document output unit 29 Theme specification unit, Theme specification unit 221 Pattern match processing unit 222 Dictionary Match processing section

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Katsumi Takahashi 2-3-1 Otemachi, Chiyoda-ku, Tokyo F-term within Nippon Telegraph and Telephone Corporation (reference) 5B009 QA00 5B075 ND03 NK02 NK32 NK42 NK46 NR03 NR12 PP02 PP12 PP25 PR08 QM10 UU06

Claims (6)

[Claims] In a structured document generation method for structuring an unstructured document based on a theme for the purpose of searching documents by theme, a theme as a search viewpoint is set in advance, and a theme is set for each theme. When a semi-structured document is input, a pattern matching with a pattern registered in advance for a character string of the document is performed.
And dictionary matching with a word and an attribute dictionary in which a plurality of attribute names are described in correspondence with each other, to extract attribute candidates for the character string of the semi-structured document. By referring to the attribute, an attribute which may appear in the semi-structured document is obtained, and the attributes which may appear in the semi-structured document have a co-occurrence relationship or an exclusive relationship. The priority is given according to the co-occurrence relationship or the exclusive relationship with reference to the attribute relationship rule indicating whether the attribute candidate is present in the attribute candidate. Tagging a character string of the semi-structured document input based on the document, and outputting the result as a structured document. 2. The structured document generation method according to claim 1, wherein the priority of the attribute candidate is compared with a predetermined threshold, and if the priority is lower than the threshold, the attribute candidate is deleted. 3. A structured document generating apparatus for structuring an unstructured document based on a theme for the purpose of searching documents by theme, wherein a theme designating means for designating a theme which is a search viewpoint in advance. Attribute set storage means for storing a basic attribute set for each theme, Pattern storage means for storing patterns including symbols, character strings, and parts of speech, Words are associated with a plurality of attribute names A dictionary storage means for storing attribute relation rules indicating whether a certain attribute and another attribute are in a co-occurrence relation or in an exclusive relation; A semi-structured document input unit for inputting a document, and when a semi-structured document is input from the semi-structured document input unit, pattern matching is performed on the character string of the document by referring to the pattern storage unit. Performing, further performing dictionary matching with reference to the dictionary storage means, extracting attribute candidates for the character string of the semi-structured document, attribute candidate extraction means, for the attribute candidates extracted in the attribute candidate extraction means, By referring to the attribute set storage means, an attribute which may appear in the semi-structured document is obtained, and by referring to the attribute relation rule storage means, an attribute which may appear in the semi-structured document is obtained. Attribute cost calculation means for assigning priorities according to whether coexisting or exclusive relations between attributes having similarity, and adopting an attribute candidate having the higher priority as an attribute; Structured document output means for tagging a character string of the input semi-structured document based on the attribute and outputting the result as a structured document Write generating device. 4. The attribute cost calculation means includes a means for comparing the priority of the attribute candidate with a predetermined threshold, and deleting the attribute candidate when the priority is lower than the threshold. Item 3. The structured document generation device according to Item 3. 5. A storage medium storing a structured document generation program for structuring an unstructured document based on a theme for the purpose of searching for a document by theme, wherein a half-structured document is inputted. A structured document input process, when the semi-structured document is input, perform pattern matching by referring to a pattern including a symbol, a character string, and a part of speech registered in advance with respect to a character string of the document, An attribute candidate extraction process for performing dictionary matching by referring to a dictionary in which words and a plurality of attribute names are registered in advance in association with each other, and extracting attribute candidates for the character strings of the semi-structured document; The attribute candidates extracted in the extraction process are likely to appear in the semi-structured document with reference to a basic attribute set registered in advance for each theme. The attribute candidate that may appear in the semi-structured document has a co-occurrence relationship or an exclusive relationship with the attribute candidate. An attribute cost calculation process for assigning priorities in accordance with an originating relationship or an exclusive relationship and employing an attribute candidate having a higher priority as an attribute; and a character of the semi-structured document input based on the adopted attribute A structured document output program for tagging a column and outputting the structured document as a structured document. 6. The attribute cost calculation process includes a process of comparing the priority of the attribute candidate with a predetermined threshold, and deleting the attribute candidate if the priority is lower than the threshold. A storage medium storing the structured document generation program according to item 5.