JP2024128597A - Similar document search device, similar document search method and program - Google Patents

Abstract

To provide a similar document retrieval device, a similar document retrieval method, and a program which can increase the search accuracy.SOLUTION: A device 10 is for retrieving a similar document as a document similar to a related document as a document related to a retrieval word to be input from among a plurality of documents registered in advance. The device includes: a related document extraction unit 14 for extracting the related document tied in advance to the retrieval word in a predetermined rule base on the basis of the input retrieval word; and a similar document retrieval unit 16 for calculating similarity of the extracted related document and each document registered in advance, and retrieving the similar document on the basis of the calculated similarity.SELECTED DRAWING: Figure 1

Description

The present invention relates to a similar document search device, a similar document search method, and a program for searching documents based on their similarity.

A conventional method of natural language processing using a computer is a search process that searches documents stored in a database for documents similar to an input document (see, for example, Patent Document 1). There is also known a technique that extracts keywords that characterize documents from the documents to be searched using a specific algorithm, such as based on frequency of occurrence, and calculates cosine similarity, which expresses the closeness of the angle between vectors in the language space of the word and keyword entered by the user, to output documents with high relevance.

Patent No. 6190904

However, when searching a certain document group A, if the similarity between the input conditions (equivalent to the above words) and document group A is extremely low, search accuracy may decrease. In other words, if document group A does not contain many keywords similar to the above words, the similarity calculation using vectors according to the conventional search method may not be effective.

For example, consider the case where the search terms (words) are set to "Structure: Steel (S) construction" and "Within 300m distance from the coast" to search and output document group A on salt damage countermeasures from among documents in the construction sector. In this case, there is a high probability that "Within 300m distance from the coast" will be treated in natural language processing in roughly the same way as "Within 3000m distance from the coast," which does not require much salt damage countermeasures, and therefore there is a high possibility that document groups other than those related to salt damage countermeasures will be output. In this way, if the meaning of the numbers contained in the search terms is not taken into consideration, there is a risk of a decrease in search accuracy.

The present invention has been made in consideration of the above, and aims to provide a similar document search device, a similar document search method, and a program that can improve search accuracy.

In order to solve the above problems and achieve the object, the similar document search device of the present invention is a device that searches for similar documents, which are documents similar to related documents, which are documents related to an input search term, from among multiple documents that have been registered in advance, and is characterized by having a related document extraction unit that extracts the related documents that are pre-linked to the search term based on a predetermined rule base, based on the input search term, and a similar document search unit that calculates the similarity between the extracted related documents and each document registered in advance, and searches for the similar documents based on the calculated similarity.

The similar document search method according to the present invention is a method for searching for similar documents, which are documents similar to related documents, which are documents related to an input search term, from among a plurality of pre-registered documents, and is characterized by having a step of extracting the related documents that are pre-linked to the search term based on a predetermined rule base, based on the input search term, and a step of calculating the similarity between the extracted related documents and each pre-registered document, and searching for the similar documents based on the calculated similarity.

The program according to the present invention is characterized in that it causes a computer to execute the similar document search method described above.

The similar document search device according to the present invention is a device that searches for similar documents, which are documents similar to related documents, which are documents related to an input search term, from among multiple documents that have been registered in advance, and includes a related document extraction unit that extracts the related documents that are linked to the input search term in advance using a predetermined rule base, based on the input search term, and a similar document search unit that calculates the similarity between the extracted related documents and each document that has been registered in advance, and searches for the similar documents based on the calculated similarity, thereby achieving the effect of improving search accuracy.

The similar document search method according to the present invention is a method for searching for similar documents, which are documents similar to related documents, which are documents related to an input search term, from among multiple documents registered in advance, and includes the steps of extracting the related documents that are linked to the search term in advance on a predetermined rule basis based on the input search term, calculating the similarity between the extracted related documents and each document registered in advance, and searching for the similar documents based on the calculated similarity, thereby achieving the effect of improving search accuracy.

FIG. 1 is a schematic diagram showing the configuration of an embodiment of a similar document search device according to the present invention. FIG. 2 is a schematic flow diagram showing an embodiment of a similar document search method according to the present invention. FIG. 3 is a diagram showing an example of a search according to this embodiment.

Below, embodiments of a similar document search device, a similar document search method, and a program according to the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to these embodiments.

As shown in FIG. 1, a similar document search device 10 according to an embodiment of the present invention includes an input unit 12, a related document extraction unit 14, a similar document search unit 16, an output unit 18, and a memory unit 20.

The input unit 12 accepts input of search terms that are used as keys in searching for similar documents, and is composed of, for example, a keyboard for an input interface and an input field provided on a display screen. The search terms are assumed to be words with a certain meaning or words such as numbers including units such as length. For example, in the case of the construction business field, the search terms may be one or more of project information (construction site, total floor area, area, structure, etc.), performance information (coastal area, soft ground, special structure, etc.), and finishing information (wall: stone, roof: waterproof, etc.). A number of options may be provided as keywords for the search terms, and one of them may be selected and input in the input unit 12. In addition, multiple search terms may be input to enable AND searches or OR searches. For example, the two search terms "structure: steel (steel) construction" and "within 300 m distance from the coast" may be input.

The related document extraction unit 14 extracts related documents from the storage unit 20 that are linked in advance with a predetermined rule base based on the search term input to the input unit 12. The related documents and the rule base are stored in the storage unit 20. The search term and information on related documents related to the search term (e.g., document titles, etc.) are registered in advance in the rule base. This allows the search term in lexical form to correspond to the related document in document form. The rule base can analyze the search term and the document from a large amount of past data, and set the relevance between the search term and the document as a rule. This relevance may be set based on, for example, a weighting value calculated from the frequency of use of the search term input in the past and the frequency of browsing of similar documents, or may be set manually in advance. If the search term is fragmentary information on a construction project, documents describing frequently occurring defect cases may be set as related documents. For example, in the above example, if the search terms are "Structure: Steel (S) construction" and "Distance from the coast within 300 m," "Documents on salt damage prevention measures" may be linked as related documents.

The similar document search unit 16 uses the related documents extracted by the related document extraction unit 14 as input documents, calculates the similarity between this input document and each document to be searched stored in the storage unit 20 by natural language processing, and searches for similar documents that are highly similar to the related document based on the calculated similarity. The similarity can be calculated, for example, by dividing each of the related documents and each document to be searched into morphemes and counting the number of words that appear in common between them, or by using a known similar document search technique such as the vector space method described in the above-mentioned Patent Document 1.

The output unit 18 outputs information about the similar documents searched for by the similar document search unit 16 as the similar document search results, and is composed of, for example, a display or a printer that displays the character strings of the similar documents. For example, the titles of the similar documents can be output as the similar document search results.

The storage unit 20 stores multiple documents that are the subject of search by the similar document search unit 16, multiple related documents, and a rule base, and is composed of, for example, a database or memory. The documents and related documents stored in the storage unit 20 are electronic documents on electronic media that contain text-format data such as electronic books, electronic files, and web pages. These electronic documents have at least a main text and a title. The electronic documents may be electronic documents such as various laws and regulations used in the construction business field, company standards, construction manuals, collections of tips, and lists of dos and don'ts.

An example of the hardware of the similar document search device 10 is a computer equipped with a CPU, RAM, ROM, a hard disk, a communication interface, etc. Similar document search can be performed by storing a program that realizes each of the above functions in the RAM or ROM and executing this program by the CPU. Such programs are also included in the scope of the present invention.

Next, a similar document search method according to an embodiment of the present invention will be described. This similar document search method is executed, for example, by each unit of the similar document search device 10 performing the processes of steps S1 to S4 shown in FIG. 2.

First, in step S1, a search term is input to the input unit 12. In the next step S2, the related document extraction unit 14 extracts related documents that are pre-linked to the search term based on the search term and the rule base. In the next step S3, the similar document search unit 16 calculates the similarity between the related document and each document stored in the memory unit 20. In the next step S4, similar documents are searched for based on the calculated similarity, and the search results are output from the output unit 18.

Figure 3 is a conceptual diagram showing a search example according to this embodiment. As shown in this figure, when one or more pieces of information are input as search terms, such as project information (construction site, total floor area, area, structure, etc.), performance information (coastal area, soft ground, special structure, etc.), and finishing information (wall: stone, roof: waterproof, etc.), related documents (documents describing defect cases) linked to the search terms are extracted from the storage unit 20 based on the set rule base. After that, natural language processing (similar document search) is performed between the extracted related documents and each document stored in the storage unit 20, and the titles of similar documents that are highly similar to the related documents are output. In the example shown in the figure, a collection of tips, a construction manual, and a list of dos and don'ts are output as similar documents.

In this way, according to this embodiment, the input search term is pseudo-converted into an intermediary related document, and then a similar document search is performed between the related documents and each document using natural language processing, thereby outputting highly related similar documents. In this way, even if each document does not have many keywords related to the search term, a highly accurate search can be performed. Therefore, the search accuracy can be improved compared to the conventional method described above.

In particular, according to this embodiment, the meaning of the numerical values in the search term, which is difficult to handle in natural language processing, is replaced with related documents, making natural language processing easier. For example, in the above example, the search term "within 300 m of the coast" is likely to be treated almost the same in natural language processing as "within 3000 m of the coast". However, by replacing "within 300 m of the coast" with related documents on salt damage countermeasures, it is possible to give the meaning that "within 300 m" is related to "salt damage". This makes it possible to associate the similar documents ultimately found with the search term through the word "salt damage".

As described above, the similar document search device of the present invention is a device that searches for similar documents, which are documents similar to related documents that are documents related to an input search term, from among multiple pre-registered documents, and includes a related document extraction unit that extracts the related documents that are pre-linked to the search term based on a predetermined rule base based on the input search term, and a similar document search unit that calculates the similarity between the extracted related documents and each pre-registered document and searches for the similar documents based on the calculated similarity, thereby improving search accuracy.

The similar document search method according to the present invention is a method for searching for similar documents, which are documents similar to related documents that are documents related to an input search term, from among multiple documents registered in advance, and includes the steps of extracting the related documents that are linked to the search term in advance on a predetermined rule basis based on the input search term, calculating the similarity between the extracted related documents and each document registered in advance, and searching for the similar documents based on the calculated similarity, thereby improving search accuracy.

As described above, the similar document search device, similar document search method, and program of the present invention are useful for searching documents based on document similarity, and are particularly suitable for improving search accuracy.

10 Similar document search device 12 Input unit 14 Related document extraction unit 16 Similar document search unit 18 Output unit 20 Storage unit

Claims (3)

A device for searching for similar documents, which are documents similar to related documents, which are documents related to an input search term, from among a plurality of documents registered in advance, comprising:
a related document extraction unit that extracts the related documents that are linked to the input search term in advance according to a predetermined rule base, based on the input search term;
A similar document search device comprising a similar document search unit that calculates the similarity between the extracted related documents and each document registered in advance, and searches for the similar documents based on the calculated similarity. A method for searching for similar documents, which are documents similar to related documents, which are documents related to an input search term, from among a plurality of documents registered in advance, comprising the steps of:
extracting the related documents that are linked to the input search term in advance based on a predetermined rule base;
A similar document search method comprising the steps of: calculating a similarity between the extracted related documents and each document registered in advance; and searching for the similar documents based on the calculated similarity. A program that causes a computer to execute the similar document search method described in claim 2.

Images