JPH07334515A - Information retrieval method and device - Google Patents

Abstract

(57) [Abstract] [Purpose] To provide an information search method capable of searching a document (document) intended by a searcher at a higher speed in a field such as a document database. [Structure] A search character string is input (step 101) and decomposed into consecutive characters (step 102). For consecutive characters, the degree of importance is calculated according to the appearance frequency and the combination of character types (steps 104 to 106), and it is checked whether or not the consecutive characters are included in the search target document (step 108). In this case, the importance of the consecutive character is added to the importance of the document to be searched (step 109).

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for retrieving information in a literature database or the like, and more particularly to an information retrieval method and apparatus capable of full-text retrieval or all-item retrieval.

[0002]

2. Description of the Related Art When searching for a property that a user needs from a database that holds a large number of information such as documents as properties, by assigning a keyword to each property in advance and searching for a keyword that matches the search key, There are a keyword search method for searching for a desired property and a direct search method for searching for a property including a search key by performing a full-text search for all information. Since the keyword search method assigns keywords to each property stored in the database in advance, it requires a considerable amount of manpower to perform the task. If any keywords are assigned, the number of keywords will be enormous. However, there is a problem that it is difficult to assign an accurate keyword, and it may not be possible to reach a desired property. Also, the direct search method is
If the number of properties to be searched is large or the amount of data for each property is large, the search cannot be completed within a realistic time range, and so-called fuzzy search cannot be generally performed. There is a problem.

In order to solve the above-mentioned problems of the conventional information retrieval method, the applicant of the present invention has disclosed in Japanese Patent Laid-Open No. 3261/1992.
No. 64 publication does not require the addition of keywords,
We proposed a database search system that enables high-speed search. This database search system obtains autocorrelation information for each property to be searched in advance, and at the time of search, obtains autocorrelation information of the search key and determines the degree of matching between the autocorrelation information of the search key and the autocorrelation information of the property. Is calculated for each property, and the property with the highest degree of matching is to be output. For example, a fixed-size binary matrix is used as the autocorrelation information. Since this search system searches based on fixed-size autocorrelation information regardless of the amount of data for each property, the search time can be greatly shortened.
For properties that include a character string that is a little different from the search key, a fairly high degree of matching can be obtained, so it is possible to perform a fuzzy search and automatically calculate the autocorrelation information. Compared to the search method, the amount of work required to build the database can be significantly reduced, and the maximum matching score is given to properties that include the search key, so you will not miss such properties. , And so on.

A specific example of the database search system described in Japanese Patent Laid-Open No. 4-326164 will be described below. Here, it is assumed that the property to be searched is an English text and each character of the text is represented by an ASCII (ASCII) code. The ASCII code is usually 8 bits, but since the most significant bit is not used as long as normal characters in English are used, only the lower 7 bits are considered. As the autocorrelation information, since each character is represented by an integer from 0 to 127 by a 7-bit code, a 128 × 128 binary matrix is used. Each element of this matrix is assumed to be initialized to "0".

First, from the English text, for each character of the English text, a consecutive character having a predetermined number of characters starting from that character is extracted. The English text is "This_is_a_pe
n. "(where" _ "represents a space), and if the predetermined number of characters is 3," Thi "," his "," is_ "," s_ "
The consecutive characters a "," _ a _ "," a_p "," _ pe "," pen "," en. "," n. "are extracted.
The character code of the second, third, and third characters is c
If the values are ₁ , c ₂ and c ₃ , the values of the elements (c ₁ and c ₂ ) and (c ₁ and c ₃ ) of the binary matrix representing the autocorrelation information are set to "1". By performing this operation for all extracted consecutive characters, the autocorrelation information of the target English text is obtained.

On the other hand, at the time of retrieval, first, the autocorrelation information of the retrieval key is obtained by the same procedure as described above. And
For each property, the binary matrix of the autocorrelation information of the property is compared with the binary matrix obtained from the search key, and each element that is "1" in the binary matrix from the search key is obtained from the property. Check whether it is "1" in the binary matrix. Of the elements of the matrix that is "1" in the binary matrix corresponding to the search key, the ratio of those that are "1" in the binary matrix corresponding to the property is the matching degree. Then, the properties are output in order from the one with the highest degree of matching.

In this specific example, the autocorrelation information is calculated based on consecutive characters. Roughly speaking, the degree of matching is calculated according to how many consecutive characters included in the search key are included in the property. Then, rather than checking the matching of consecutive characters verbatim, by converting the autocorrelation information and then calculating the matching degree, the search can be performed at extremely high speed.

[0008]

[Patent Document 1] Japanese Patent Application Laid-Open No. 4-32616
By the database search system disclosed in No. 4,
As described above, it has become possible to perform a document search at high speed without using a keyword. However, the number of documents and the amount of information stored in the literature database are increasing rapidly, and it is required to realize faster and more reliable literature retrieval.

It is an object of the present invention to provide an information retrieval method and apparatus capable of retrieving what a retrieval person intends at a higher speed.

[0010]

An information retrieval method of the present invention is an information retrieval method for retrieving a desired document from a set of retrieval target documents based on an input retrieval character string, A consecutive-character extracting step of forming a consecutive-character group for search by extracting consecutive-characters having a character length from the search character string; and, for each consecutive character included in the consecutive-character group for search, the importance of the consecutive character is calculated. For the required importance determining step and each of the search target documents, it is checked for each consecutive character included in the search consecutive character group whether the consecutive character is included in the search target document, and the consecutive character is included. In this case, a search step of adding the importance level corresponding to the consecutive character to the importance level of the search target document, and the output of the search result according to the importance level of each search target document after the search step is performed. And an output step to be performed.

The information retrieval apparatus of the present invention is an information retrieval apparatus for retrieving a desired document from a set of retrieval target documents based on an inputted retrieval character string, and an input for inputting the retrieval character string. Means, a consecutive-character extracting means for extracting a consecutive character having a predetermined character length from the search character string to form a consecutive-character group for search, and an importance storage means for storing the importance of each document to be searched. , The importance for the consecutive character is determined for each consecutive character included in the consecutive character group for search, and the consecutive character is searched for each consecutive character included in the consecutive character group for search for each search target document. A search unit that checks whether the document is included in the target document, and, if the consecutive character is included, adds the importance corresponding to the consecutive character to the importance of the search target document in the importance storage unit; Refer to the importance degree storage means, and search for each of the above Depending on the importance of each book and an output means for outputting the retrieval result.

[0012]

When a document database is searched, it is common to focus on the local structure of the document itself. It is unlikely that you would like to perform a search based on the correlation between the content written in the location of interest and the content written, for example, five pages ahead. When paying attention to the local structure, rather than using a search key with a length of several words to several tens of words as it is, as described in "Prior Art", the search key is decomposed into consecutive characters and this series is used. It is more efficient to search based on the letters.

By the way, considering the situation in which the searcher is actually trying to perform a search, this searcher must select a search key with some meaning. Further, when considering a document to be searched, not all the words included in this document have the same weight, but characteristic words that are useful for identifying the document and words that are not so are mixed. In the conventional search method, especially in the case of consecutive characters, the contribution of each consecutive character to the search result is equal and does not reflect the meaning of the searcher's choice or the characteristic content of the document. , I often hit unintended documents.

In the present invention, the search key (search character string) is decomposed into consecutive characters, and then it is judged whether or not the consecutive characters are characteristic for the search, and more characteristic consecutive characters are searched. Since it greatly contributes to the search result, it is possible to surely perform the search according to the intention of the searcher. Specifically, for each consecutive character, the proportion that contributes to the search result of that consecutive character is determined as the importance, and if the consecutive character is included in the search target document, the importance of the consecutive character is set as the search target. It may be added to the importance (evaluation value) of the document. As a method of determining the importance for consecutive characters, for example,
For each consecutive character used in the search, determine the frequency of occurrence of that consecutive character in all the documents to be searched. Decrease the importance of consecutive characters that occur more frequently. Select the character type that makes up consecutive characters for each consecutive character. There is a method of determining the importance of the consecutive characters according to the combination of the character types, and these methods may be used together.

The above method for calculating the importance is based on the fact that the more commonly appearing in each document to be searched, the lower the degree of specifying the desired document is. For example, various inflection endings, "the" and auxiliary verbs in English sentences, and particles and auxiliary verbs in Japanese sentences correspond to those that are likely to appear in common. On the other hand, the above method is based on the fact that kanji, hiragana, numbers, etc. are mixed in Japanese sentences, but the combination of kanji often has a characteristic meaning as a idiom. There is. Even in languages other than Japanese, for example, in Chinese, there is a distinction between a kanji that can be a particle and a kanji that is not a particle, and the character type can be determined according to this distinction. In addition, in English and the like, the character type can be determined depending on whether it is a capital letter or a small letter or a symbol such as a number, a Greek letter, or a hyphen. In English,
It is also possible to process the past type ending "ed" and the adverb ending "ly" as different character types.

In the present invention, a specific method of calculating the importance for each consecutive character is, for example, the language or field of the document to be searched (for example, technical literature, newspaper article,
Depending on the searcher's habit (what kind of search key is often used, etc.), the purpose of the search (whether to perform fuzzy search), and the search character string itself. It can be changed and can be changed adaptively. By changing the method of calculating the degree of importance as necessary, a more accurate search can be performed.

[0017]

Embodiments of the present invention will now be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of an information retrieval apparatus according to an embodiment of the present invention, and FIG. 2 is a flow chart showing a processing procedure when information is retrieved by the method of the present invention using this information retrieval apparatus. is there.

The information search device 11 is for executing a search for document information stored as search target documents in the database storage unit 10, and is a search character string for inputting a search character string designated by a searcher. An input unit 12, a consecutive-character extracting unit 13 that extracts consecutive characters having a predetermined character length from the input search character string, and the importance of the consecutive character for each consecutive character extracted from the search character string. A search engine unit 14 that calculates and actually searches each search target document in the database storage unit 10 based on these consecutive characters, and when calculating an importance degree for each consecutive character extracted from the search character string. Parameter storage unit 15 for storing parameters used for
And an importance storage unit 16 that stores the importance of each search target document, and an output unit that refers to the importance storage unit 16 and performs normalization according to the importance of each search target document and outputs a search result. 17 and 17. The set of consecutive characters extracted from the search character string is referred to as a search consecutive character group.

The search engine unit 14 calculates the importance of each consecutive character in the consecutive-characters group for search for the consecutive character. In the present embodiment, the first-type importance determined by the appearance frequency of consecutive characters. Then, two types of importance, that is, the second kind of importance determined from the character types forming the consecutive characters, are obtained for each consecutive character. The importances of the first type and the second type are 0 or positive real numbers, respectively, and are set such that the larger the value, the greater the contribution to the search result. Hereinafter, these importance levels will be described.

The importance of the first type is calculated by the search engine unit 14 by calculating the appearance frequency of the consecutive characters through all the search target documents stored in the database storage unit 10. The smaller the appearance frequency is, the larger the value is. The higher the appearance frequency, the smaller the value. Here, this appearance frequency is obtained by dividing the number of documents including the consecutive characters by the total number of documents, and is represented by a real number from 0 to 1. FIG. 2 is a graph in which the x-axis represents the frequency of appearance and the y-axis represents the value of the importance of the first type, showing an example of a function representing the relationship between the frequency of appearance and the importance of the first type. There is. As is clear from this graph, the importance of the first type is represented by a function that monotonically decreases with respect to the appearance frequency. Since consecutive characters appearing in all documents to be searched are meaningless for use in searching, it is preferable to set the importance of the first type to 0 for such consecutive characters. desirable.

If the character length of the consecutive characters is two, the second-class importance is determined by the search engine unit 14 according to the combination of the first and second character types forming the consecutive characters. It is determined. Specifically, the parameter storage unit 15 is provided with a calculation table 21 that represents the value of the second-type importance according to the combination of character types, and by referring to this calculation table 21, each connection Required for each character. FIG. 3 shows an example of the contents of the calculation table 21 when the search target character is a Japanese document. In the example shown in the figure, the character types are classified into hiragana, katakana, kanji, alphanumeric characters, and symbols, and when the consecutive characters are composed of kanji only, the importance of the second type becomes the largest value. . As can be seen from the combination of kanji and hiragana, when the character type of the first character and the character type of the second character are interchanged, the value of the same second type importance does not always result. When the first character is Kanji and the second character is Hiragana, it is overwhelming that the combination of the last character of the idiom (noun) + particle is overwhelming, and the order of the combination of character types is reversed. , Because it is considered less characteristic. The handling when the character length of consecutive characters is 3 characters or more is basically the same as that described here.

The search engine unit 14 is adapted to execute the actual search processing after determining the importance of the first type and the second type. As a search algorithm,
For example, there is one described in the above-mentioned JP-A-4-326164. The search processing determines, for each search target document, whether or not each consecutive character of the search consecutive characters group is included in the search subject document, and when the consecutive character is included, This is performed by adding the importance of the first type and the second type corresponding to the consecutive character to the importance of the search target document in the importance storage unit 16. In practice, the search processing may be executed collectively after calculating the first-type and second-type importances for all consecutive characters in the search consecutive-character group. Repeat one for each consecutive character by extracting one consecutive character, obtaining the importance of the first and second types for that consecutive character, and then checking whether that consecutive character is included in each search target document. You may make it execute.

An example of the structure of the importance storage unit 16 is shown in FIG. The importance storage unit 16 is the database storage unit 10.
It is configured as a table including the document number assigned to each search target document in the table and the importance of the search target document corresponding to the document number. The initial value of importance is 0. The degree of importance of each search target document corresponds to the degree of coincidence in the above-mentioned “conventional technology”.

Next, referring to FIG. 5, a procedure of information retrieval performed by using the information retrieval apparatus 10 based on the method of the present invention will be described.
To explain.

First, a search character string is input through the search character string input unit 12 (step 101), and the search character string is decomposed into consecutive characters having a predetermined character length by the consecutive character extracting unit 13 (step 102). . If the length of the consecutive characters is 2 characters and the search character string is, for example, "I will make a business trip to Osaka.", "Osaka", "Saka ni", "Ide", "Business trip", "Chang", "Yes" , And “ru.” Are extracted to form a search consecutive character group.

Next, the following processing is performed for each consecutive character in the retrieval consecutive character group. That is, it is determined whether or not unprocessed consecutive characters remain (step 103), and if there is remaining unprocessed consecutive characters, the unprocessed consecutive characters are searched for by the search engine unit 14 through all the search target documents. The appearance frequency is calculated (step 104), and the first importance for the consecutive character is calculated according to the calculated appearance frequency (step 1).
05), the degree of importance of the second type for the consecutive characters is determined according to the combination of the character types forming the consecutive characters (step 105). Next, it is checked whether or not there is an unsearched document for the consecutive character in the database storage unit 10 that is a search target document (step 107). Select one document,
It is determined whether the consecutive character is included in the document (step 108). If it is not included, the process directly returns to step 107. If it is included, the importance of the first type and the second type for the consecutive character is added to the importance of the document in the importance storage unit 16. (Step 1
09), and returns to step 107. If it is determined in step 107 that there is no unsearched document remaining, step 10 is performed to search for the next consecutive character included in the consecutive-character group for search.
Return to 3.

When two consecutive characters are extracted from the above-mentioned search character string "I will travel to Osaka.", The consecutive characters "Suru" and "Ru." Are likely to appear in each document. The importance of the first type becomes small. Further, when the calculation table as shown in FIG. 3 is used, the importance of the second type with respect to the consecutive character "Saka ni" becomes small. On the other hand, consecutive characters "Osaka"
Is composed of two Kanji characters, so the importance of the second type becomes large. If the frequency of appearance of “Osaka” through each search target document is low and the frequency of occurrence of “business trip” is high to some extent, the importance of the first type is “Osaka” is larger than “Business trip”. Become. After all, a search with a large degree of contribution of “Osaka” will be made overall.

If it is determined in step 103 that there are no unprocessed consecutive characters remaining, it means that the retrieval process based on all consecutive characters in the retrieval consecutive character group has been completed.
Then, the control is shifted to (3), and the importance value of each document stored in the importance storage unit 16 is normalized (step 110). Here, the normalization is a process of multiplying each importance by the same coefficient so that the maximum importance becomes 1. The longer the search string (search key), the greater the number of consecutive characters in the consecutive search character group, which tends to increase the raw importance of each document. However, by normalizing in this way, ,
Due to such a difference in the search character strings, it is possible to correct the variation in the importance value. Then, the normalized importances are arranged in descending order (step 111), and the importance of each document is output as a list to output the search result (step 112), and the process ends.

[0029]

As described above, according to the present invention, the consecutive characters are extracted from the search character string, the importance for each consecutive character is obtained, and the contribution ratio of each consecutive character to the search result is determined according to the importance. By changing it, the characteristic consecutive characters in the search are reflected more in the search result, and there is an effect that the search can be surely performed according to the intention of the searcher. As a method of determining the importance for consecutive characters, for each consecutive character used in the search, the frequency of occurrence of that consecutive character is calculated through all the documents to be searched, and the higher the occurrence frequency, the lower the importance. It is possible to perform a more reliable search by adopting a method in which the character type that forms a consecutive character is obtained for each character and the importance of the consecutive character is determined by the combination of the character types.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an information search device according to an embodiment of the present invention.

FIG. 2 is a graph showing an example of a function representing the relationship between the appearance frequency and the first type importance.

FIG. 3 is a diagram showing an example of a configuration of a calculation table showing correspondence between character types and second type importances.

FIG. 4 is a diagram showing a configuration of an importance degree storage unit.

5 is a flow chart showing a processing procedure when information is searched by the method of the present invention using the information search device of FIG. 1. FIG.

[Explanation of symbols]

 10 database storage unit 11 information retrieval device 12 search character string input unit 13 consecutive character extraction unit 14 search engine unit 15 parameter storage unit 16 importance storage unit 17 output unit 21 calculation table 101 to 112 steps

Claims (7)

[Claims] 1. An information retrieval method for retrieving a desired document from a set of retrieval target documents based on an inputted retrieval character string, wherein a consecutive character having a predetermined character length is extracted from the retrieval character string. A consecutive-character extracting step for extracting and forming a consecutive-character group for searching; an importance determining step for obtaining the degree of importance of the consecutive-character for each consecutive character included in the consecutive-character group for search; For each consecutive character included in the consecutive character group for search, it is checked whether the consecutive character is included in the search target document. If the consecutive character is included, the importance corresponding to the consecutive character is determined. An information search method comprising: a search step of adding the importance of the search target document; and an output step of outputting a search result according to the importance of each search target document after the search step is performed. 2. The importance determining step calculates, for each consecutive character included in the search consecutive character group, the appearance frequency of the consecutive character through all search target documents, and the smaller the appearance frequency is, the smaller the appearance frequency is. The information retrieval method according to claim 1, which is a step of determining the importance of the consecutive characters such that the value becomes a value and the value becomes large if the appearance frequency is small. 3. The importance determining step is a step of, for each consecutive character included in the search consecutive character group, determining the importance of the consecutive character according to a combination of character types constituting the consecutive character. The information search method according to claim 1. 4. An information retrieval method for retrieving a desired document from a set of retrieval target documents based on an inputted retrieval character string, wherein a consecutive character having a predetermined character length is extracted from the retrieval character string. A consecutive-character extracting step of extracting and forming a consecutive-letter group for searching, and calculating the appearance frequency of the consecutive-character through all the search target documents for each consecutive character included in the consecutive-letter group for searching. If the frequency is high, the value becomes small, and if the appearance frequency is low, the value becomes large.
An importance determining step of determining the importance of the type and determining the importance of the second type for the consecutive character according to the combination of the character types forming the consecutive character; For each consecutive character included in the consecutive character group, it is checked whether the consecutive character is included in the search target document, and if the consecutive character is included, the first and second types corresponding to the consecutive character An information retrieval method including a retrieval step of adding the degree of importance to the degree of importance of the retrieval target document, and an output step of outputting a retrieval result according to the degree of importance of each retrieval target document after the retrieval step is performed. . 5. An information retrieval device for retrieving a desired document from a set of documents to be retrieved based on an inputted retrieval character string, comprising an input means for inputting the retrieval character string, and a predetermined character. A consecutive-character extracting unit that extracts a consecutive consecutive character from the search character string to form a consecutive-character group for retrieval, an importance storage unit that stores the degree of importance of each document to be searched, and the consecutive-character for retrieval Whether the consecutive character is included in the search target document is determined for each consecutive character included in the group, and the degree of importance for the consecutive character is determined for each consecutive character included in the search consecutive character group. And if the consecutive character is included, refer to the searching means for adding the importance degree corresponding to the consecutive character to the importance degree of the search target document in the importance degree storing means, and the importance degree storing means. However, depending on the importance of each document to be searched, Information retrieval apparatus and an output means for outputting search results Te. 6. The information search device according to claim 5, wherein the importance of consecutive characters is determined according to a function that monotonically decreases with respect to the appearance frequency of the consecutive characters in all search target documents. 7. The degree of importance for a consecutive character is determined according to a combination of character types forming the consecutive character.
Information retrieval device described in.