JPH0954777A - Information retrieval device - Google Patents

Abstract

(57) [Abstract] [Purpose] The present invention relates to an information retrieval device used when retrieving a large amount of documents using an electronic computer. For a large amount of data, the amount of information that the data has is small and the capacity is small. It is an object of the present invention to provide an information retrieval device capable of performing high-speed retrieval by doing so and preventing search omission by full-text retrieval. [Structure] Similar to an unnecessary word deleting unit that deletes words that are not the search target from the search target data stored in the search target data storage unit, using the unnecessary word dictionary, deletes overlapping portions of character strings in the search target data. By providing the search processing means for performing full-text search on the compressed data created from the search target data by the duplicate character string deleting means, the search omission can be reduced and the search can be performed at high speed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information retrieving apparatus used when retrieving a large number of documents using an electronic computer.

[0002]

2. Description of the Related Art In recent years, as various documents have been digitized, the demand for searching a large number of documents has increased. In response to these requests, many conventional search devices extract a keyword from a document, add the keyword to the document and register it, and perform a method for the keyword when searching. It is adopted.

Hereinafter, such a conventional search device will be described with reference to FIG. In FIG. 24, reference numeral 2401 is a search target data storage unit, 2402 is a morpheme analysis processing unit, 2403 is a morpheme analysis dictionary, 2404 is word data, 2405 is a keyword extraction unit, 2406 is a keyword extraction dictionary, 2407 is keyword data, and 240.
Reference numeral 8 is search processing means, 2409 is input means, and 2410 is output means.

The operation of the search device configured as described above will be described below. First, when a registration start signal is input from the input means, the morphological analysis processing means 240
2 performs morpheme analysis processing on one piece of search target data stored in the search target data storage unit 2401 with reference to the morphological analysis dictionary 2403.
Create

When the morphological analysis process is completed, the keyword extracting means 2405 then operates the keyword extracting dictionary 240.
6 is used to create the keyword data 2407. After the series of operations described above are performed for all the search target data stored in the search target data storage unit 2401, when the search condition is input from the input unit 2409, the search processing unit for the keyword data 2407 is searched. 2408 performs a keyword search and outputs the search target data stored in the search target data storage unit 2401 from the matching result to the output means 2
Output to 410.

[0006]

However, in the configuration of the conventional search device described above, if the input search condition does not exist in the keyword data, no search is made and the keyword is extracted by morphological analysis. Whether it can be searched depends on whether the morphological analysis was performed correctly. In other words, if the morphological analysis is not performed correctly, the keywords will be missed, and this will cause missed searches.

In addition, there is a method of full-text search as a method for preventing the above-mentioned omission of search. However, since a character string not necessary for the search is searched, the search speed decreases as the amount of data increases. become.

There is a method using an index file to increase the search speed, but there is a problem that it takes a lot of time to create an index for a large amount of data.

The present invention solves the above-mentioned problems of the prior art. Using the full-text search that suppresses the omission of search, the capacity of the search target data is maintained without losing the information amount of the data for a large amount of data. It is an object of the present invention to provide an information search device that searches at high speed by reducing the size.

[0010]

In order to achieve this object, an information retrieval apparatus of the present invention comprises a retrieval target data storage section,
An unnecessary word dictionary that stores words that are not search targets, and an unnecessary word deleting unit that deletes words that are not search targets from the search target data stored in the search target data storage unit using the unnecessary word dictionary; Similarly, a duplicate character string deleting unit that deletes a duplicated portion of the character string in the search target data, a compressed data storage unit that stores the unnecessary word deleting unit and the compressed data created from the search target data by the duplicate deleting process, A search processing means for performing a full text search on the compressed data, an input means for inputting a registration start signal and a search condition, and an output means for outputting a search result are provided.

[0011]

The present invention eliminates morpheme analysis processing and eliminates unnecessary characters registered in the dictionary by unnecessary word deleting means and provides duplicate character string deleting means by providing duplicate character string deleting means. By deleting it, the data capacity is further reduced. Further, by providing a search processing means for searching the whole text of the data whose data volume is compressed in this way, search omission is prevented and a high-speed search is realized.

[0012]

【Example】

(First Embodiment) A first embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of an information retrieval apparatus according to an embodiment of the present invention. In FIG. 1, 1
01 is a search target data storage unit, 102 is an unnecessary word deleting unit, 103 is an unnecessary word dictionary, 104 is a duplicate character string deleting unit, 105 compressed data storage unit, 106 is a search processing unit,
107 is an input means and 108 is an output means.

First, the search conditions, search target data, unnecessary word dictionary, and compressed data in this embodiment will be described.

One search condition is a collation character string and sum, product,
Represented by logical symbols that represent logical relationships such as negation,
It is input by the input means 107.

The search target data in this embodiment is a document and is stored in the search target data storage unit 101.
One piece of document data is composed of a text, an image, or the like representing the content of the document. Further, one search target data has a header for data identification such as a search target data number, and a plurality of search target data exist in one file with the above-mentioned header as a delimiter.

The unnecessary word dictionary 103 stores unnecessary words. Unnecessary words here mean that in a specific document,
Words that have no meaning in themselves and are not searched
For example, it means a symbol, a tag, a suffix, a connecting word, a noun peculiar to the document, or the like. Further, the unnecessary word is a word that does not need to be updated.

The compressed data is the character string left between the unnecessary words, that is, the search target, which is obtained by removing the words that are not the search target as unnecessary words from the search target data. Duplicates of necessary words are removed, and the words are registered by separating them with a delimiter. As a delimiter, a 1-byte code that is not the search target is used.

One piece of compressed data is created from one piece of search object data, and the same header as the corresponding search object data is used as a header, and a plurality of pieces of compressed data are separated in one file by separating the aforementioned headers. Exists. FIG. 2 shows an example of the search target data, the unnecessary word dictionary 103, and the compressed data described above.

The operation of the information retrieving apparatus constructed as above will be described. The overall flow is shown in FIG.
The overall flow is roughly divided into data registration processing and search processing. The data registration processing is divided into unnecessary word deletion processing and duplicate character string deletion processing. The data registration process is performed for each search target file until all the search target data are completed.

First, the data registration process will be described. As the data registration processing, the unnecessary word deleting unit 102
The duplicated character string deleting means 104 creates compressed data to be actually searched from the searched data in advance.

First, when a data registration start command is issued from the input means 107, the unnecessary word deletion means 102 performs unnecessary word deletion processing on one piece of the search target data in the file stored in the search target data storage unit 101. To start. Here, the flow of the unnecessary word deletion processing is shown in FIG.

Further, as shown in FIG. 5, unnecessary word deleting means 1
02 is an unnecessary word calculation unit 102a and a necessary word calculation unit 102b
And a necessary word storage table 102c.

As the unnecessary word deletion processing, the unnecessary word position extraction unit 102a refers to the unnecessary word dictionary 103, compares the unnecessary words stored in the unnecessary word dictionary with the search target data, and searches the unnecessary data in the search target data. Extract the position of a word and its length. In this case, unnecessary words are extracted by the longest matching method.

FIG. 6 shows the operation flow of the longest match method. First, the search target data is compared with the unnecessary word dictionary 103 character by character from the beginning (step 1), and the characters are shifted by one character until a prefix-matching character is found (step 2) (step 3). When a prefix-matched character is found (step 2), it is combined with the next character (step 4) and it is checked again whether the prefix is matched (step 5). If there is no prefix match, the preceding character string becomes an unnecessary word and the start position and length of the character are obtained (step 6).

If there is a prefix match (step 5), the above operation is repeated until the prefix match is no longer met (step 7). If a perfect match is also established for the forward-matched word, the word is a word registered in the unnecessary word dictionary 103, but if the forward match is established again when combined with the next character, the A longer character string including the word is registered in the unnecessary word dictionary 103. A specific example is shown in FIG.

Subsequently, the necessary word calculation unit 102b calculates a character string other than the unnecessary words, that is, a necessary word to be searched from the obtained unnecessary word position and length, and the necessary word is stored in the necessary word storage table 102c. To store. An example is shown in FIG. This is the end of the unnecessary word deletion process.

Next, the flow of the duplicate character string deletion processing will be described with reference to FIG. As the duplicate character string deleting process, the duplicate character string deleting unit 104 refers to the necessary word position storage table 102c to check the duplication of each necessary word, and extracts the necessary non-overlapping words. At that time, partial matching of necessary words is recognized.

An example is shown in FIG. If the character string A and the character string B exist and the character string B is included in the character string A (denoted as A⊇B), the character string B is treated as a duplicate character string. For example, when a television is compared with a television, the television is included in the television, and thus a duplicate character string.

The header of the search target data to be processed is stored in the compressed data storage unit 105, and subsequently, the necessary non-overlapping words extracted by the above processing are separated and arranged by the above-described delimiter to create compressed data. And store.
When the duplication check / storage is completed for all the necessary words of one search target data, the duplication deletion process is ended. The data registration process ends when the duplicate character string deletion process ends for all the search target data.

Next, the search processing will be described. FIG. 11 shows a series of flows of search processing. When the search condition is input from the input unit 107 to all the compressed data created by the above operation, the search processing unit 106 performs a search. As a search method, a full-text search using the document itself as a search target is used. Specifically, a full text search is performed on the character string between the delimiter character of the compressed data and the next delimiter character. At this time, a method of directly searching data or a method of creating an index file and performing an index search can be used. When performing an index search, as one of the data registration processes described above, the process of creating an index is performed after creating the compressed data.

After performing the search process, the search processing means 106 extracts the header information of the collated data, and from the information of the header, the corresponding search is performed from the search target files stored in the search target data storage unit 101. The target data is extracted and sent to the output means 108 together with the header information for output.

Specifically, referring to FIG. 2, the search condition is "image data" and the search processing means 106 searches the compressed data. Then, since “image data” exists in the character string of “image data transmission” of document number 2, the search processing means 106 causes the document number 2 to be displayed.
The retrieval target data of the document number 2 stored in the retrieval target data storage unit 101 is taken out, and the header information of the document number 2 and the retrieval target data are sent to the output means 108 for output.

As described above, according to this embodiment, the unnecessary word deleting means 102 removes the character strings that are not the object of the search, and then the duplicate deleting means 104 deletes the plural character strings which are not necessary for the search. Since the amount of information can be compressed and the capacity can be reduced while keeping the amount of information as it is, the search can be performed with less search omission and can be performed at high speed. In addition, since the capacity of the search target data is small, the index file can be made small in the search system using the index file.

Further, since the full-text search is performed on the compressed data, it is possible to prevent the search omission like the conventional keyword search, and in addition, it is possible to perform the search with more free search conditions than the conventional exact match keyword search.

Although a plurality of search target data are stored in one file in the above, a plurality of files may be stored. Further, it is possible to store one search target data in one file by using the header of one search target data as a file name. In that case, the search processing means 10
The reference numeral 6 retrieves the file name of the collated data after performing the retrieval processing on the compressed data, retrieves the corresponding retrieval target data file from the file name, retrieves the retrieval target data stored therein, and retrieves the file. The name and the name are sent to the output means 108 for output.

Further, since the search target data storage section 101 can be provided on the optical disk, the storage space for the search target data can be reduced.

(Second Embodiment) A second embodiment of the present invention will be described below with reference to the drawings. FIG. 12 is a block diagram of an information search device according to an embodiment of the present invention. In FIG. 12, reference numeral 101 denotes a search target data storage unit, and 102.
Is an unnecessary word deleting unit, 103 is an unnecessary word dictionary, 104 is a duplicate character string deleting unit, 105 is a compressed data storage unit, 106 is a search processing unit, 107 is an input unit, 108 is an output unit, 1
201 is unnecessary word deleting means 102 and duplicate character string deleting means 1
A compression range storage unit for registering beforehand a range for compressing data according to 04.

The compression range storage unit 1201 of the information retrieval apparatus configured as above will be described. The compression range storage unit 1201 registers the compression range input from the input unit 107 in advance before starting the data registration process. As a compression range, a combination of a compression start position and a compression end position represented by an offset from the start position of the search target data,
Alternatively, specify tags for the compression start position and the compression end position. An example is shown in FIG.

Next, the flow of processing will be described. When an instruction to start data compression processing is input from the input unit 107, the unnecessary word deletion unit 103 causes the compression range storage unit 1 to operate.
Referring to 201, the compression start position and the compression end position in the search target data are obtained. Subsequently, unnecessary word deletion processing is performed on the range of the search target data corresponding to the position obtained as described above, and then the duplicate character string deletion unit 104 performs duplicate character string deletion processing. The structure of the search target data handled here, the unnecessary word deleting process, and the duplicate character string deleting process are the same as those in the first embodiment. Data to be searched that is not compressed subsequently,
That is, as shown in FIG. 13, the uncompressed data is stored in the compressed data as it is with the delimiter character string used in the duplication deletion process as a delimiter from the compressed data. After that, the flow of search processing is performed in the same manner as in the first embodiment.

When the compression range of the data to be searched can be designated as in the present embodiment, it is desired to compress only the text part of a document having a word enumeration part and a part written in the text. , Only the desired place can be targeted for compression, and more free search target data can be handled.

(Embodiment 3) A third embodiment of the present invention will be described below with reference to the drawings. FIG. 14 is a block diagram of an information retrieval apparatus according to an embodiment of the present invention. In FIG. 14, reference numeral 101 denotes a search target data storage unit, and 102.
Is an unnecessary word deleting unit, 103 is an unnecessary word dictionary, 104 is a duplicate character string deleting unit, 105 is a compressed data storage unit, 107 is an input unit, 108 is an outputting unit, 1401 is an unnecessary word deleting unit 102 and a duplicate character string deleting unit 104. A compression range plural storage unit 1402 capable of designating and registering a plurality of ranges for compressing data by means of a search processing means capable of performing a search for each compression / non-compression range.

The compression range plural storage section 1401 of the information retrieval apparatus configured as described above will be explained. The compression range storage unit 1401 stores the compression range input from the input unit 107 in advance before starting the data registration process. The compression range designation method is the same as in the second embodiment. However, it is different from the second embodiment in that a plurality of compression ranges can be designated. FIG. 15 shows an example of the compression range storage unit 1401.

Next, the overall flow will be described. When a command to start data compression processing is input from the input unit 107, the unnecessary word deletion unit 103 refers to the compression range multiple storage unit 1401 and sets the compression start position and the compression end position in the search target data to 1 Can be paired. Subsequently, unnecessary word deletion processing is performed on the range of the search target data corresponding to the position obtained as described above, and then the duplicate character string deletion unit 104 performs duplicate character string deletion processing. The structure of the search target data handled here, the unnecessary word deleting process, and the duplicate character string deleting process are the same as those in the first embodiment. When the duplicate character string deletion processing is completed for one compression range, a delimiter character that delimits a compression range different from the delimiter character used to delimit the necessary word is stored. As the delimiter of the compression range, a 1-byte code that is not the search target is used, as with the delimiter of the necessary word.

With the above, the data registration process for one compression range is completed. Subsequently, the search target data that is not compressed, that is, the non-compressed data is delimited by the delimiter character of the compression range and stored as it is. The above-described data registration processing for all the compression designated ranges and the storage processing of all the non-compressed data are repeatedly performed. An example of the compressed data created in this way is shown in FIG.

When the data registration process is completed, the search processing means 1402 starts the search process as in the case of the flow of the first embodiment. The search method is the same as in the first embodiment. However, the point that the search is performed within the range delimited by the delimiter of the compression range by the above-mentioned data registration processing is different from the first embodiment column. That is, the compression range delimiter becomes the search range delimiter in the search process. When the search condition is met in such one range, the matching process of the character string of one search target data ends immediately. The subsequent operation is similar to that of the first embodiment, and the header information of the collated data is taken out and the retrieval result is sent to the output means 108. The following is performed for all search target data.

In this way, the logical range used as the search condition can be effectively used by providing the search range and performing the search for each piece of data classified as a tagged block. For example, it is possible to perform a search with a search condition that is valid for adjacent characters.

An example will be described with reference to FIG. When the search intention is to know about "a search device that can handle image data online", the search condition is "online & image data & search device", and the document number 1 shown in the figure is "Claim 1" and "Bill". When one data consisting of "item 2" and "explanation of drawing" is set as the search target data, if the search range is not set, the document number 1 is hit as corresponding to the above search formula, but the range is set. If document number 1
Won't hit and you can get the intended result.

Of course, when the search target is the whole,
It can be used for various data to be searched because the compression range can be the whole.

Further, although unnecessary word deletion processing and duplicate character string deletion processing were performed for each compression range, unnecessary words were first deleted in the entire compression specified range, and then duplicate character strings were specified for each specified compression range. A deletion process can also be performed.

(Fourth Embodiment) A fourth embodiment of the present invention will be described below with reference to the drawings. FIG. 17 is a block diagram of an information search device according to an embodiment of the present invention. In FIG. 17, reference numeral 101 denotes a search target data storage unit, and 102.
Is an unnecessary word deleting unit, 103 is an unnecessary word dictionary, 104 is a duplicate character string deleting unit, 105 is a compressed data storage unit, 1401
Reference numeral 1701 denotes a compression range storage unit; 1701 is a search range designating unit for designating a range to be searched in the search target data;
Reference numeral 02 denotes a search processing means having a function of extracting a search range number from a search condition and sending it to the search range means 1701.
Reference numeral 7 is an input means, and 108 is an output means.

First, the search target data, the compressed data and the search conditions in this embodiment will be described with reference to FIG. The structure of the search target data is similar to that of the third embodiment. Each search target data has a structure for each block that is common to all search target data. For example, in FIG. 18, the purpose, the description of the figure, the purpose again, the description of the figure and the repeated tag are included, and the purpose and the description are set as one unit (one search target data).

The compressed data is the same as in the third embodiment, and is separated by a delimiter character that separates the compressed and non-compressed ranges.
Similar to the search target data, it has a structure common to each compressed data.

The search condition is the order of the data range from the beginning of one search target data that specifies the range to be searched along with the collation character string and the logical symbols indicating the logical relationship such as sum, product, negation. Attach the range number. In FIG. 18, one search target data is divided into two search ranges by one search range delimiter.

The flow of the information search apparatus having the search condition, the search target data, the search range specifying means 1701 and the search means 1702 as described above will be described. The entire process is divided into a data registration process and a search process, and the data registration process is the same as that of the third embodiment.

Next, the search processing performed by the search processing means 1702 will be described. When the search condition is input from the input unit 107, the search processing unit 1702
Obtains a search range number to be searched from the search condition and sends it to the search range specifying means 1701.

Subsequently, the search range designating means 1701 scans from the head of one compressed data, counts the delimiters of the compressed data and non-compressed data range, and the search processing means 1
The start position of the data range corresponding to the search range number sent from 702 is found. Then, the found start position is designated in the search processing means 1702. The search processing means 1702 performs the search processing from the designated start position to the end of the range. The search processing method is the same as in the first embodiment. The above search processing is performed for all compressed data.

According to this embodiment, search target data having the same structure can be searched by range, specifically, by item, and target data can be obtained. .

(Embodiment 5) Hereinafter, a fifth embodiment of the present invention will be described with reference to the drawings. FIG. 19 is a block diagram of an information search device according to an embodiment of the present invention. In FIG. 19, reference numeral 101 denotes a search target data storage unit, 103
Is an unnecessary word dictionary, 104 is a duplicate character string deleting unit, 105 is a compressed data storage unit, 107 is an input unit, 108 is an output unit, 1901 is an unnecessary word and the header information of the extraction target search target data is an unnecessary word described later. An unnecessary word deleting unit having a function of storing in a storage table, 1902 is an unnecessary word storage table for holding unnecessary words extracted by the unnecessary word deleting unit 1901 and header information of search target data of an extraction destination, and 1903 is the unnecessary word It is a search processing unit having a function of searching the storage table 1902.

The structure of the unnecessary word storage table 1902 of this embodiment will be described with reference to FIG. As shown in the figure, in the unnecessary word storage table 1902, the unnecessary word dictionary 10
A list of pairs of unnecessary words registered in No. 3 and header information of search target data in which the unnecessary words are deleted is stored.

The flow of the information search device having the unnecessary word storage table 1902, the unnecessary word deleting means 1901 and the search processing means 1903 will be described. As a whole, it is divided into data registration processing and search processing. Data registration process is third
As in the above embodiment, the unnecessary word deleting process and the duplicate character string deleting process are performed. However, the unnecessary word deleting unit 1901 stores the unnecessary word extracted when performing the unnecessary word deleting process and the header information of the search target data of the extraction destination in the unnecessary word storage table 1901.

For example, in FIG. 20, the character string "" stored in the unnecessary word dictionary is displayed.

[Purpose] "is an unnecessary word to be deleted from the search target data, and therefore is extracted from the document numbers 1 and 2 of the search target data by the unnecessary word deleting unit 1901. Next, the unnecessary word deleting unit 1901 stores the character string “in the unnecessary word storage table 1902.

[Purpose] "and document number 1 and document number 2 are stored as a pair. Hereinafter, similarly, a pair of an unnecessary word and header information is stored in the unnecessary word storage table 1902. The registration process of the data that performs the above operation for all the search target data ends.

Next, the flow of search processing will be described with reference to FIG. First, the search processing means 1903 is the input means 1
When the search condition is input from 07, the matching character string is extracted from the search condition. Next, the search processing means 1903 refers to the unnecessary word storage table to check whether the collation character string is stored. If it is stored, it is deleted from the search target data as an unnecessary word (that is, not stored in the compressed data). If it is not stored in the unnecessary word storage table 1902, it means that it is not deleted from the search target data.

The matching character string is the unnecessary word storage table 1
If it is not stored by referring to 902, the search processing means 1903 performs a search process on all the compressed data as in the first embodiment.

If it is stored, the retrieval processing means 190
3 obtains the header information of the search target data stored from the table. Next, the search target file is acquired from the header information. Subsequently, the search target file and its header information obtained by the above operation are output as a search result 1
Send to 08. Next, data other than the search target data sent to the output unit 108, that is, the output unit 108.
The search processing is performed for the compressed data that does not have the header information of the search target data sent to the. The search processing here is similar to that of the first embodiment.

According to the present invention, the unnecessary word deleting means 1901 is deleted even if the word is deleted from the search target data as an unnecessary word.
Since the delete record is held in the unnecessary word storage table 1902 and the search processing unit 1903 performs the search by referring to the delete record, it is possible to simultaneously perform the search with a small amount of search miss while compressing the search target data.

(Embodiment 6) A sixth embodiment of the present invention will be described below with reference to the drawings. FIG. 22 is a block diagram of an information search device in one embodiment of the present invention. In FIG. 22, reference numeral 101 denotes a search target data storage unit, 103
Is an unnecessary word dictionary, 104 is a duplicate character string deleting unit, 105 compressed data storage unit, 107 is an input unit, 108 is an output unit, 1901 is an unnecessary word deleting unit, 1902 is an unnecessary word storage table, 1903 is a search processing unit, 2201 Deletes the specified unnecessary word from the unnecessary word dictionary 103, obtains header information of the compressed data in which the unnecessary word is deleted by referring to the unnecessary word storage table 1902, and re-converts the unnecessary word into the corresponding compressed data. It is a means for reproducing data to be stored.

The data reproducing means 2201 of this embodiment will be described with reference to FIG. Data reproduction means 2201
When an unnecessary word to be stored is input and designated from the input means 107 to the unnecessary word dictionary 103, first, the designated unnecessary word is deleted from the unnecessary word dictionary 103. For example, in FIG. 23, “Koto” is a target to be deleted from the search target data (FIG. 23 (a)).

Next, referring to the unnecessary word storage table 1902, the header information of the compressed data in which the unnecessary word has been deleted is obtained (FIG. 23 (b)). Then, the corresponding compressed data is searched for from the obtained header information, and an unnecessary word is attached to the last part (FIG. 23 (c)). Next, the data reproducing means 22
When 01 has attached the designated unnecessary word to all the corresponding compressed data, the header information of the unnecessary word in the unnecessary word storage table 1902 and the search target data in which the unnecessary word is deleted is deleted (FIG. 23). (B)).

The above operation is started immediately when a command is input from the input means 107 when the data registration process and the search process are not performed, and is reflected in the next data registration process and the search process.

According to this embodiment, the data reproducing means 220
By providing 1, the unnecessary word dictionary as an unnecessary word 1
Registration can be canceled immediately even if registered in 03, and data can be reproduced by attaching specified unnecessary words to compressed data that has already been deleted as unnecessary words, making it easier to use information search The device can be realized.

[0071]

As described above, the information retrieval apparatus of the present invention is
By providing the unnecessary word deleting unit and the duplicate deleting unit that can reduce the capacity of the search target data without losing the information amount of the search target data, the search omission can be prevented and the search can be performed at high speed. Compared to the keyword search, the full-text search reduces the leakage of the keywords, which is easy to use, and the full-text search can be performed from the character string including the keyword. Therefore, the information retrieval device having the excellent search accuracy can be realized. .

Particularly, the larger the volume of data, the more the same character string to be searched appears, so the data volume can be reduced by the duplication deletion process, and the effect is great. Further, by reducing the capacity of the data to be searched, the number of indexes is reduced and the information retrieval apparatus is excellent in economic effect.

Further, if the compression range of the search target data can be specified, only the desired location in the document can be compressed, so that it is possible to select the search between the compressed data and the non-compressed data, and more free search target data is handled. be able to.

In addition, by performing a search for each data segmented as a tagged block, the logical expression used as a search condition can be effectively utilized.
You can also search for characters that are close together.

Besides. Search target data having the same structure can be searched for each range and each block, which makes it possible to obtain target data.

By holding a deletion record in which unnecessary words have been deleted from the search target data, a search can be performed by referring to the deletion record, and the search target data is compressed while the search is rarely missed. Can be realized at the same time.

Further, even if the word is once registered as an unnecessary word, the registration can be immediately deleted. In addition, the specified unnecessary word is attached to the compressed data already deleted as the unnecessary word to reproduce the data. It is possible to realize an information retrieval device that is easier to use.

[Brief description of drawings]

FIG. 1 is a configuration diagram of an information search device according to a first embodiment of the present invention.

FIG. 2 is a conceptual diagram of data in the first embodiment of the present invention.

FIG. 3 is a flowchart showing the overall operation in the first embodiment of the present invention.

FIG. 4 is a flowchart showing unnecessary word deletion processing in the first embodiment of the present invention.

FIG. 5 is a configuration diagram of unnecessary word deleting means in the first exemplary embodiment of the present invention.

FIG. 6 is a flow chart showing a longest matching method when extracting unnecessary words in the first embodiment of the present invention.

FIG. 7 is a conceptual diagram showing a longest matching method in the first embodiment of the present invention.

FIG. 8 is a conceptual diagram of necessary word calculation processing and a necessary word storage table according to the first embodiment of this invention.

FIG. 9 is a flowchart showing a duplicate character string deletion process according to the first embodiment of this invention.

FIG. 10 is a conceptual diagram showing a duplicate character string deletion process according to the first embodiment of this invention.

FIG. 11 is a flowchart showing search processing in the first embodiment of the present invention.

FIG. 12 is a configuration diagram of an information search device according to a second embodiment of the present invention.

FIG. 13 is a conceptual diagram of data in the second embodiment of the present invention.

FIG. 14 is a configuration diagram of an information search device according to a third embodiment of the present invention.

FIG. 15 is a conceptual diagram of data in the third embodiment of the present invention.

FIG. 16 is a conceptual diagram showing search conditions and search results in the third embodiment of the present invention.

FIG. 17 is a configuration diagram of an information search device according to a fourth embodiment of the present invention.

FIG. 18 is a conceptual diagram of data according to the fourth embodiment of the present invention.

FIG. 19 is a configuration diagram of an information search device according to a fifth embodiment of the present invention.

FIG. 20 is a conceptual diagram of data in the fifth embodiment of the present invention.

FIG. 21 is a flowchart showing search processing in the fifth embodiment of the present invention.

FIG. 22 is a configuration diagram of an information search device according to a sixth embodiment of the present invention.

FIG. 23 is a processing diagram of the data reproducing means in the sixth embodiment of the present invention.

FIG. 24 is a block diagram of a conventional information retrieval device.

[Explanation of symbols]

 101 search target data storage unit 102 unnecessary word deleting unit 102a unnecessary word position extracting unit 102b necessary word calculating unit 102c necessary word storage table 103 unnecessary word dictionary 104 duplicate character string deleting unit 105 compressed data storage unit 106 search processing unit 107 input unit 108 Output means 1201 Compressed range storage section 1401 Compressed range plural storage section 1402 Search processing means 1701 Search range designating means 1702 Search processing means 1901 Unnecessary word deleting means 1902 Unwanted word storage table 1903 Search processing means 2201 Data reproducing means 2401 Search target data storage section 2402 Morphological analysis processing means 2403 Morphological analysis dictionary 2404 Word data 2405 Keyword extraction means 2406 Keyword extraction dictionary 2407 Keyword data 2408 Search processing means 24 9 input means 2410 output means

Claims (11)

[Claims] 1. A search target data storage unit for storing search target data, and a compressed data storage unit for storing data obtained by deleting a duplicated portion of a character string from the search target data stored in the search target data storage unit. An input unit for inputting search conditions, a search unit for searching the compressed data storage unit for data matching the search conditions input from the input unit, and the search unit stored in the search target data storage unit. An information search device comprising display means for displaying search target data corresponding to a search result. 2. A retrieval target data storage unit for storing retrieval target data, and compressed data for storing data obtained by deleting a character string designated as an unnecessary word from the retrieval target data stored in the retrieval target data storage unit. A storage unit; an input unit for inputting search conditions; a search unit for searching the compressed data storage unit for data matching the search conditions input from the input unit; and a unit for storing the search target data storage unit. An information search device comprising display means for displaying search target data corresponding to a search result of the search means. 3. A search target data storage unit that stores search target data, an unnecessary word dictionary that stores words that are not to be searched, and the unnecessary word from the search target data stored in the search target data storage unit. By the unnecessary word deleting means for deleting words that are not to be searched using the dictionary, the duplicate character string deleting means for deleting the overlapping portion of the character string in the search target data, the unnecessary word deleting means and the duplicate deleting processing A compressed data storage unit that stores compressed data created from the search target data; an input unit that inputs search conditions;
An information retrieval device comprising a retrieval processing means for performing a full text retrieval on the compressed data according to the retrieval conditions and an output means for outputting a retrieval result of the retrieval processing means. 4. The information retrieving apparatus according to claim 3, wherein the unnecessary word deleting means uses the longest matching method for detecting the longest matching character string according to the prefix matching condition as the unnecessary word extracting method. 5. The information retrieving apparatus according to claim 3, wherein the duplicate character string deleting means uses a method of detecting a duplicate character string by prefix matching, intermediate matching, or perfect matching as the duplicate character string deleting method. 6. The information retrieving apparatus according to claim 3, wherein the compressed data stored in the compressed data storage section is delimited by a plurality of types of delimiters that change the range of the retrieval target. 7. The information retrieval device according to claim 3, further comprising a compression range storage section for storing a range in which the unnecessary word deleting means and the duplicate character string deleting means compress data. 8. The information search device according to claim 3, wherein the compression range storage unit stores a plurality of compression ranges, and the search processing unit searches for each of the compression ranges. 9. The information search apparatus according to claim 8, wherein the search processing means is provided with a function of performing a search by specifying a compression range or search target data outside the compression range. 10. An unnecessary word storage table is provided for storing unnecessary words deleted by the unnecessary word deleting means from the search target data using the unnecessary word dictionary, and the unnecessary word storage table is used as the search target of the search processing means. The information search device according to claim 3, wherein 11. The unnecessary word registration is canceled by canceling unnecessary word registration from the unnecessary word dictionary, and checking the unnecessary word storage table in which the unnecessary word deleting means stores the unnecessary words deleted from the search target data by using the unnecessary word dictionary. 11. The information retrieving apparatus according to claim 10, further comprising a data reproducing unit that adds the unnecessary word canceled to the compressed data and cancels the unnecessary word registration from the unnecessary word storage table.