JP5488424B2 - Information processing apparatus, control method therefor, and program - Google Patents

Description

  The present invention relates to an information processing apparatus, a control method therefor, and a program, and more particularly, to a technique for detecting an electronic document including a character string that approximates a character string in an input electronic document from registered electronic documents. .

  With the spread of computers and the Internet, opportunities to handle electronic documents are increasing day by day. However, since electronic documents can be easily copied, various problems such as electronic information leakage and copyright infringement have occurred.

  With regard to information leakage, products such as DLP (Data Leak Prevention) have recently attracted attention. DLP is an information leakage countermeasure product installed on a network or on a client PC, and can control transmission of confidential documents to the outside based on conditions that characterize the confidential documents set in advance.

  Previously, conditions were set according to the sender / receiver and keywords in the text and complicated work was required, but recently, confidential documents themselves are registered, and registered confidential documents or their derivative documents are sent. Products with a function to control In addition, when creating reports at universities and the like, it is a problem to create a report by partially copying and combining them from documents on the Internet, and products that detect these have also appeared.

  In order to realize such a product, not only a document with exactly the same content as a specific document, but also a technique for finding a document that approximates a query document derived by partial duplication or small-scale editing. Is needed.

  Conventionally, as a technique for searching for a document similar to a specified document, a similar sentence search for calculating a similarity based on a matching degree of a component of a document, or a direct comparison of character strings constituting two documents is used. There is DP matching that can determine the approximation of.

  Patent Document 1 and Japanese Patent Application Laid-Open Publication No. 2005-259542 disclose a method of performing a search according to the degree of matching with a search word using a transposed index with respect to the appearance position of a concatenated character.

  Also, in Patent Document 3, a method of searching for a document having a high degree of coincidence with a query document by expressing the constituent elements of the document as a hash value and expressing the contracted character string (fingerprint) and comparing the fingerprints Is disclosed.

Japanese Patent Laid-Open No. 08-235212 JP 09-259140 A Special table 2008-541272

  However, in the conventional similar sentence search, the similarity is basically calculated based on the degree of coincidence of the components of the document, the order and positional relationship of the components are not considered, and if similar components are included, There is a problem in that even documents with greatly different descriptions are detected.

  When DP matching or the like is used, the degree of approximation of documents can be obtained by direct comparison of character strings constituting two documents. However, when the number of documents to be detected increases, the number of comparisons increases in proportion to the number of documents. There is a problem that the processing time increases.

  Furthermore, the methods disclosed in Patent Document 1 and Patent Document 2 are for short search words, and all connected character strings constituting the search words are evaluated, so the processing amount increases as the search words become longer. However, there is a problem that the time for the search process becomes long.

  In Patent Document 3, only characteristic components are targeted when fingerprints are generated, and the speed is increased by reducing the number of comparison points. However, the selection criteria for comparison points are statistics for each document. Since the standard is different between the registered document and the inquiry document (a document including a character string serving as a search word) based on the value, there is a possibility that different components are compared. In particular, the problem becomes significant when a document including only a part of a document to be searched is used as an inquiry document.

  The present invention has been made in order to solve the above-described problem, and according to the number of partial character strings in the registered document having the same positional relationship as the partial character strings constituting the query document, An object of the present invention is to provide a mechanism for determining a registered document to be approximated with high accuracy by calculating the degree of approximation.

The present invention provides a partial character string obtained by decomposing a sentence included in a registered document indicating a document to be approximated to an inquiry document indicating a designated document, a position of the partial character string in the registered document, An information processing apparatus for determining the registered document that approximates the designated query document, wherein the partial character string is the same as the partial character string obtained by decomposing the query document. A partial character string acquisition unit for acquiring a partial character string included in the registered document stored in the storage unit, a position of the partial character string with respect to the inquiry document, and a registration document acquired by the partial character string acquisition unit obtained from the position of the partial character string, the portion having a partial string to substrings with the registered document to the inquiry documents, the same positional relationship A calculating unit that calculates the degree of approximation between the registered document and the query document using the number of character strings; and a determining unit that determines a registered document that approximates the query document according to a calculation result by the calculating unit. It is characterized by that.

The present invention also provides a partial character string obtained by decomposing a sentence included in a registered document indicating a document to be approximated by an inquiry document indicating a specified document, and the position of the partial character string in the registered document. And an information processing apparatus control method for determining the registered document that approximates the designated inquiry document, wherein the partial character string acquisition means of the information processing apparatus includes the inquiry document. A partial character string that is the same as the partial character string obtained by decomposing the character string, and that includes the partial character string included in the registered document stored in the storage unit; and the calculation of the information processing apparatus The query statement is obtained from a position of the partial character string with respect to the query document and a position of the partial character string with respect to the registered document acquired by the partial character string acquisition step. A calculation step of using the number of the partial strings having a partial string to substrings with the registered document, the same positional relationship, to calculate the approximate degree of the registered document and the query document for the information The determination unit of the processing device includes a determination step of determining a registered document that approximates the inquiry document according to the calculation result of the calculation step.

The present invention also provides a partial character string obtained by decomposing a sentence included in a registered document indicating a document to be approximated by an inquiry document indicating a specified document, and the position of the partial character string in the registered document. A program that can be read and executed by an information processing apparatus that determines the registered document that approximates the designated query document, and the information processing apparatus disassembles the query document. A partial character string that is the same as the partial character string obtained by the above, and a partial character string acquisition unit that acquires a partial character string included in the registered document stored in the storage unit; and a position of the partial character string with respect to the query document And the partial character string for the query document and the registration sentence obtained from the position of the partial character string for the registered document acquired by the partial character string acquisition means By using the number of the partial strings having a partial character string, the same positional relationship with respect to, and calculating means for calculating an approximate degree between the query document and the registration document, in accordance with the calculated result of said calculation means, query documents It is made to function as a determination means which determines the registration document approximated to.

  According to the present invention, it is possible to provide an approximate document search apparatus, a control method therefor, and a program that can realize a search for an approximate document at high speed. In particular, since effective components for discrimination of registered documents are selected from the components of the query document based on the statistical values in the entire registered document set, it is also effective for searching when the query document contains only a part of the registered document. Play.

It is a figure which shows the structure of the approximate document search apparatus in embodiment of this invention. It is a figure which shows the hardware constitutions of the various terminals in embodiment of this invention. It is a figure which shows the basic processing flow of the document registration part in the approximate document search apparatus in embodiment of this invention. It is a figure which shows the structure of the registration document preservation | save area | region in embodiment of this invention. It is a figure which shows an example of the bibliographic information table in embodiment of this invention. It is a figure which shows an example of the transposition index in embodiment of this invention. It is a figure which shows an example of the document registration process in embodiment of this invention. It is a figure which shows the basic processing flow of the approximate search process part in embodiment of this invention. It is a figure which shows the basic processing flow of the statistical information acquisition process of the approximate search process part in embodiment of this invention. It is a figure which shows an example of the type | formula which calculates | requires the appearance probability with respect to the registration document set of the connection character string in embodiment of this invention. It is a figure which shows an example of the type | formula which calculates | requires the information content with respect to the registration document set of the connection character string in embodiment of this invention. It is a figure which shows an example of the type | formula which calculates | requires the information entropy with respect to the registration document set of the specific character in embodiment of this invention. It is a figure which shows an example which calculates | requires a statistics value from the information contained in the transposition index in embodiment of this invention. It is a figure which shows the basic processing flow of the evaluation connection character string selection process of the approximate search process part in embodiment of this invention. It is a figure which shows an example of the evaluation connection character string selection process in embodiment of this invention. It is a figure which shows an example of correction | amendment of the positional information in the evaluation concatenated character string selection process in embodiment of this invention, and an aggregation in correction | amendment appearance position. It is a figure which shows the basic processing flow of the approximation calculation process of the approximate search process part in embodiment of this invention. It is a figure which shows an example of the type | formula which calculates | requires the approximation degree in embodiment of this invention. It is a figure which shows an example of the approximation calculation process in embodiment of this invention. It is a figure which shows the basic processing flow of the approximation calculation process of the approximate search process part in the 2nd Embodiment of this invention. It is a figure which shows an example of the type | formula which calculates | requires the approximation degree in 2nd Embodiment of this invention. It is a figure which shows an example of the approximation calculation process in the 2nd Embodiment of this invention. It is a figure which shows the flow of the process which specifies the tolerance | permissible area in the 2nd Embodiment of this invention. It is a figure which shows the basic processing flow of the approximation calculation process of the approximate search process part in the 3rd Embodiment of this invention. It is a figure which shows an example of the approximation calculation process in the 3rd Embodiment of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a diagram showing a configuration of an approximate document search apparatus according to an embodiment of the present invention.
Note that the configuration of the approximate document search device in FIG. 1 is an example, and it goes without saying that there are various configuration examples depending on the application and purpose.
Reference numeral 100 denotes an approximate document search apparatus. The approximate document search apparatus 100 includes a document registration unit 101, a registered document information storage area 102, and an approximate document search unit 103.

  The document registration unit 101 decomposes the character string constituting the registered document 110 input to the approximate document search apparatus 100 into partial character strings, and displays all the partial character strings and the appearance positions together with the bibliographic information of the registered document. It is added to the information storage area 102. That is, the partial character string is obtained by decomposing a sentence included in the registered document.

  The approximate document search unit 103 decomposes the character string constituting the query document 111 input to the approximate document search apparatus 100 into partial character strings, refers to the registered document information storage area 102, and registers all partial character strings. Statistical information (including information amount and information entropy described later) in the document and appearance position information (appearance position) in the registered document set are acquired, and a document that approximates the query document 111 is acquired based on the acquired statistical information and appearance position information. A search is made from the document set registered in the registered document information storage area 102, and a list of approximate documents is output as an approximate search result 112 according to the degree of approximation.

  Next, the hardware configuration of the approximate document search apparatus 100 in FIG. 1 will be described with reference to FIG.

  FIG. 2 is a diagram illustrating a hardware configuration of various terminals according to the embodiment of the present invention.

  The CPU 201 comprehensively controls each device and controller connected to the system bus 204.

  Further, the ROM 202 or the external memory 211 is necessary to realize a BIOS (Basic Input / Output System) or an operating system program (hereinafter referred to as an OS), which is a control program of the CPU 201, or a function executed by each server or each PC. Various programs to be described later are stored. The RAM 203 functions as a main memory, work area, and the like for the CPU 201.

  The CPU 201 implements various operations by loading a program necessary for execution of processing into the RAM 203 and executing the program.

  An input controller (input C) 205 controls input from a pointing device such as a keyboard 209 or a mouse (not shown).

  A video controller (VC) 206 controls display on a display device such as a CRT display (CRT) 210. The display device may be a liquid crystal display as well as a CRT. These are used by the administrator as needed. The present invention is not directly related.

  A memory controller (MC) 207 is a hard disk (HD), floppy disk (registered trademark FD) or PCMCIA card slot for storing boot programs, browser software, various applications, font data, user files, editing files, various data, and the like. Controls access to an external memory 211 such as a compact flash memory connected via an adapter.

  A communication I / F controller (communication I / FC) 208 is connected to and communicates with an external device via a network, and executes communication control processing in the network. For example, Internet communication using TCP / IP is possible.

  Note that the CPU 201 enables display on the CRT 210 by executing outline font rasterization processing on a display information area in the RAM 203, for example. In addition, the CPU 201 enables a user instruction with a mouse cursor (not shown) on the CRT 210.

  An approximate document search program for realizing the present invention is recorded in the external memory 211 and is executed by the CPU 201 by being loaded into the RAM 203 as necessary. Further, the definition file 213 and various information tables 214 used by the program according to the present invention are stored in the external memory 211, and detailed description thereof will be described later.

(Document registration process)
Next, a basic processing flow of the document registration unit 101 in the approximate document search apparatus 100 will be described with reference to FIG.

  FIG. 3 is a diagram showing a processing flow in the document registration processing of the document registration unit.

  In step S <b> 301, the document registration unit 101 registers bibliographic information of a document to be registered in the bibliographic information table 401. At this time, the document ID is acquired as an identifier for uniquely identifying the registered document.

In step S302, the document registration unit 101 divides the character string constituting the registered document into two concatenated character strings, and synthesizes the document ID that is the document identifier and the appearance position of the two concatenated character strings with the two concatenated character strings. Generate location information.
In step S303, the document registration unit 101 starts repetitive processing for the two-character string acquired in step S302.

  In step S <b> 304, the document registration unit 101 adds a two-character string and position information to the transposed index 402. Simultaneously with the registration, the appearance frequency of the two concatenated character strings stored in the transposed index 402 is added by one.

  FIG. 4 is a conceptual diagram of the registered document information storage area 102. The registered document information storage area 102 includes a bibliographic information table 401 and a transposed index 402.

  In step S305, the document registration unit 101 returns the process to step S303 if there is still a two-character string to be processed, and ends the process if there is no two-character string to be processed.

(Specific example of processing here)
Next, a case where a document “Osaka weather July 10th. Doc” consisting of a single sentence “Today is a sunny day but the wave height of Osaka Bay” will be described in detail.

  In step S301, the document registration unit 101 registers “Osaka weather July 10th. Doc” as the bibliographic information of the document to be registered in the bibliographic information table 401 as shown in FIG. At this time, the document ID “15” given as an identifier for uniquely identifying the registered document is acquired.

  In step S <b> 302, the document registration unit 101 divides the character string “Today is a sunny day but the height of Osaka Bay” constituting the registered document into two consecutive character strings, and is a document identifier for the two consecutive character strings. Position information is generated by combining the document ID and the appearance position of the two concatenated character strings. For example, for the first two-character string “today”, position information “15: 1” is generated from the document ID “15” and the appearance position “1”, and the next two-character string “day is”. On the other hand, the position information “15: 2” is generated from the document ID “15” and the appearance position “2”.

  In step S <b> 303, the document registration unit 101 starts repetitive processing from the first two concatenated character strings “today”.

  In step S304, the document registration unit 101 adds the two concatenated character string “today” and the position information “15: 1” to the transposed index 402 as shown in FIG. In the transposed index 402, since the appearance frequency “17” and the position information “... 13:14 14:32” are registered for the two-character string “today”, the position information “15: 1” is set. In addition, “... 13:14 14:32 15: 1” is added, and the appearance frequency is added by 1 to “18”.

  In step S305, the document registration unit 101 returns the process to step S303 because there is the next two concatenated character string “day is”.

  Similarly, when the process is repeated for all the two concatenated character strings, as shown in FIG. 7, the transposed index 402 for any two concatenated character strings that appear in “Osaka Weather July 10th. It is possible to acquire the appearance position from

(Approximate document search process)
Next, a basic processing flow of the approximate document search unit 103 in the approximate document search apparatus 100 will be described with reference to FIG.

  FIG. 8 is a diagram showing a schematic flow in the approximate document search process of the approximate document search unit 103.

  In step S <b> 801, the approximate document search unit 103 acquires a two-character string that forms the query document and a statistical value in the registered document set of the query document.

  In step S802, the approximate document search unit 103 determines the document discrimination contribution degree of the two concatenated character strings based on the two concatenated character strings and the statistical information of the entire query document acquired in S801 and a predetermined selection criterion. A document with a high contribution degree of document discrimination is selected as an evaluation concatenated character string. For the evaluation concatenated character string, position information is acquired from the transposed index 402, and the evaluation concatenated character string is corrected by the appearance position in the query document. The appearance position corrected here is referred to as corrected position information.

  In step S803, the approximate document search unit 103 aggregates for each corrected position information of the evaluation concatenated character string acquired in step S802, calculates an approximate degree for each document including the corrected position information, and approximates based on the calculated approximate degree. Identify the document to be used.

In the present embodiment, the information amount of the connected character string is used as the contribution degree of document discrimination. In addition, information entropy indicating the degree of randomness in the registered document set of the character set constituting the query document is used as a selection criterion. It is considered that the information necessary for the judgment is obtained when the information amount for the connected character string to be evaluated is accumulated in descending order and the accumulated value of the information amount exceeds the information entropy (degree of randomness) as a selection criterion.
In this way, it is possible to reduce processing for a component having a low discrimination effect by not evaluating a connected character string having a small degree of contribution to document discrimination from the registered document set.
Details of steps S801 to S803 will be described later.
(Statistical information acquisition processing)
Next, a detailed flow of step S801 in the approximate document search process of the embodiment will be described with reference to FIG.
FIG. 9 is a diagram showing a detailed flow of statistical information acquisition processing in approximate document search processing.
In step S <b> 901, the approximate document search unit 103 decomposes the character string that forms the query document into a two-character string.

  In step S902, the approximate document search unit 103 sets the reference value of the statistical information of the query document stored in the temporary area to 0.

  In step S903, the approximate document search unit 103 starts the iterative process for the two concatenated character strings divided in step S901.

  In step S904, the approximate document search unit 103 acquires the total number of all the connected character strings registered and the appearance frequency for the two connected character strings being processed from the transposed index 402, and uses the formula shown in FIG. Then, the appearance probability P (W) in the registered document set of the two concatenated character strings is calculated, and the information amount I (W) is calculated using the formula shown in FIG. Also, the information entropy E (C) is calculated for the first character of the two-character string using the formula shown in FIG. 12, and when the two-character string is the end, the information entropy is calculated for all characters. To do.

  In step S905, the approximate document search unit 103 adds the information entropy acquired in step S904 to the reference value in the temporary area.

  In step S906, the approximate document search unit 103 returns the process to step S903 if there is still a two-character string to be processed. If there is no two concatenated character string to be processed, the process is terminated.

(Evaluation concatenated character string selection process)
Next, the detailed flow of the process of step S802 in the approximate document search process of the embodiment will be described with reference to FIG.

  FIG. 14 is a diagram showing a detailed flow of the evaluation concatenated character string selection process in the approximate document search process.

  In step S1401, the approximate document search unit 103 secures and initializes an area for storing the appearance position buffer in the memory.

  In step S1402, the approximate document search unit 103 rearranges the concatenated character string and statistical information set, which are acquired in step S801, constituting the query document in descending order of document discrimination contribution degree (concatenated character string information amount). .

  In step S1403, the approximate document search unit 103 starts an iterative process for the rearranged connected character strings.

  In step S <b> 1404, the approximate document search unit 103 acquires the appearance position of the connected character string in the registered document set from the transposed index 402.

  In step S <b> 1405, the approximate document search unit 103 obtains, as a corrected appearance position, a value obtained by subtracting the position where the connected character string appears in the query document from all the obtained appearance positions.

  The corrected appearance position is equivalent to obtaining the head position on the registered document of the query document when it is assumed that the query document appears in the registered document, and the concatenated character string having the same corrected appearance position is the same on the registered document. This indicates that there is a possibility that the query document is configured. The more connected character strings having the same corrected appearance position, the higher the possibility that the query document is configured.

  In step S1406, the approximate document search unit 103 adds the corrected appearance position acquired in step S1405 to the appearance position buffer. When the appearance position information having the same corrected appearance position is not registered in the appearance position buffer, it is registered as the appearance position information together with the appearance position in the inquiry document with a match number of 1. When appearance position information having the same corrected appearance position is already registered in the appearance position buffer, the number of matches of the registered appearance position information is incremented by 1, and the appearance position in the query document of the concatenated character string is already registered If it is smaller than the appearance position in the query document, the appearance position in the query document of the registered appearance position information is updated with the appearance position in the query document of the connected character string being processed.

  In step S1407, the approximate document search unit 103 determines whether the integrated value of the information amount of the processed connected character string exceeds the information entropy that is the selection criterion acquired in step S801. If the integrated value of the information amount does not exceed the information entropy, the process proceeds to step S1408. If the integrated value of the information amount exceeds the information entropy, the process ends.

  In step S1408, the approximate document search unit 103 returns the process to step S1403 if there is a connected character string to be processed. If there is no connected character string to be processed, the process ends.

(Approximation degree calculation process)
Next, a detailed flow of the process of step S803 in the approximate document search process of the embodiment will be described with reference to FIG.

  FIG. 17 is a diagram showing a detailed flow of the approximation calculation process in the approximate document search process.

  In step S1701, the approximate document search unit 103 secures and initializes the result buffer area in the memory.

  In step S1702, the approximate document search unit 103 secures and initializes a document information buffer area in the memory.

  In step S1703, the approximate document search unit 103 starts the iterative process for the appearance position information stored in the appearance position buffer in step S802.

  In step S1704, the approximate document search unit 103 determines whether the number of matches of the corrected appearance positions of the appearance position information is equal to or greater than a specified value. If the number of matches is greater than or equal to the specified value, the process proceeds to step S1705. If the number of matches is less than the specified value, the process proceeds to step S1706.

  In step S1705, the approximate document search unit 103 adds appearance position information to the document information buffer.

  In step S1706, if the document information buffer is empty, the approximate document search unit 103 moves the process to step S1712. If the document information buffer is not empty, the process proceeds to step S1707.

  In step S1707, the approximate document search unit 103 determines whether the appearance position information currently being processed is the end in the processing order, or whether the next appearance position information and the appearance position information currently being processed have different document IDs. Determine.

  If the appearance position information currently being processed is at the end, or if the next occurrence position information has a different document ID, the process moves to step S1708. If the appearance position information currently being processed is not the end, and the next appearance position information has the same document ID, the process proceeds to step S1712.

  In step S1708, the approximate document search unit 103 calculates the degree of approximation using the expression shown in FIG. 18 from the appearance position information registered in the document information buffer. The calculation method of the degree of approximation is an example, and other calculation methods may be used.

  In step S1709, the approximate document search unit 103 determines whether the obtained degree of approximation is equal to or greater than a specified value. If the degree of approximation is greater than or equal to the specified value, the process proceeds to step S1710. If the degree of approximation is less than the specified value, the process proceeds to step S1711.

  In step S1710, the approximate document search unit 103 associates the document ID with the degree of approximation (calculation result) and registers it in the result buffer.

  In step S1711, the approximate document search unit 103 initializes the document information buffer.

  In step S1712, if there is appearance position information that should still be processed, the approximate document search unit 103 returns the process to step S1703. If there is no appearance position information to be processed, the process proceeds to step S1713.

  In step S1713, the approximate document search unit 103 rearranges the result buffers in descending order of approximation.

  In step S1714, the approximate document search unit 103 stores the contents of the result buffer in the approximate search result 112 and ends the process.

(Specific example of processing here)
Next, the case where an approximate sentence search is performed by an inquiry document “Naturally sunny sky but the height of Osaka Bay” will be described in detail.

  The approximate document search unit 103 starts the statistical information acquisition process shown in FIG. 9 for the inquiry document when the inquiry document “Today's fine weather is in Osaka Bay” is input.

  In step S <b> 901, when the query document “Today's fine weather is in Osaka Bay” is input, the approximate document search unit 103 disassembles the character string that forms the query document, and displays “Today” and “Nichiharu”. , “Sunny sky”, “heavenly”,...

  In step S902, the approximate document search unit 103 sets the reference value of the statistical information of the query document secured in the temporary area to zero.

  In step S903, the approximate document search unit 103 starts processing for the first two-character string “today” divided in step S901. At this time, it is assumed that the transposition index 402 stores the contents as shown in FIG. 13 and the total number of registered concatenated character strings is “5000”.

  In step S904, the approximate document search unit 103 obtains the appearance frequency “18” from the transposed index 402 for the two-character string “today”. Since the total number of concatenated character strings is “5000”, the occurrence probability of “today” in all registered documents of two concatenated character strings is obtained using the formula shown in FIG. 10, and P (today) = 18/5000 = 0. Get 0036. Furthermore, using the equation shown in FIG. 11, the information amount I (today) = 8.1178 of the two-character string “today” is obtained.

  In the transposed index 402 of FIG. 13, the sum of the appearance frequencies of the two concatenated character strings starting with “book” is 284, and the approximate document search unit 103 uses the formula shown in FIG. Information entropy E (book) = 2.629217.

  In step S905, the approximate document search unit 103 adds the “book” information entropy E (book) = 2.629217 acquired in step S904 to the selection reference value stored in the temporary area.

  In step S906, the approximate document search unit 103 returns the process to step S903 because there is a two-character string “Hibara” to be processed next.

  When the approximate document search unit 103 proceeds in the same manner and repeats the process up to the last two consecutive character strings “Shi”, the appearance frequency shown in FIG. 15 is obtained for all the two consecutive character strings constituting the query document. Get a list of appearance probabilities and information volumes. Moreover, 42.3371 which is the total value of the character information entropy is obtained as a reference value for selection, and the statistical information acquisition process of FIG. 9 is terminated.

  Next, the approximate document search unit 103 uses the evaluation concatenated character string shown in FIG. 14 for the appearance frequency, appearance probability, information amount list and selection reference value 42.3371 shown in FIG. 15 acquired in the statistical information acquisition process. Perform the selection process.

  In step S1401, the approximate document search unit 103 secures the appearance position buffer in the memory and empties it.

  In step S1402, the approximate document search unit 103 rearranges the list of appearance frequencies, appearance probabilities, and information amounts shown in FIG. 15 in descending order of information amount.

  In step S <b> 1403, the approximate document search unit 103 starts processing for the connected character string “Hanwan” that is the head as a result of the rearrangement.

  In step S <b> 1404, the approximate document search unit 103 acquires “... 13:23 14:43 15:10” as the appearance position in the registered document set of the concatenated character string “Sakawan” from the transposed index 402.

  In step S1405, the approximate document search unit 103 subtracts the position “9” where the concatenated character string appears in the query document from the acquired appearance position “... 13:23 14:43 15:10” as shown in FIG. Then, “... 13:14 14:34 15: 1” is acquired as the corrected appearance position.

  In step S1406, the approximate document search unit 103 adds the corrected appearance position “... 13:14 14:34 15: 1” acquired in step S1405 to the appearance position buffer. The appearance position “13:14” is registered in the appearance position buffer as the appearance position information together with the inquiry document appearance position “9” and the number of matches “1”. Similarly, for the appearance positions “14:34” and “15: 1”, the appearance position information is registered together with the inquiry document appearance position “9” and the number of matches “1”.

  In step S1407, the approximate document search unit 103 moves the process to step S1408 because the integrated value of the information amount of the processed concatenated character string is 9.9658 and does not exceed the information entropy “42.3371” serving as the selection criterion.

  In step S1408, the approximate document search unit 103 returns the process to step S1403 because there is a connected character string “Large” to be processed next.

  In step S1403, the approximate document search unit 103 starts processing for the concatenated character string “Large”.

  In step S <b> 1404, the approximate document search unit 103 acquires “... 12:33 14:25 15: 8” as the appearance position in the registered document set of the concatenated character string “Large” from the transposed index 402.

  In step S1405, the approximate document search unit 103 subtracts the position “7” where the concatenated character string appears in the query document from the acquired appearance position “... 12:33 14:25 15: 8” as the corrected appearance position “. 12:26 14:18 15: 1 ”.

  In step S1406, the approximate document search unit 103 adds the corrected appearance position “... 12:26 14:18 15: 1” acquired in step S1405 to the appearance position buffer. For the appearance positions “12:26” and “14:18”, the appearance position information is registered in the appearance position buffer together with the inquiry document appearance position “7” and the number of matches “1”. Since the appearance position information has already been registered for the appearance position “15: 1”, the inquiry document appearance position “9” is changed to a smaller inquiry document appearance position “7” with respect to the registered appearance position information. Replace, add 1 to the number of matches “1” to make “2”.

  In step S1407, the approximate document search unit 103 has an integrated value of the information amount of the processed concatenated character string of 9.9658 + 9.1178 = 19.0836, and does not exceed the information entropy “42.3371” serving as the selection criterion. The process moves to step S1408.

  The approximate document search unit 103 processes the concatenated character strings “wave height” and “sunny sky” in the same manner, and when the processing is performed up to “today”, the integrated value of the information amount of the processed concatenated character string is 9.9658 + 9.1178 + 8.8283 + 8. .4804 + 8.1178 = 44.5100, which exceeds the information entropy “42.3371” serving as the selection criterion, so that the iterative process is terminated, the appearance position buffer in the state shown in FIG. 19 is obtained, and the evaluation concatenated character shown in FIG. End the column selection process.

  Next, the approximate document search unit 103 performs an approximation calculation process shown in FIG. 17 on the appearance position buffer in the state shown in FIG.

  In step S1701, the approximate document search unit 103 secures the result buffer area in the memory and empties it.

  In step S1702, the approximate document search unit 103 secures an area of the document information buffer in the memory and empties it.

  In step S <b> 1703, the approximate document search unit 103 starts processing for the first appearance position information “3:25” stored in the appearance position buffer.

  In step S1704, the approximate document search unit 103 moves the process to step S1706 because the number of matches “1” of the corrected appearance position “3:25” of the appearance position information is less than the specified value (predetermined value) “3”.

  In step S1706, the approximate document search unit 103 moves the process to step S1712 because the document information buffer is empty.

  In step S1712, the approximate document search unit 103 returns the process to step S1703 because there is the next appearance position information “12:26”.

  The approximate document search unit 103 repeats the same processing for the appearance position information “12:26”, “13:14”, “14:18”, “14:34”, and “15: 0”.

  In step S <b> 1703, the approximate document search unit 103 starts processing for the first appearance position information “15: 1” stored in the appearance position buffer.

  In step S1704, the approximate document search unit 103 moves the process to step S1705 because the number of matches “4” of the corrected appearance position “15: 1” of the appearance position information is equal to or greater than the specified value “3”.

  In step S1705, the approximate document search unit 103 adds the appearance position information “15: 1” to the document information buffer.

  In step S1706, the approximate document search unit 103 moves the process to step S1707 because the appearance position information “15: 1” is registered in the document information buffer and is not empty.

  In step S1707, the approximate document search unit 103 has the document ID of the appearance position information “15: 1” currently being processed as “15” and the document ID of the next appearance position information “16: 5” as “16”. Since the document ID is different, the process proceeds to step S1708.

  In step S1708, the approximate document search unit 103 calculates an approximation degree from the appearance position information registered in the document information buffer using the formula shown in FIG. 18, and obtains an approximation degree of 0.75.

  In step S1709, the approximate document search unit 103 moves the process to step S1710 because the obtained degree of approximation 0.75 is equal to or greater than the specified value 0.5 of the degree of approximation.

  In step S1710, the approximate document search unit 103 associates the document ID “15” with the degree of approximation 0.75, and registers it in the result buffer.

  In step S1711, the approximate document search unit 103 empties the document information buffer.

  In step S1712, the approximate document search unit 103 returns the process to step S1703 because there is next appearance position information “16: 5”.

  In step S <b> 1703, the approximate document search unit 103 starts processing for the first appearance position information “16: 5” stored in the appearance position buffer.

  In step S1704, the approximate document search unit 103 moves the process to step S1705 because the number of matches “3” of the corrected appearance position “16: 5” of the appearance position information is equal to or greater than the specified value “3”.

  In step S1705, the approximate document search unit 103 adds the appearance position information “16: 5” to the document information buffer.

  In step S1706, the approximate document search unit 103 moves the process to step S1707 because the appearance position information “16: 5” is registered in the document information buffer and is not empty.

  In step S1707, the approximate document search unit 103 sets the document ID of the appearance position information “16: 5” currently being processed to 16, the document ID of the next appearance position information “19:32” is 19, and the document ID. Are different, the process proceeds to step S1708.

  In step S1708, the approximate document search unit 103 calculates an approximation degree from the appearance position information registered in the document information buffer using the formula shown in FIG. 18, and obtains an approximation degree of 0.5.

  In step S1709, the approximate document search unit 103 moves the process to step S1710 because the obtained degree of approximation 0.5 is equal to or greater than the prescribed value 0.5 of the degree of approximation.

  In step S1710, the approximate document search unit 103 associates the document ID “16” with the degree of approximation of 0.5 and registers it in the result buffer.

  In step S1711, the approximate document search unit 103 empties the document information buffer.

  In step S1712, the approximate document search unit 103 returns the process to step S1703 because there is next appearance position information “19:32”.

  The approximate document search unit 103 repeats the process after the appearance position information “19:32”, processes all the appearance position information, and moves the process to step S1713.

  In step S1713, the approximate document search unit 103 rearranges the document ID “15” (approximation 0.75) and document ID “16” (approximation 0.5) registered in the result buffer in descending order of approximation. .

  In step S <b> 1714, the approximate document search unit 103 sets the document ID “15” (approximation 0.75) and the document ID “16” (approximation 0.5) as the approximate search result 112 using the contents of the result buffer as the search result. Store and finish the process.

(Second Embodiment)
(Approximation degree calculation process that allows positional deviation)

  Next, as an example, an approximate document that determines a matching section even when the positional relationship between the connected character strings selected in the query document and the positional relationship between the connected character strings in the corresponding registered document are within an allowable value The search device will be described with reference to FIG.

  This embodiment is different from the first embodiment only in the approximation calculation processing.

  FIG. 20 shows a processing flow for calculating the degree of approximation in which it is determined that the positional relationship between the connected character strings selected in the inquiry document and the positional relationship between the connected character strings in the corresponding registered document are within the allowable values. FIG.

  FIG. 20 is a diagram showing a detailed flow of the approximation calculation process of the approximate document search process in the second embodiment.

  The processing from step S1701 to step S1703 is the same as the approximation calculation processing (FIG. 17) in the first embodiment.

  In step S2001, the approximate document search unit 103 specifies an allowable section using the current appearance position information as a starting point. The permissible section is acquired as a set of appearance position information that continues with a deviation within a predetermined permissible value (difference in correction position information). Specifically, it is acquired as position information of the head and the end determined as the allowable section. At the same time, the total sum of the numbers of coincidence of the appearance position information included in the allowable section is obtained at the same time. Details of the allowable section specifying process will be described later.

  In step S2002, the approximate document search unit 103 determines whether the sum of the numbers of matches of the appearance position information included in the allowable section obtained in step S2001 is equal to or greater than a specified value. If the number of matches is greater than or equal to the specified value, the process proceeds to step S2003. If the number of matches is less than the specified value, the process proceeds to step S2004.

  In step S2003, the approximate document search unit 103 adds all appearance position information included in the allowable section to the document information buffer.

  In step S2004, the approximate document search unit 103 sets the appearance position information at the end of the allowable section as the appearance position information currently being processed.

  The processing from step S1706 to step S1707 is similar to the approximation calculation processing (FIG. 17) in the first embodiment.

  In step S1708, the approximate document search unit 103 calculates the degree of approximation using the expression shown in FIG. 21 from the appearance position information registered in the document information buffer. The formula for calculating the degree of approximation is an example, and weighting may be performed according to the amount of deviation.

  The processing from step S1709 to step S1714 is the same as the approximation calculation processing (FIG. 17) in the first embodiment.

(Permissible section identification process)
Next, details of the allowable section specifying process in step S2001 will be described with reference to FIG.

  FIG. 23 is a diagram showing a process flow for specifying the allowable section in step S2001.

  In step S <b> 2301, the approximate document search unit 103 sets 0 as the total sum of the number of matches in the entire allowable section.

  In step S2302, the approximate document search unit 103 sets the appearance position information given as the starting point as the head of the allowable section.

  In step S2303, the approximate document search unit 103 starts the iterative process from the appearance position information given as the starting point.

  In step S2304, the approximate document search unit 103 adds the number of matches of the appearance position information being processed to the total number of matches of the entire allowable section.

  In step S2305, the approximate document search unit 103 sets the appearance position information being processed as the end of the allowable section.

  In step S2306, the approximate document search unit 103 obtains a difference between the corrected appearance position of the appearance position information being processed and the next appearance position information as a position shift. When the document to which the corrected appearance position belongs is different, a value exceeding the allowable value such as a maximum value that can be expressed by a computer is set as a position deviation value.

  In step S2307, the approximate document search unit 103 determines whether or not the value of the positional deviation is within an allowable value. If it is within the allowable value, the process proceeds to step S2308. When the allowable value is exceeded, the iterative process is interrupted and the allowable section specifying process is terminated.

  In step S2308, the approximate document search unit 103 sets 1 to the number of appearance position information shifts. Instead of the number of deviations, the amount of deviation may be held, and the degree of approximation may be obtained by weighting according to the amount of deviation.

  In step S2309, if there is appearance position information that should still be processed, the approximate document search unit 103 returns the process to step S2303. If there is no appearance position information to be processed, the process ends.

(Specific example of processing here)
Next, a description will be given of a case where an approximate sentence search is performed using the same inquiry document “Today's clear sky but Osaka Bay wave height” as in the specific example of the first embodiment.

  Similar to the first embodiment, the approximate document search unit 103 obtains an appearance position buffer as shown in FIG. 22 as a result of processing step S801 and step S802.

  The approximate document search unit 103 performs an approximation calculation process shown in FIG. 20 on the appearance position buffer in the state shown in FIG.

  In step S1701, the approximate document search unit 103 secures the result buffer area in the memory and empties it.

  In step S1702, the approximate document search unit 103 secures an area of the document information buffer in the memory and empties it.

  In step S <b> 1703, the approximate document search unit 103 starts processing for the first appearance position information “3:25” stored in the appearance position buffer.

  The approximate document search unit 103 repeats the same processing for the appearance position information (corrected appearance position) “3:25” “12:26” “13:14” “14:18” “14:34”.

  In step S <b> 1703, the approximate document search unit 103 starts processing for the first appearance position information “15: 0” stored in the appearance position buffer.

  In step S2001, the approximate document search unit 103 obtains the sum of the number of matches between the allowable section and the allowable section from the appearance position information “15: 0” as a starting point by the allowable section specifying process shown in FIG. “15: 1” is obtained as “15: 1” as the end of the permissible section (predetermined position range), and “5” is obtained as the total sum of the numbers of coincidence of the permissible sections.

  In step S2002, the approximate document search unit 103 moves the process to step S2003 because the sum “5” of the number of coincidences of allowable sections starting from the appearance position information “15: 0” is equal to or greater than the specified value “3”. .

  In step S2003, the approximate document search unit 103 adds the appearance position information “15: 0” and the appearance position information “15: 1” included in the allowable section to the document information buffer.

  In step S2004, the approximate document search unit 103 sets the appearance position information “15: 1” at the end of the allowable section as the appearance position information being processed.

  In step S1706, the approximate document search unit 103 has the appearance position information “15: 0” and the appearance position information “15: 1” registered in the document information buffer, and is not empty, the process proceeds to step S1707.

  In step S1707, the approximate document search unit 103 has the document ID of the appearance position information “15: 1” currently being processed as “15” and the document ID of the next appearance position information “16: 5” as “16”. Since the document ID is different, the process proceeds to step S1708.

  In step S1708, the approximate document search unit 103 calculates an approximation degree from the appearance position information registered in the document information buffer using the formula shown in FIG. 21, and obtains an approximation degree of 0.975.

  In step S1709, the approximate document search unit 103 moves the process to step S1710 because the obtained approximation degree 0.975 is equal to or greater than the prescribed value 0.5 of the approximation degree.

  In step S1710, the approximate document search unit 103 associates the document ID “15” with the degree of approximation of 0.975 and registers it in the result buffer.

  In step S1710, the approximate document search unit 103 empties the document information buffer.

  Thereafter, similar processing is performed, and the approximate document search unit 103 sets the document ID “15” (approximation level 0.975), document ID “16” (approximation level 0.5), and document ID as documents that approximate the query document. “22” (degree of approximation 0.725) is stored in the approximate search result 112 and the process is terminated.

  The document ID “22” that is considered to be closer to the inquiry document than the document ID “16” cannot be detected in the first embodiment. It becomes possible to detect with a high degree of approximation, and a result close to human sense is obtained.

(Third embodiment)
(Excluding overlapping sections)
Next, when there are a plurality of positional relationships of connected character strings that match the positional relationship of the connected character strings selected in the query document, an approximate document search that limits only one matching portion to one registered document. The apparatus will be described with reference to FIG.

  FIG. 24 is a diagram showing a detailed flow of the approximation calculation process of the approximate document search process in the third embodiment.

  This embodiment is different from the first embodiment only in the approximation calculation processing.

  FIG. 24 shows an approximation calculation for limiting the number of matching points to only one when there are a plurality of positional relationships of concatenated character strings that match the positional relationship of the concatenated character strings selected in the query document. It is a figure which shows the processing flow.

  The processing from step S1701 to step S1703 is the same as the approximation calculation processing (FIG. 17) in the first embodiment.

  In step S2401, the approximate document search unit 103 determines whether or not appearance position information having an inquiry document appearance position of the appearance position information currently being processed is registered in the document information buffer. If registered, the process proceeds to step S2402. If not registered, the process proceeds to step S1705.

  In step S2402, the approximate document search unit 103 determines whether the number of matches of the appearance position information currently being processed is greater than the number of matches of the appearance position information having the same query document appearance position as the appearance position information currently being processed. Determine. If the number of matches of the appearance position information currently being processed is larger, the process proceeds to step S2403. If the number of matches of the appearance position information registered in the document buffer is larger, the process proceeds to step S1706.

  In step S2403, the approximate document search unit 103 deletes the appearance position information having the query document appearance position of the appearance position information currently being processed from the document information buffer, and registers the appearance position information currently being processed in the document information buffer. .

  The processing from step S1705 to step S1714 is the same as the approximation calculation processing (FIG. 17) in the first embodiment.

(Specific example of processing here)
Next, a description will be given of a case where an approximate sentence search is performed using the same inquiry document “Today's clear sky but Osaka Bay wave height” as in the specific example of the first embodiment.

  Similar to the first embodiment, the approximate document search unit 103 obtains an appearance position buffer as shown in FIG. 25 as a result of processing step S801 and step S802.

  In step S1701, the approximate document search unit 103 secures the result buffer area in the memory and empties it.

  In step S1702, the approximate document search unit 103 secures an area of the document information buffer in the memory and empties it.

  In step S <b> 1703, the approximate document search unit 103 starts processing for the first appearance position information “3:25” stored in the appearance position buffer.

  The approximate document search unit 103 repeats the same processing as in the first embodiment for the appearance position information “3:25” to “25: 5”.

  In step S 1703, the approximate document search unit 103 starts processing for the first appearance position information “39:29” stored in the appearance position buffer.

  In step S1704, the approximate document search unit 103 moves the process to step S2401 because the matching number “3” of the corrected appearance position “39:29” of the appearance position information is equal to or greater than the specified value “3”.

  In step S2401, the approximate document search unit 103 determines whether or not the document information buffer has the appearance position information having the inquiry document appearance position “17” of the corrected appearance position “39:29” of the appearance position information.

  Since there is no appearance position information having the inquiry document appearance position “17” in the document information buffer, the process proceeds to step S1705.

  In step S1705, the approximate document search unit 103 adds the appearance position information “39:29” together with the query document appearance position “17” to the document information buffer.

  In step S1706, the approximate document search unit 103 moves the process to step S1707 because the appearance position information “39:29” is registered in the document information buffer and is not empty.

  In step S1707, the approximate document search unit 103 sets the document ID of the appearance position information “39:29” currently being processed to “39” and the document ID of the next appearance position information “39:49” to “39”. Since the document IDs are the same, the process proceeds to step S1712.

  In step S1712, the approximate document search unit 103 returns the process to step S1703 because there is next appearance position information “39:49”.

  In step S <b> 1703, the approximate document search unit 103 starts processing for the first appearance position information “39:49” stored in the appearance position buffer.

  In step S1704, the approximate document search unit 103 moves the process to step S2401 because the number of matches “3” of the corrected appearance position “39:49” of the appearance position information is equal to or greater than the specified value “3”.

  In step S2401, the approximate document search unit 103 determines whether or not the appearance information having the query document appearance position “17” of the corrected appearance position “39:49” of the appearance position information is in the document information buffer.

  Since the appearance position information “39:29” having the inquiry document appearance position “17” is in the document information buffer, the process proceeds to step S2402.

  In step S2402, the approximate document search unit 103 matches the number of matches “3” of the appearance position information “39:49” being processed with the number of matches “3” of the appearance position information “39:29” registered in the document information buffer. ”And the appearance position information“ 39:49 ”being processed is not large, and the process proceeds to step S1706.

  Thereafter, when processing similar to that of the first embodiment is performed, a degree of approximation of 0.5 is obtained for the document with the document ID “39”.

  Although the document ID “39” does not match the query document as much as the description, the document ID “39” is the query document in the result of the approximate document search shown in the first embodiment. The degree of approximation is 1.0, which is the same as in the case of perfect match. This is because the location that matches the query document appears multiple times in the registered document. In the present embodiment, since the evaluation is performed only once when it appears a plurality of times, the degree of approximation is also 0.5 for the document ID “39”, and a result close to a human sense can be obtained.

As described above, according to the present embodiment, the degree of approximation between the registered document and the query document is calculated according to the number of partial character strings in the registered document that have the same positional relationship as the partial character strings constituting the query document. Thus, it is possible to determine a registered document that approximates with high accuracy.

In addition, based on the statistical value of the entire registered document set for the components that appear in the query document, a mechanism is provided that enables high-precision and high-speed search for approximate documents by selecting valid components for discrimination of registered documents. I can do it.

The embodiment of the present invention has been described in detail above. However, the present invention can take an embodiment as a system, an apparatus, a method, a program that can be read and executed by a relay processing apparatus, a storage medium, or the like. Specifically, the present invention may be applied to a system composed of a plurality of devices, or may be applied to an apparatus composed of a single device.

Another object of the present invention is to supply a storage medium storing software program codes for realizing the functions of the above-described embodiments to a system or apparatus, and the computer (or CPU or MPU) of the system or apparatus stores the storage medium. Needless to say, this can also be achieved by reading and executing the program code stored in.

In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the program code constitute the present invention.

As a storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (basic system or operating system) running on the computer based on the instruction of the program code. Needless to say, a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.

Further, after the program code read from the storage medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function is determined based on the instruction of the program code. It goes without saying that the CPU or the like provided in the expansion board or function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

DESCRIPTION OF SYMBOLS 100 Approximate document search apparatus 101 Document registration part 102 Registered document information storage area 103 Approximate document search part 110 Registration document 111 Query document 112 Approximate search result

Claims (12)

A memory for storing a partial character string obtained by decomposing a sentence included in a registered document indicating a document to be approximated by a query document indicating a specified document, and a position of the partial character string in the registered document An information processing apparatus for determining the registered document that approximates the designated inquiry document, comprising:
A partial character string that is the same as the partial character string obtained by decomposing the query document, and that acquires a partial character string included in the registered document stored in the storage means; and
The same partial character string for the query document and partial character string for the registered document, obtained from the position of the partial character string with respect to the query document and the position of the partial character string with respect to the registered document acquired by the partial character string acquisition unit Calculating means for calculating the degree of approximation between the registered document and the inquiry document using the number of partial character strings having a positional relationship;
Information amount calculation means for calculating the information amount of the partial character string obtained according to the appearance probability obtained from the appearance frequency of the partial character string of the registered document stored in the storage means for the partial character string of the inquiry document When,
Information for calculating the information entropy of the registered document obtained according to the appearance frequency for the first character of the partial character string of the query document included in the registered document stored in the storage unit for the partial character string of the query document Entropy calculating means,
A determination unit that determines a registered document that approximates an inquiry document according to a calculation result by the calculation unit;
With
The calculation means is integrated each time the positional relationship between the partial character string for the query document and the partial character string for the registered document is calculated in descending order of the information amount of the partial character string calculated by the information amount calculation means. The information processing apparatus , wherein the partial character string acquisition unit, the calculation unit, and the determination unit are executed until the information amount exceeds the calculated information entropy .   The calculation means performs the calculation by setting the positional relationship of a partial character string with respect to the registered document and the positional relationship of the partial character string with respect to the query document as a positional relationship in which a deviation is within an allowable range as the same positional relationship. The information processing apparatus according to claim 1.   The information processing apparatus according to claim 1, wherein the calculation unit uses a number of the positional relationships that matches as the number of the partial character strings. A correction means for correcting the position of the partial character string of the registered document stored in the storage means according to the position of the partial character string of the inquiry document;
4. The calculation unit according to claim 1, wherein when the correction position of the registered document obtained by correction by the correction unit is the same position, the calculation unit is used as the number having the same positional relationship. The information processing apparatus according to claim 1. The calculating means uses, as the number of partial character strings, the number of partial character strings having the same positional relationship at the correction position where the number of partial character strings determined to have the same positional relationship is equal to or greater than a predetermined value. The information processing apparatus according to claim 4 . It said determining means, a registration document approximation degree calculated is equal to or larger than a predetermined value by the calculation means, to any one of claims 1 to 5, characterized in that to determine the registration document, which approximates the query document The information processing apparatus described. The calculation means calculates the number of the partial character strings when the positional relationship between the partial character string for the query document and the partial character string for the registered document has the same positional relationship within a predetermined position range. the information processing apparatus according to any one of claims 1 to 6, characterized by using as few. The calculation means has the same positional relationship when the correction position difference of the registered document obtained by being corrected by the correction means for a partial character string with respect to the inquiry document is within an allowable value. as the number, the information processing apparatus according to any one of claims 4 to 7, characterized in that to calculate. The information processing apparatus according to claim 8 , wherein the calculating unit calculates the degree of approximation between the registered document and the inquiry document according to a difference between the number having the same positional relationship and a correction position. When the correction position of the registered document obtained by correction by the correction unit is different for partial character strings at the same position included in the inquiry document, the calculation unit has partial character strings having the same positional relationship. wherein the number of often used as the number of the partial strings information processing apparatus according to any one of claims 4 to 9, wherein. A memory for storing a partial character string obtained by decomposing a sentence included in a registered document indicating a document to be approximated by a query document indicating a specified document, and a position of the partial character string in the registered document A method of controlling an information processing apparatus for determining the registered document that approximates the designated inquiry document, comprising:
The partial character string acquisition means of the information processing device is the same partial character string as the partial character string obtained by decomposing the query document, and the partial character string included in the registered document stored in the storage means A partial character string acquisition step to be acquired;
The calculation means of the information processing apparatus obtains the partial character string for the inquiry document and the registration obtained from the position of the partial character string for the inquiry document and the position of the partial character string for the registered document acquired by the partial character string acquisition step. A calculation step of calculating the degree of approximation between the registered document and the query document using the number of the partial character strings having the same positional relationship with the partial character strings with respect to the document;
The partial character obtained by the information amount calculation means of the information processing apparatus according to the appearance probability obtained from the appearance frequency of the partial character string of the registered document stored in the storage means for the partial character string of the inquiry document An information amount calculating step for calculating the information amount of the column;
The information entropy calculating means of the information processing apparatus targets the partial character string of the inquiry document according to the appearance frequency with respect to the first character of the partial character string of the inquiry document included in the registered document stored in the storage means. An information entropy calculating step of calculating information entropy of the obtained registered document;
A determination step in which the determination unit of the information processing apparatus determines a registered document that approximates an inquiry document according to a calculation result of the calculation step;
Run
The calculation step is integrated each time the positional relationship between the partial character string for the query document and the partial character string for the registered document is calculated in descending order of the information amount of the partial character string calculated by the information amount calculation step. The control method of the information processing apparatus , wherein the partial character string acquisition step, the calculation step, and the determination step are executed until the information amount exceeds the calculated information entropy . A memory for storing a partial character string obtained by decomposing a sentence included in a registered document indicating a document to be approximated by a query document indicating a specified document, and a position of the partial character string in the registered document A program that can be read and executed by an information processing apparatus that determines the registered document that approximates the specified inquiry document,
The information processing apparatus;
A partial character string that is the same as the partial character string obtained by decomposing the query document, and that acquires a partial character string included in the registered document stored in the storage means; and
The same partial character string for the query document and partial character string for the registered document, obtained from the position of the partial character string with respect to the query document and the position of the partial character string with respect to the registered document acquired by the partial character string acquisition unit Calculating means for calculating the degree of approximation between the registered document and the inquiry document using the number of partial character strings having a positional relationship;
Information amount calculation means for calculating the information amount of the partial character string obtained according to the appearance probability obtained from the appearance frequency of the partial character string of the registered document stored in the storage means for the partial character string of the inquiry document When,
Information for calculating the information entropy of the registered document obtained according to the appearance frequency for the first character of the partial character string of the query document included in the registered document stored in the storage unit for the partial character string of the query document Entropy calculating means,
A determination unit that determines a registered document that approximates an inquiry document according to a calculation result by the calculation unit ;
To function ,
The calculation means is integrated each time the positional relationship between the partial character string for the query document and the partial character string for the registered document is calculated in descending order of the information amount of the partial character string calculated by the information amount calculation means. Until the amount of information exceeds the calculated information entropy, the partial character string acquisition unit, the calculation unit, and the determination unit function as executions .

Images