JP2007317131A - Document management method, document retrieval method and device, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To retrieve a document using the same/similar image from documents, which includes images and managed in the form of paper mediums, by using an image feature. <P>SOLUTION: A document, which includes an image and is not digitalized but printed on a paper medium and the like, is registered in a database while associating an image feature with identification information of the document. When the image feature of the image inputted as retrieval information is extracted and the database is searched with the image feature, the document with the image having the same/similar feature is retrieved. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a document management method, a document search method, an apparatus, and a program, and more particularly, by digitizing a document managed on a paper medium, extracting an image area, and storing and managing it on a database. The present invention relates to a document management method, a document retrieval method, an apparatus, and a program that enable retrieval of documents including images.

  Conventional document management techniques include a technique for managing a document update history (first technique) and a document management technique (second technique) for documents written in a language such as HTML or XML. .

  The first technique described above is a technique in which document management information is output as numbers or barcodes at the time of printing, and the pattern information of the barcode is detected by pattern matching at the time of retrieval to obtain the management information of the corresponding document ( For example, see Patent Document 1).

On the other hand, the second technique described above utilizes the fact that a document described in HTML, XML, etc. is structured by tags (representing attributes of information), and stores information on locations corresponding to specific tags. This is a technique for managing (structuring) documents by extracting and creating a database together with inheritance relationships of documents (see, for example, Patent Document 2).
Japanese Patent Publication No. 7-76905 “Document Management Device” JP 2003-330938 A “Structured document storage method and search method thereof”

  However, both the first technique (Patent Document 1) and the second technique (Patent Document 2) are based on the premise that an electronic document is created or an electronic document has already been created and already exists in a database. Documents that are managed only with this are excluded.

  The first technique (Patent Document 1) is a method of extracting meta information (revision history, management information, etc.) from information such as a barcode of an electronic document. The document is searched using the meta information as a key. Needs to store these meta information in the database in advance. Therefore, it is unsuitable to apply the first technique as it is and search for a document from meta information.

  On the other hand, the second technique (Patent Document 2) can retrieve information, images, and the like described in a document, but is premised on being structured with tags. Therefore, it is impossible to extract and manage information from a document including only characters and images without a tag.

  The present invention has been made in view of the above points, and uses an image feature to search for an image mixed document in which the same or similar images are used for an image mixed document managed on a paper medium. It is an object of the present invention to provide a document management method, a document search method, an apparatus, and a program that can execute the above-described process.

  FIG. 1 is a diagram for explaining the principle of the present invention.

The present invention (Claim 1) is a document management method for managing an image mixed document managed in a database by digitizing a document including mixed images.
A document input unit inputs an electronic document or a paper medium document, and if the document is a paper medium, a document input step (step 1) for digitizing the document;
An image area extracting step (step 2) in which the image area extracting means extracts an image area from the input electronic document;
An image feature extraction step (step 3) in which the image feature extraction means extracts an information sequence representing the feature of the image itself from the extracted image region as an image feature;
The image feature registration means performs an image feature registration step (step 4) for registering the image feature in the database in association with the document identification information.

The present invention (Claim 2) further performs a document identification information embedding step of embedding document identification information in the image itself,
In the image feature registration step,
The image feature and the image in which the document identification information is embedded are associated and registered in the database.

The present invention (Claim 3) is a document management method for managing information related to mixed image documents stored in a database,
An image feature search step for determining whether or not there is an identical or / and similar image feature in the database each time the image feature is registered in the database;
When the same or / and similar image feature exists in the database in the image feature search step, the image feature information updating means adds the document identification information corresponding to the existing image feature information in the database. If the image feature information does not exist, the image feature information update step of adding the image feature information of the registered image feature to the database is performed.

  FIG. 2 is a principle configuration diagram of the present invention.

The present invention (Claim 4) is a document management apparatus that digitizes a document having mixed images and manages the mixed image document managed by the database 106.
An electronic document or a paper medium document is input. If the document is a paper medium, a document input unit 101 for digitizing the document;
Image area extraction means 103 for extracting an image area from the electronic document;
Image feature extraction means 104 for extracting an information sequence representing the characteristics of the image itself from the extracted image region as an image feature;
Image feature registration means 105 for registering image features in the database 106 in association with document identification information.

The present invention (Claim 5) further includes document identification information embedding means for embedding document identification information in the image itself,
The image feature registration means 105
Means for associating and registering the image feature and the image in which the document identification information is embedded in the database;

The present invention (Claim 6) is a document management apparatus for managing information related to mixed image documents stored in the database 106,
Image feature search means 107 that searches the database 106 to determine whether or not there is an identical or / and similar image feature each time an image feature is registered;
In the image feature search means 107, when the same or / and similar image feature exists in the database 106, the existing image feature information in the database 106 is updated. Image feature information updating means 108 for adding image feature information to the database 106;
Have

The present invention (Claim 7) is an image mixed document search method for performing a search based on an image in a document with respect to a database in which an electronic image mixed document is managed,
A document input step in which the document input means inputs an electronic document including an image to be searched, an image to be searched itself, or a paper medium document, and if the document is a paper medium, the document input step for digitizing the document (Step 11),
An image area extracting step (step 12) in which the image area extracting means extracts an image area from the electronic document;
An image feature extraction step (step 13) in which the image feature extraction means extracts an information string representing the feature of the image itself from the extracted image region as an image feature;
An image feature search step (step 14) in which the image feature search means searches the database for the same image feature as the image feature;
The document output means performs a document output step (step 15) for outputting a document associated with the image feature retrieved from the database.

Further, according to the present invention (claim 8), the image feature search means performs a document identification information acquisition step of acquiring the document identification information from the image associated with the image feature obtained in the image feature search step,
In the document output step,
A document is acquired from the database based on the document identification information and output.

The present invention (Claim 9) is an image mixed document search apparatus that performs a search based on an image in a document with respect to a database in which the digitized image mixed document is managed,
An electronic document including an image to be searched, an image to be searched itself, or a paper medium document; if the document is a paper medium, a document input unit 101 for digitizing the document;
Image area extraction means 103 for extracting an image area from the electronic document;
Image feature extraction means 104 for extracting an information sequence representing the characteristics of the image itself from the extracted image region as an image feature;
Image feature retrieval means 107 for retrieving the same image feature as the image feature from the database;
Document output means 110 for outputting a document associated with the image feature retrieved from the database.

Further, the present invention (Claim 10), in the image feature search means 107
Including document identification information acquisition means for acquiring document identification information from an image associated with the image feature obtained by the image feature search means,
The document output means 110
Means for obtaining and outputting a document from the database 106 based on the document identification information is included.

  The present invention (Claim 11) is a document management program for causing a computer to execute each means of the document management apparatus according to Claims 4 to 6.

  The present invention (Claim 12) is a document search program for causing a computer to execute each means of the document search apparatus according to Claims 9 to 10.

  According to the present invention, a non-digitized image-containing document printed on a paper medium is digitized, the feature amount of the image extracted from the document is extracted, and the database is associated with the identification information of the document. By storing them in the image, it is possible to search image mixed documents in which the same or similar images are used.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
In the present embodiment, processing for registering a document in which input images are mixed in a database will be described.

  FIG. 3 shows the configuration of the document management apparatus according to the first embodiment of the present invention.

  A document management apparatus 100 shown in FIG. 1 includes a document input unit 101, a content storage memory 102, an image area extraction unit 103, an image feature extraction unit 104, an image feature registration unit 105, and a database 106.

  The content storage memory 102 stores an electronic file of a document obtained by the document input unit 101, an image obtained by the image region extraction unit 103, and an image feature corresponding to each image obtained by the image feature extraction unit 104.

  FIG. 4 is a flowchart of the operation of the document management apparatus according to the first embodiment of the present invention. The operation of the apparatus configuration in FIG. 3 will be described below with reference to FIG.

Step 201) Document input step:
The document input unit 101 inputs an electronic document or a paper medium document. A document managed on a paper medium is digitized using an input device such as a scanner. The digitized data is stored in the content storage memory 102 as an electronic file. Here, a scanner is taken as an example of electronic means, but a digital camera may be used.

Step 202) Image region extraction step:
The image area extraction unit 103 extracts an area classified as an image from the electronic document stored in the content storage memory 102 and stores the extracted area in the content storage memory 102. In order to extract an image region from an electronic document, for example, there are the following methods.

  By binarizing a color document and applying a filter suitable for document area extraction and a filter suitable for chart area extraction to the same document image, a character area is extracted from the former and a chart area is extracted from the latter. Technology (Katsuyama, Kurokawa, Takebe, Fujiki, Naoi: "Color document image layout analysis method using two types of binarization methods for character string extraction / chart extraction" Shingaku Sodai, D-12- 88, p220 (2006)). By using this technique, the obtained chart area may be regarded as an image area, or the chart area may be further separated into a chart area and a table area, and the figure area may be extracted as an image area. Good. The chart area can be extracted by obtaining a histogram in the horizontal direction and the vertical direction, and taking the case where the histograms in both directions have peaks periodically as the table area.

Step 203) Image feature extraction step:
The image feature extraction unit 104 acquires the image obtained by the image region extraction unit 103 from the content storage memory 102, and extracts an image feature that uniquely identifies the image. When a plurality of images are extracted from one type of document, image features are extracted in the same manner for all images. The obtained image features are stored in the content storage memory 102. Image features include, for example, an image divided into blocks, and average pixel values for each block arranged for all blocks (Jia Li, James Z Wang, Gio Wiederhold, “IRM: Integrated Region Matching for Image Retrieval”, Proceedings of the eighth ACM international conference on Multimedia, pp.147-156 (2000)), dividing the image into blocks, and the average pixel value of each block in all blocks (for example, what is counted from the darkest one) (Takada, Abe, Kawamura: “Content identification method with transformation tolerance”, Image Recognition and Understanding Symposium (MIRU2005), 2005).

Step 204) Image feature registration step:
The image feature registration unit 105 extracts the image feature corresponding to each image included in one document obtained by the image feature extraction unit 104 from the content storage memory 102 and associates it with the document information (document management number) in the database 106. Store. Correspondence between image features and document information at the time of registration in the database 106 is, for example, a method of managing document management numbers and image features in a table format as shown in FIG. 5, or as shown in FIG. There is a method of managing the document management number by embedding a digital watermark in the image itself and as shown in FIG. 7, the image feature and the information indicating the location of the image (image storage location) are paired.

[Second Embodiment]
In the present embodiment, a process for updating the database 106 in which image features and document information are stored will be described.

  FIG. 8 shows the configuration of the document management apparatus according to the second embodiment of the present invention.

  In the figure, the same reference numerals are given to the same components as those in FIG. 3, and the description thereof is omitted.

  The document management apparatus 200 shown in FIG. 8 includes a function for searching for image features and a function for updating image feature information. An image feature registration unit 105, a database 106, an image feature search unit 107, and an image feature information update unit. 108, a content storage memory 102. Although this configuration is described separately from FIG. 3, the configuration may be integrally configured by adding the image feature search unit 107 and the image feature information update unit 108 to the configuration of FIG. 3. .

  Hereinafter, the operation of the document management apparatus 200 will be described.

  FIG. 9 is a flowchart of the operation in the second embodiment of the present invention.

Step 204) Image feature registration step:
This corresponds to step 204 in the first embodiment.

Step 205) Image feature search step:
In step 204 described above, the image feature search unit 107 searches for whether or not the same or similar image feature exists in the database 106 for the image feature registered in the database 106, and the result Is temporarily stored in the content storage memory 102.

  Hereinafter, a case where the image feature information is managed in the format shown in FIG. 5 will be described. In the search, if the input image feature and the character string of the image feature in the database 106 completely match, it is determined that they are the same image feature. If there is a high similarity of the image feature, a similar image feature is used. Judge that there is. Note that the similarity of image features is, for example, the inner product between vectors when character strings of image features are regarded as multidimensional vectors, the number of mismatches when character strings are compared for each element, or the mismatch rate. The former or the latter also has a higher similarity as the numerical value is smaller. The image feature search unit 107 obtains a management number including an image determined to be identical or similar to the search target image.

  When the image feature information is managed in the format shown in FIG. 7, the location (image storage location) of the image having the same or similar image feature is obtained.

Step 206) Image feature information update step:
The image feature information update unit 108 retrieves the result obtained by the image feature search unit 107 from the content storage memory 102 and updates the information stored in the database 106.

  Hereinafter, a case where image feature information is managed in the format shown in FIG. 5 will be described in detail.

  When the image feature information update unit 108 matches the image feature input from the image feature registration unit 105 in the database 106, that is, as shown in FIG. 10A, the document feature number “EF-0099”. When the image feature data included in “” matches “234342567567”, the image feature information is updated by adding the document management number “EF-0099” to the image feature column. If the image feature input from the image feature registration unit 105 is not determined to be the same or similar to all the image features in the database 106, as shown in FIG. The image feature “004567891234” of “1022” is newly added. Further, when there is an image feature similar to the image feature input from the image feature registration unit 105 on the database 106, that is, as shown in FIG. 10C, the image feature “123123123446 of the document management number“ EF-0003 ”. "Is similar to the image feature" 123123123446 "of the document management number" AB-0001 ", a new document management number corresponding to the image feature is newly added, and at the same time, the image feature" 123123123456 "retrieved as a similar image feature “123123123446” is added as a similar image feature to the column of “No.”.

  As described above, the information in the database 106 is updated by determining whether the new image feature registered from the image feature registration unit 105 matches the image feature on the database 106 with respect to the document management number.

  Also, it is obvious that the information can be updated in the same manner when the image feature information is managed in a format as shown in FIG.

[Third Embodiment]
In the present embodiment, search processing will be described.

  FIG. 11 shows the configuration of the search device according to the third embodiment of the present invention. In this figure, the same components as those in FIGS. 3 and 8 are denoted by the same reference numerals, and description thereof is omitted.

  11 includes a content storage memory 102, an image area extraction unit 103, an image feature extraction unit 104, a database 106, an image feature search unit 107, a search information input unit 109, and a document output unit 110.

  Although the search device 300 shown in FIG. 11 has an independent configuration, the search device 300 may be included in the document management device shown in FIGS.

  In the following, components that do not overlap with the configurations of FIGS. 3 and 8 will be described with reference to the flowchart of FIG. In the flowchart shown in FIG. 12, the same steps as those in FIGS. 4 and 9 are given the same step numbers.

Step 207) Search information input step:
The search information input unit 109 inputs a document including a search target image or the search target image itself. When the input target is a paper medium, it is digitized via a device such as a scanner and stored in the content storage memory 102 as in the first embodiment. If the input target is electronic data, it is stored in the content storage memory 102 as it is.

Step 202) Image region extraction step:
When the data stored in the content storage memory 102 by the search information input unit 109 is in the form of a document, the image region extraction unit 103 extracts the image region, and the obtained one or more image data are stored in the content storage memory. 102.

Step 203) Image feature extraction step:
The image feature extraction unit 104 extracts image features from the image data stored in the content storage memory 102.

Step 205) Image feature search step:
The image feature search unit 107 searches for an image feature that matches the extracted image feature and the image feature stored in the database 106, and a document management number that contains an image that matches or is similar to the input image feature. Information on the storage location of the image (when managed in the format shown in FIG. 7) is obtained (when managed in the format shown in FIG. 5). In the latter case, the corresponding image itself is temporarily displayed on an external device such as a monitor, and then document management information embedded as digital watermark information is detected from an image designated by the user with a mouse or a keyboard. .

Step 208) Document output step:
The document output unit 110 outputs an electronic document corresponding to the document management number obtained from the image feature search unit 107 to an external device. Here, the external device that displays the electronic document and the image itself may be a monitor such as a CRT or a printer.

  In the present invention, the functions of the document management device and the search device shown in FIGS. 3, 8, and 11 in the first to third embodiments are constructed as a program and used as the document management device and the search device. It can be installed and executed on a computer to be distributed, or distributed via a network.

  Further, the constructed program can be stored in a portable storage medium such as a hard disk, a flexible disk, or a CD-ROM, and can be installed or distributed in a computer.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

  The present invention can be applied to a technique for digitizing an image mixed document and managing it in a database, particularly a color document image recognition technique.

It is a figure for demonstrating the principle of this invention. It is a principle block diagram of this invention. It is a block diagram of the document management apparatus in the 1st Embodiment of this invention. It is a flowchart of operation | movement of the document management apparatus in the 1st Embodiment of this invention. It is an example (the 1) of the database in the 1st Embodiment of this invention. It is an example which embeds the management number in the 1st Embodiment of this invention using a digital watermark. It is an example (the 2) of the database in the 1st Embodiment of this invention. It is a block diagram of the document management apparatus in the 2nd Embodiment of this invention. It is a flowchart of the operation | movement in the 2nd Embodiment of this invention. It is an example of the database in the update process in the 2nd Embodiment of this invention. It is a block diagram of the search device in the 3rd Embodiment of this invention. It is a flowchart of operation | movement of the document search device in the 3rd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Document management apparatus 101 Document input means, document input part 102 Content storage memory 103 Image area extraction means, Image area extraction part 104 Image feature extraction means, Image feature extraction part 105 Image feature registration means, Image feature registration part 106 Database 107 Image Feature search unit, image feature search unit 108 Image feature information update unit, image feature information update unit 109 Search information input unit, search information input unit 110 Document output unit, document output unit 200 Document management device 300 Document search device

Claims (12)

A document management method that digitizes a document containing mixed images and manages the mixed image document managed in a database,
A document input means for inputting an electronic document or a paper medium document, and if the document is a paper medium, a document input step for digitizing the document;
An image region extracting step for extracting an image region from the electronic document;
An image feature extraction step, wherein the image feature extraction means extracts an information sequence representing the feature of the image itself from the extracted image region as an image feature;
An image feature registration means for registering the image feature in the database in association with the identification information of the document;
A document management method characterized by: Further performing a document identification information embedding step of embedding the document identification information in the image itself,
In the image feature registration step,
Registering the image feature and the image in which the document identification information is embedded in the database in association with each other;
The document management method according to claim 1. A document management method for managing information related to mixed image documents stored in a database,
An image feature search step for searching the database to determine whether or not there is an identical or / and similar image feature in the database each time the image feature is registered in the database. ,
When the same or / and similar image feature exists in the database in the image feature search step, the image feature information update unit additionally adds document identification information corresponding to the existing image feature information in the database. An image feature information update step of adding the image feature information of the registered image feature to the database,
A document management method characterized by: A document management apparatus that digitizes a document containing mixed images and manages the mixed image document managed by a database,
An electronic document or a paper medium document is input, and if the document is a paper medium, a document input means for digitizing the document;
Image area extraction means for extracting an image area from the electronic document;
Image feature extraction means for extracting an information string representing the characteristics of the image itself from the extracted image region as an image feature;
Image feature registration means for registering the image feature in a database in association with identification information of the document;
A document management apparatus comprising: A document identification information embedding unit that embeds the document identification information in the image itself;
The image feature registration means includes:
Means for associating and registering the image feature and the image in which the document identification information is embedded in the database;
The document management apparatus according to claim 4. A document management apparatus for managing information related to mixed image documents stored in a database,
Image feature search means for determining whether or not there is an identical or / and similar image feature in the database each time the image feature is registered in the database;
In the image feature search means, if the same or / and similar image feature exists in the database, it is updated by adding the identification information of the document corresponding to the existing image feature information in the database. If not, image feature information update means for adding image feature information of the registered image features to the database;
A document management apparatus comprising: An image mixed document search method for performing a search based on an image in a document with respect to a database in which an electronic image mixed document is managed,
A document input step in which the document input means inputs an electronic document including an image to be searched, an image to be searched itself, or a paper medium document, and if the document is a paper medium, the document input step for digitizing the document When,
An image region extracting step for extracting an image region from the electronic document;
An image feature extraction step, wherein the image feature extraction means extracts an information sequence representing the feature of the image itself from the extracted image region as an image feature;
An image feature search means for searching the image feature for the same image feature as the image feature from the database;
A document output means for outputting a document associated with the image feature retrieved from the database;
A document search method characterized by: The image feature search means performs a document identification information acquisition step of acquiring document identification information from an image associated with the image feature obtained in the image feature search step;
In the document output step,
Obtaining and outputting a document from the database based on the document identification information;
The document search method according to claim 7. An image mixed document search apparatus that performs a search based on an image in a document with respect to a database in which an electronic image mixed document is managed,
An electronic document including a search target image, or a search target image itself, or a paper medium document; if the document is a paper medium, a document input unit that digitizes the document;
Image area extraction means for extracting an image area from the electronic document;
Image feature extraction means for extracting an information sequence representing the characteristics of the image itself from the extracted image region as an image feature;
Image feature search means for searching the database for the same image feature as the image feature;
Document output means for outputting a document associated with the image feature retrieved from the database;
A document search apparatus characterized by comprising: The image feature search means includes:
Including document identification information obtaining means for obtaining document identification information from an image associated with the image feature obtained by the image feature retrieval means,
The document output means includes
Means for obtaining and outputting a document from the database based on the document identification information;
The document search apparatus according to claim 9. On the computer,
7. A document management program for causing each means of the document management apparatus according to claim 4 to be executed. On the computer,
11. A document search program for causing each means of the document search apparatus according to claim 9 to be executed.

Images