JP2006195886A - Image processing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To relieve burden on a network under an extreme overhead due to making all servers objects to be searched concerning a method for retrieving an original file from the server, based on an extracted keyword. <P>SOLUTION: Character strings are listed in order of generation frequency from whole documents preserved in the server and are registered and preserved in a file. A retrieval side uses the information, so as not to have necessity to search all the servers. Consequently, the burden on the network is relieved and retrieval time is shortened. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing system for converting image data read by an image processing apparatus such as a copying machine into vector data reusable by so-called document creation application software such as Word.

  In recent years, paperless offices are rapidly becoming paperless as environmental problems are screamed. In other words, it is possible to construct a document management system by reading a paper document that has been conventionally stored in a binder or the like with a scanner and converting it into a portable document format (hereinafter referred to as PDF) and storing it as a database in an image storage device. On the other hand, in the MFP with the expanded function, when recording an image in advance, the pointer information in the image storage device in which the image file exists is recorded as additional information on the cover or description information of the document, and is stored again. When the document is copied or reused, the storage location of the original electronic file is detected from the pointer information, and the original information of the electronic file is directly used to reduce the storage of the entire paper document (for example, patents) Reference 1).

  However, as described above, pointer information must be printed on an image output on the MFP side. In general, a bar code, a digital watermark, or the like is used as the pointer information, but the output image itself may be crushed by the pointer information.

In order to solve this problem, it is possible to specify the original electronic file using the keyword character string, image, layout, etc. in the scanned image without printing the pointer information on the MFP side.
JP 2004-46537 A

  However, in the document management system as described in the conventional example, in a situation where there are a plurality of document management servers, in order to specify in which server the original document is stored, all the document management servers are As the number of servers increases, the search time becomes longer and the load on the network becomes larger.

  Accordingly, an object of the present invention is to shorten the search time required for original document search and reduce the load on the network.

To achieve the above objective,
According to the first aspect of the present invention, the electronic file in the document management server is read by using vectorization means from the image information obtained by the means for reading and scanning the original before searching for the electronic file of the original. Even in an environment where there are multiple document management servers on the network, it is possible to extract the characteristics of each scanned document, compare the similarities, and perform a search preferentially from a document management server with a high similarity. The search time can be shortened by reducing the load on the network.

  According to the second aspect of the present invention, in the image processing system according to the first aspect, the means for identifying the electronic file is a means for recognizing additional information indicating the storage location of the electronic file additionally recorded on the document. The original electronic file can be easily identified from the information.

The invention described in claim 3 is the image processing system according to claim 1,
The electronic file identification means has means to search for specific information described in the manuscript from the files stored in the storage means. As a result of the search, the electronic file is specified by matching the specific information. The original electronic file can be easily specified even for a document in which no information is recorded.

  According to a fourth aspect of the present invention, in the image processing system according to the first aspect, the vectorizing means vectorizes the character portion into character font data by the OCR means for optically recognizing the character area in the document. High quality.

  Further, the invention according to claim 5 is characterized in that, in the image processing system according to claim 1, the vectorization means divides the document into a plurality of objects and vectorizes each object independently. Vectorized objects can be handled independently.

  In the image processing system according to claim 6, in the image processing system according to claim 1, the vectorization means converts the vectorized object into, for example, the rtf format that can be handled by existing document creation software. Can be reused on other document creation application software.

  According to a seventh aspect of the present invention, in the image processing system according to the first aspect, what character string is most frequently used by performing character recognition processing using the vectorized OCR as a feature extraction means. The character string can be extracted as a feature.

  According to an eighth aspect of the present invention, in the image processing system according to the first aspect, the character string using the largest font is specified by performing character recognition processing using the vectorized OCR as a feature extraction means. The character string can be extracted as a feature.

  In the image processing system according to claim 9, in the image processing system according to claim 1, by calculating the vectorized object division as the feature extraction means, it is possible to calculate what shape and size of the object is included. And can be extracted as features.

  According to a tenth aspect of the present invention, in the image processing system according to the first aspect, as the feature extraction means, the vectorization is performed and the image data is vectorized, whereby the features of the image itself are digitized and extracted as features. I can do it.

  The invention according to claim 11 is the image processing system according to claim 1, wherein the similarity is calculated by comparing the characteristics of the electronic document in the document management server with the characteristics of the image data read and scanned. In addition, since a search can be performed from a document management server having a high degree of similarity, the time required for the search can be reduced and the load on the network can be reduced.

  According to the first aspect of the present invention, the electronic file in the document management server is obtained by using the vectorizing means before searching the electronic file of the original document from the image information obtained by the means for reading and scanning the original document. In an environment where there are multiple document management servers on the network, it is possible to extract the features of each scanned and scanned document, compare their similarities, and perform search preferentially from the document management servers with high similarity. The search time can be shortened by reducing the load on the network.

  According to a second aspect of the present invention, in the image processing system according to the first aspect, the means for identifying the electronic file is means for recognizing additional information indicating a storage location of the electronic file additionally recorded on the document. So you can easily identify the original electronic file from the image information.

According to invention of Claim 3, in the image processing system of Claim 1,
The electronic file identification means has means to search for specific information described in the manuscript from the files stored in the storage means. As a result of the search, the electronic file is specified by matching the specific information. The original electronic file can be easily specified even for a document in which no information is recorded.

  According to the fourth aspect of the present invention, in the image processing system according to the first aspect, the vectorizing means vectorizes the character portion into character font data by the OCR means for optically recognizing the character area in the document. The department is high-grade.

  According to the invention described in claim 5, in the image processing system according to claim 1, the vectorization means divides the document into a plurality of objects, and vectorizes each object independently. Vectorized objects can be handled independently.

  According to the invention described in claim 6, in the image processing system according to claim 1, the vectorization means converts the vectorized object into, for example, the rtf format that can be handled by existing document creation software. Can be reused on existing document creation application software.

  According to a seventh aspect of the present invention, in the image processing system according to the first aspect, by performing character recognition processing using the vectorized OCR as a feature extraction means, what character string is most frequently used. Whether it is used can be specified, and the character string can be extracted as a feature.

  According to an eighth aspect of the present invention, in the image processing system according to the first aspect, the character string using the largest font is obtained by performing the character recognition process using the vectorized OCR as the feature extraction means. The character string can be extracted as a feature.

  According to the ninth aspect of the invention, in the image processing system according to the first aspect, what shapes and sizes of objects are included by using the vectorized object division as a feature extraction means. Can be calculated and extracted as a feature.

  According to a tenth aspect of the present invention, in the image processing system according to the first aspect, the vectorization is performed as the feature extraction means and the image data is vectorized, whereby the features of the image itself are digitized and extracted as features. I can do it.

  According to the invention described in claim 11, in the image processing system described in claim 1, the similarity is obtained by comparing the characteristics of the electronic document in the document management server with the characteristics of the image data read and scanned. Since the search can be performed from the document management server having a high degree of similarity, the time required for the search can be reduced and the load on the network can be reduced.

  The best mode for carrying out the invention will be described below.

  Embodiments of the present invention will be described. FIG. 1 is a block diagram showing a configuration example of an image processing system according to the present invention. This image processing system is realized in an environment where the office 10 and the office 20 are connected by the Internet 104. A LAN 107 constructed in the office 10 is connected to the MFP 100, a management PC 101 that controls the MFP 100, a client PC (external storage means) 102, a document management server 106, its database 105, and a proxy server 103. The LAN 107 and the LAN 108 in the office 20 are connected to the Internet 104 via the proxy server 13. In the present invention, the MFP 100 is in charge of an image reading unit of a paper document and a part of image processing for the read image signal, and the image signal is input to the management PC 101 using the LAN 109. The management PC is a normal PC and includes an image storage unit, an image processing unit, a display unit, and an input unit. A part of the management PC is integrated with the MFP 100.

  FIG. 2 is a configuration diagram of the MFP 100. In FIG. 2, an image reading unit 110 including an auto document feeder (hereinafter referred to as ADF) irradiates a bundle or one original image with a light source (not shown), and forms an original reflection image on a solid-state image sensor with a lens. A raster-like image reading signal is obtained as image information having a density of 600 DPI from the solid-state imaging device. In the normal copying function, the image signal is processed into a recording signal by the data processing unit 115. In the case of copying every plural number, recording data for one page is temporarily stored in the recording device 111 and then sequentially output to the recording device 112. Then, an image is formed on the paper.

  On the other hand, the print data output from the client PC 102 is converted into raster data that can be recorded by the data processing device 115 from the LAN 107 via the network IF 114, and then formed as a recorded image on paper by the recording device.

  An operator's instruction to the MFP 100 is performed from a key operation unit equipped in the MFP, a keyboard input to the management PC, and an input device 113 including a mouse, and these series of operations are not shown in the data processing device 115. Controlled by the department.

  On the other hand, the status display of operation input and the display of image data being processed are performed on the display device 116. The storage device 111 is also controlled by the management PC, and data exchange and control between the MFP and the management PC are performed using the network IF 117 and the directly connected LAN 109.

  The processing outline is as follows.

  Next, an overview of the entire image processing according to the present invention will be described with reference to FIG.

  In FIG. 3, first, the image reading unit 110 of the MFP 100 is operated to scan one original in a raster pattern, and an image information input process 120 obtains a 600 DPI-8-bit image signal. The image signal is preprocessed by the data processing unit 115 and stored in the storage device 111 as image data for one page. The CPU of the management PC 101 first separates the area from the stored image signal into a character / line image portion and a halftone image portion, and the character portion is further divided into blocks or a line or a line. Separated into organized tables and figures, each segmented. On the other hand, the image portion expressed in halftone is divided into so-called independent objects for each block, such as an image portion of a block separated into rectangles, a background portion, and the like (step 121).

  At this time, an object corresponding to a character portion in the document image is detected, character recognition is performed by OCR, a character string used most frequently in the document image is extracted as a keyword character string, and this keyword character string is used. Thus, it is determined from which document management server on the network the search should be started in order (step 137).

  At this time, a two-dimensional barcode recorded as additional information in the original image or an object corresponding to the URL is detected, and the URL recognizes the character by OCR. If the two-dimensional barcode is used, the mark is decoded (step 122). Pointer information in the storage device storing the original electronic file of the original is detected (step 123). As other means for adding pointer information, there are a so-called digital watermarking method that is not directly visualized, such as a method of embedding information in a character interval and a method of embedding in a halftone image.

  If the pointer information is detected, the process branches to step 125, and the original electronic file is searched from the address indicated by the pointer. The document management server that searches for the electronic file in the database 105 in the hard disk in the client PC in FIG. 1 or in the document management server 105 connected to the LAN of the office 10 or 20 according to the search order acquired in step 137. Alternatively, one of the storage devices 111 included in the MFP 100 itself is searched in order. If the electronic file is not found in step 125, if it is found but is a so-called image file represented by PDF or tiff, or if pointer information itself does not exist, the process branches to step 126.

  Step 126 is a so-called document search processing routine.

  First, in step 122, a word is extracted from the result of OCR performed on each character block and a full text search is performed, or a so-called layout search is performed from the array of each object and the attribute of each object. As a result of searching in the document management server search order obtained in step 137, if an electronic file with a high degree of similarity is found, a thumbnail or the like is displayed (step 127), and an operator needs to be selected from a plurality of operators. The file is specified by the input operation. If the candidate is one file, the process automatically branches from step 128 to step 134 to notify the storage address. If the electronic file is not found in the search process of step 126, or if it is found but is a so-called image file represented by PDF or tiff, the process branches to step 129.

  Step 129 is a conversion processing unit from image data to vector data, which converts the image data into an electronic file close to the original electronic file. First, for the character block that has been OCR in step 122, the character size, style, and font are further recognized and converted into font data that is visually faithful to the character obtained by scanning the document. Tables and graphic blocks composed of one line are outlined. The image block is processed as an individual JPEG file as image data. These vectorization processes are performed for each object, and further, layout information of each object is stored, converted into, for example, rtf (step 130), and stored in the storage device 111 as an electronic file (step 131).

  First, in step 132, index information for search is generated and added to the search index file so that the vectorized document image can be directly searched as an electronic file when the same processing is performed thereafter. Further, if it is determined in step 136 that the process that the operator wants to perform is recording, the process branches to step 133 to add pointer information to the file as image data. Similarly, when the electronic file can be specified by the search process, the process branches from step 128 to step 134 in order to directly specify the electronic file. In the case where the storage address is notified to the operator and recorded on the current paper, Similarly, pointer information is added to the electronic file.

  If the electronic file can be specified from the pointer information in step 125, if the electronic file can be specified by the search process, or converted into an electronic file by vectorization, the storage address of the electronic file is notified to the operator in step 134. To do.

  Note that, using the electronic file itself obtained by the present invention as described above, for example, processing, storage, transmission and recording of a document can be performed in step 135. These processes reduce the amount of information, increase the storage efficiency, shorten the transmission time, and are very advantageous as high-quality data when recording and displaying, compared to the case of using image data.

  Hereinafter, each processing block will be described in detail.

  First, the block selection process shown in step 121 will be described.

  The block selection process is as follows.

In the block selection process, the image data of one page read in step 120 shown on the right in FIG. 6 is recognized as a block for each object, as shown on the left, and each block is designated as a character / drawing / photo / line / This is a process of determining an attribute such as a table and dividing it into areas having different attributes.
An example of the block selection process will be described below.

  First, the input image is binarized into black and white, and contour tracking is performed to extract a block of pixels surrounded by a black pixel contour. For a black pixel block with a large area, contour tracing is also performed for white pixels inside, and a white pixel block is extracted, and the black pixel block is recursively extracted from the white pixel block with a certain area or more. Extract lumps.

  The black pixel blocks obtained in this way are classified by size and shape, and are classified into regions having different attributes. For example, if the aspect ratio is close to 1 and the size is within a certain range, the pixel block corresponding to the character is used, the portion where the adjacent characters can be grouped in a well-aligned manner is the character region, and the flat pixel block is the line region. The area occupied by the black pixel block that is larger than the size and contains the square white pixel block in a well-aligned manner is the table region, the region where the irregular pixel block is scattered is the photo region, and the pixel block of any other shape is used. A drawing area, etc.

  FIG. 7 shows block information for each block obtained by the block selection process.

  Information for each block is used as information for vectorization or search described below.

  Detection of pointer information is as follows.

  Next, the OCR / OMR process for extracting the file storage position shown in step 122 from the image information will be described.

  FIG. 8 is a flowchart showing a process of decoding a two-dimensional barcode (QR code symbol) added to a document image and outputting a data character string. An example of a document 310 to which a two-dimensional barcode is added is shown in FIG.

  First, an image representing the original 310 stored in the page memory in the data processing device 115 is scanned by a CPU (not shown), and a predetermined two-dimensional barcode symbol 311 is obtained from the result of the block selection process described above. Detect position. The QR code position detection pattern is composed of the same position detection element patterns arranged in three of the four corners of the symbol (step 300).

  Next, the format information adjacent to the position detection pattern is restored, and the error correction level and mask pattern applied to the symbol are obtained (step 301).

  After determining the symbol model number (step 302), the mask process is canceled by performing an XOR operation on the encoded area bit pattern using the mask pattern obtained from the format information (step 303).

  The symbol character is read in accordance with the arrangement rule corresponding to the model, and the message data and the error correction code word are restored (step 304).

  It is detected whether or not there is an error on the restored code (step 305). If an error is detected, the process branches to step 306 and is corrected.

  Based on the mode indicator and the character number indicator, the data code word is divided into segments from the error-corrected data (step 307).

  Finally, the data character is decoded based on the specification mode, and the result is output (step 308).

  The data incorporated in the two-dimensional bar code represents the address information of the corresponding file, and is composed of path information including a file server name and a file name, for example. Alternatively, it consists of a URL to the corresponding file.

  In the present embodiment, the document 310 to which pointer information is assigned using a two-dimensional barcode has been described. However, when pointer information is directly recorded as a character string, a block of a character string according to a predetermined rule is replaced with the previous block. The address information of the original file can be obtained directly by detecting each character of the character string indicating the pointer information by detecting the selection process.

  Alternatively, pointer information can be obtained by adding a modulation that is difficult to visually recognize the character block 312 or the character string of the character block 312 of the document 310 in FIG. Can be granted. As for the so-called watermark information, pointer information can be obtained by detecting an interval between characters when performing character recognition processing described later. It is also possible to add pointer information as a digital watermark in the natural image 314.

  The process for assigning priorities to the servers to be searched is as follows.

  Next, a method for assigning priorities to the document management server for searching for electronic files will be described with reference to FIGS.

  First, processing on the document management server side will be described with reference to FIG.

  On the document management server side, when a document is newly registered in the storage device, it is determined whether or not the document is image data (step 140). If it is not image data, the character string used most frequently in the document is extracted as a keyword character string by using character recognition processing (step 146). Information associating the extracted keyword character string with its own host name or its own IP address is stored (step 143). The information stored in step 143 is transmitted to MFP 100 in FIG. 10 (step 144), and the transmitted information is stored in the file on the MFP side (step 145).

  In step 140, if the registered document is image data, the image data is vectorized (step 141), and the character string used most frequently is extracted from the document as a keyword character string. (Step 142).

  Next, processing on the MFP side will be described with reference to FIG.

  By using character recognition processing for the result of BS processing (step 121) and OCR (step 122) processing on the input image data, the most frequently used image data is used. The extracted character string is extracted as a keyword character string (step 150). The extracted keyword character string is compared with the keyword character string in the file saved in step 145 (step 151), and the priority of which document management server should be searched according to the matching degree of the character string is determined. Turn on (step 152).

  The file search based on the pointer information is as follows.

  Next, retrieval of an electronic file from pointer information will be described using the flowchart of FIG.

First, the file server is specified based on the address included in the pointer information. (Step 400)
Here, the file server refers to the client PC 102, the document management server 106 including the database 105, and the MFP 100 itself including the storage device 111.

  Here, the address is URL or path information including a server name and a file name.

  If the file server can be identified, the address is transferred to the file server (step 401). When the file server receives the address, it searches for the corresponding file (step 402). If the file does not exist (step 403-N), the MFP is notified.

  If the file exists (step 403-Y), as described with reference to FIG. 3, the file address is notified (step 134), and if the processing desired by the user is acquisition of image file data, the MFP (Step 408).

  The file search process is as follows.

  Next, details of the file search process shown in step 126 of FIG. 3 will be described with reference to FIGS.

  As described above, the process of step 126 is performed when there is no pointer information in the input document (input file) in step 124, or when the pointer information is present but the electronic file is not found, or when the electronic file is not found. Performed when the file is an image file.

  Here, it is assumed that each block and input file extracted as a result of step 122 includes information (block information and input file information) shown in FIG. Examples of information contents include attributes, coordinate positions, width and height sizes, and presence / absence of OCR information. Attributes are classified into characters, lines, photographs, pictures, tables and others. Further, for the sake of simple explanation, the blocks are arranged in the order of the smallest coordinate X, that is, in the order of (for example, X1 <X2 <X3 <X4 <X5 <X6), block 1, block 2, block 3, block 4, block 5, block 6 Named. The total number of blocks is the total number of blocks in the input file. In the case of FIG.

  FIG. 12 shows a flowchart for performing a layout search for a file similar to the input file from the database using these pieces of information. Here, it is assumed that the database file includes the same information as in FIG.

  The flow of the flowchart is to sequentially compare the input file and the file in the database. First, in step 510, initialization such as a similarity rate described later is performed. Next, in step 511, the total number of blocks is compared. If true, the information of the blocks in the file is further sequentially compared. In block information comparison, an attribute similarity rate, a size similarity rate, and an OCR similarity rate are calculated in steps 513, 515, and 518, respectively, and an overall similarity rate is calculated based on them in step 522. About the calculation method of each similarity rate, since a well-known technique is used, description is abbreviate | omitted.

  In step 523, if the overall similarity is higher than a preset threshold Th, the file is listed as a similarity candidate in step 524. However, N, W, and H in the figure are the total number of blocks of the input file, each block width, and each block height, and ΔN, ΔW, and ΔH consider errors based on the block information of the input file. n, w, and h are the total number of blocks in the database file, each block width, and each block height. Further, although not shown, the position information XY may be compared at the time of size comparison in step 514.

  As described above, as a result of the search, the similarity is higher than the threshold value Th, and the database file (step 524) stored as a candidate is displayed as a thumbnail or the like (step 127). If the operator needs to be selected from a plurality of files, the file is specified by the operator's input operation.

  The vectorization process is as follows.

  When the original file does not exist in the file server, the image data shown in FIG. 20 is vectorized for each block. Next, the vectorization shown in step 129 will be described in detail. First, character recognition processing is performed on each character in the character block.

"Character recognition"
The character recognition unit recognizes an image cut out in character units using a pattern matching method, and obtains a corresponding character code. This recognition process recognizes the character type with the closest distance by comparing the observed feature vector obtained by converting the feature obtained from the character image into a numerical sequence of several tens of dimensions and the dictionary feature vector obtained for each character type in advance. The resulting process. There are various known methods for extracting a feature vector. For example, there is a method characterized by dividing a character into meshes, and using a mesh number-dimensional vector obtained by counting character lines in each mesh as line elements according to directions.

  When character recognition is performed on the character area extracted in block selection (step 121), first, horizontal writing and vertical writing are determined for the corresponding area, lines are cut out in the corresponding directions, and then characters are cut out. Get a character image. Horizontal / vertical writing can be determined by taking a horizontal / vertical projection of the pixel value in the corresponding area, and determining that the horizontal projection area is large when the horizontal projection variance is large, and vertical writing area when the vertical projection variance is large. . For horizontal writing, character strings and characters are decomposed by cutting out lines using horizontal projection, and then cutting out characters from the vertical projection of the cut lines. For vertically written character areas, horizontal and vertical may be reversed. At this time, the character size can be detected.

"Font recognition"
Multiple character feature vectors used for character recognition are prepared for the character shape type, that is, font type, and the font type is output along with the character code when matching. I can do it.

"Vectorization of characters"
Using the character code and font information obtained by the character recognition and font recognition, the character portion information is converted into vector data using outline data prepared in advance. When the original document is color, the color of each character is extracted from the color image and recorded together with vector data.

  Through the above processing, the image information belonging to the character block can be converted into vector data whose shape, size and color are almost faithful.

"Vectorization of non-character parts"
In the block selection process (step 121), the outline of the pixel block extracted therein is converted into vector data for a region that is a drawing, line, or table region.

  Specifically, a point sequence of pixels forming an outline is divided by points regarded as corners, and each section is approximated by a partial straight line or curve. The corner is a point where the curvature is maximized. As shown in FIG. 13, the point where the curvature is maximized is that a string is placed between k points Pi-k and Pi + k that are separated from the arbitrary point Pi by left and right. It is obtained as a point where the distance between the string and Pi becomes maximum when drawn. Furthermore, let R be the chord length / arc length between Pi−k and Pi + k, and a point where the value of R is equal to or less than a threshold value can be regarded as a corner. Each section after being divided by the corners can be vectorized by using a least square method for a straight line and a curve using a cubic spline function.

  Further, when the target has an inner contour, it is similarly approximated by a partial straight line or a curve using the point sequence of the white pixel contour extracted by block selection.

  As described above, the outline of a figure having an arbitrary shape can be vectorized by using the contour line approximation. If the original document is in color, the figure color is extracted from the color image and recorded together with vector data.

  Furthermore, as shown in FIG. 14, when an outer contour and an inner contour or another outer contour are close to each other in a certain section, two contour lines can be combined and expressed as a line having a thickness. . Specifically, when a line is drawn from each point Pi of a certain contour to a point Qi that is the shortest distance on another contour, and each distance PQi is on average less than or equal to a certain length, the interval of interest has a midpoint PQi as a point sequence The line is approximated by a straight line or a curve, and its thickness is the average value of PQi. A table ruled line that is a line or a set of lines can be efficiently expressed as a set of lines having the above-described thickness.

  The vectorization using the character recognition process for the character block has been described above. As a result of the character recognition process, the character having the closest distance from the dictionary is used as the recognition result, but this distance is not less than a predetermined value. Are not necessarily identical to the original characters and are often erroneously recognized as characters having similar shapes. Therefore, in the present invention, as described above, such characters are handled in the same manner as general line drawings, and the characters are outlined. That is, even a character that is erroneously recognized by the conventional character recognition process is not vectorized into an erroneous character, and can be vectorized by outline rendering that is visually faithful to image data.

  In the present invention, since it is not possible to vectorize a block determined to be a photograph, it is left as image data.

  Figure recognition is as follows.

  A process of grouping the vectorized dividing lines for each graphic object after vectorizing the outline of the arbitrarily shaped graphic as described above will be described.

  FIG. 15 shows a flowchart until the vector data is grouped for each graphic object. First, the start point and end point of each vector data are calculated (700). Next, a graphic element is detected using the start point and end point information of each vector (701). The detection of a graphic element is to detect a closed graphic formed by a dividing line. In detection, the detection is performed by applying the principle that each vector constituting the closed shape has vectors connected to both ends thereof. Next, other graphic elements or dividing lines existing in the graphic element are grouped to form one graphic object (702). If there is no other graphic element or dividing line in the graphic element, the graphic element is set as a graphic object.

  FIG. 16 shows a flowchart for detecting a graphic element. First, unnecessary vectors not connected to both ends are removed from the vector data, and a closed graphic component vector is extracted (710). Next, from the closed figure constituent vectors, the starting point of the vector is used as a starting point, and the vectors are sequentially followed in the clockwise direction. The process is repeated until the start point is returned, and all the passed vectors are grouped as a closed graphic constituting one graphic element (711).

  In addition, all closed graphic constituent vectors inside the closed graphic are also grouped. Further, the same processing is repeated with the starting point of a vector not yet grouped as a starting point. Finally, among the unnecessary vectors removed at 710, those joined to the vector grouped as a closed figure at 711 are detected and grouped as one figure element (712).

  As described above, the graphic block can be handled as an individual graphic object that can be reused individually.

  The conversion process to application data is as follows.

  By the way, the image data for one page is subjected to block selection processing (121), and the result of vectorization processing (129) is converted as an intermediate data format file as shown in FIG. 79. It is called Document Analysis Output Format (DAOF).

  FIG. 17 shows the data structure of DAOF.

  In FIG. 17, reference numeral 791 denotes a header, which holds information relating to document image data to be processed. In the layout description data portion 792, attributes such as TEXT (character), TITLE (title), CAPTION (caption), LINEART (line drawing), EPICTURE (natural image), FRAME (frame), TABLE (table), etc. in the document image data. The attribute information of each block recognized every time and its rectangular address information are held. The character recognition description data portion 793 holds character recognition results obtained by character recognition of TEXT blocks such as TEXT, TITLE, and CAPTION. The table description data portion 794 stores details of the structure of the TABLE block. The image description data portion 795 cuts out image data of blocks such as PICTURE and LINEART from the document image data and holds them.

  Such a DAOF is not only used as intermediate data but may be stored as a file itself. In this file state, it is not possible to reuse individual objects in a so-called general document creation application. I can't. Therefore, the process 130 for converting DAOF into application data will be described in detail.

  FIG. 18 is an overall schematic flow.

  In 8000, DAOF data is input.

  8002 generates a document structure tree that is the source of application data.

  In step 8004, actual data in the DAOF is poured based on the document structure tree to generate actual application data.

  FIG. 19 is a detailed flow of the 8002 document structure tree generation unit, and FIG. 82 is an explanatory diagram of the document structure tree. As a basic rule of overall control, the flow of processing shifts from a micro block (single block) to a macro block (an aggregate of blocks).

  Hereinafter, the block refers to the micro block and the entire macro block.

  8100 performs regrouping based on the relevance in the vertical direction in units of blocks. Immediately after the start, judgment is made in units of micro blocks.

  Here, the relevance can be defined by the fact that the distance is close and the block width (height in the horizontal direction) is substantially the same.

  Information such as distance, width, and height is extracted with reference to GAOF.

  FIG. 20A shows an actual page configuration, and FIG. 20B shows its document structure tree. As a result of 8100, T3, T4, and T5 are generated as one group V1, and T6 and T7 are generated as one group V2 in the same hierarchy.

  8102 checks whether or not there is a separator in the vertical direction. For example, the separator is physically an object having a line attribute in the DAOF. Also, logically, it is an element that explicitly divides a block in the application. If a separator is detected here, it is subdivided at the same level.

  8104 uses the group length to determine whether there are no more divisions.

  If the group length in the vertical direction is the page height, the document structure tree generation ends.

  In the case of FIG. 20, since there is no separator and the group height is not the page height, the process proceeds to 8106.

  In step 8106, regrouping is performed based on the relevance in the horizontal direction in units of blocks. Here too, the first time immediately after the start is determined in units of microblocks.

  The definition of the relevance and the determination information is the same as in the vertical direction.

  In the case of FIG. 20, H1 is generated as T1 and T2 and H2 is generated as V1 and V2 as a group in the same hierarchy one above V1 and V2.

  8108 checks for the presence of a horizontal separator.

  In FIG. 20, since there is S1, this is registered in the tree, and a hierarchy of H1, S1, and H2 is generated.

  8110 uses the group length to determine whether there are no more divisions.

  If the horizontal group length is the page width, the document structure tree generation ends.

  If not, the process returns to 8102, and the relevance check in the vertical direction is repeated again at the next higher level.

  In the case of FIG. 20, since the division width is the page width, the process ends here, and finally V0 of the highest hierarchy representing the entire page is added to the document structure tree.

  After the document structure tree is completed, application data is generated in 8006 based on the information.

  In the case of FIG. 20, the details are as follows.

  That is, since there are two blocks T1 and T2 in the horizontal direction, H1 has two columns, and after T1 internal information (refer to DAOF, text of character recognition result, image, etc.) is output, the column is changed and the internal of T2 Information is output, and then S1 is output.

  Since H2 has two blocks V1 and V2 in the horizontal direction, it outputs as 2 columns, V1 outputs its internal information in the order of T3, T4, T5, then changes the column and outputs the internal information of T2, T7 of V2 To do.

  As described above, conversion processing to application data can be performed.

  The addition of pointer information is as follows.

  Next, the pointer information addition process shown in step 133 will be described.

  When the document to be processed is specified by the search process, or when the original file can be reproduced by vectorization, when the document is recorded, pointer information is added when recording on paper. Thus, the original file data can be easily obtained when various processes are performed again using this document.

  FIG. 21 is a flowchart showing a process of encoding a data character string as pointer information with a two-dimensional barcode (QR code symbol: JIS X0510) 311 and adding it to an image.

  The data incorporated in the two-dimensional barcode represents the address information of the corresponding file, and is composed of path information including a file server name and a file name, for example. Alternatively, the URL includes a URL to the corresponding file, a file ID managed in the database 105 in which the corresponding file is stored, or a storage device included in the MFP 100 itself.

  First, an input data string is analyzed in order to identify different characters to be encoded. Further, an error detection and error correction level is selected, and a minimum model number that can accommodate input data is selected (step 900).

  Next, the input data string is converted into a predetermined bit string, and an indicator indicating a data mode (numeric, alphanumeric, 8-bit byte, kanji, etc.) and a termination pattern are added as necessary. Further, it is converted into a predetermined bit code word (step 901).

  At this time, in order to perform error correction, the code word string is divided into a predetermined number of blocks according to the model number and the error correction level, an error correction code word is generated for each block, and is added after the data code word string ( Step 902).

  The data code words of the respective blocks obtained in step 902 are connected, and the error correction code word of each block and, if necessary, the remainder code word are followed (step 903).

  Next, the code word module is arranged in the matrix together with the position detection pattern, separation pattern, timing pattern, alignment pattern, and the like (step 904).

  Further, an optimal mask pattern is selected for the symbol coding region, and the mask processing pattern is converted into a module obtained in step 904 by an XOR operation (step 905).

  Finally, format information and model number information are generated in the module obtained in step 905 to complete a two-dimensional code symbol (step 906).

  The above-described two-dimensional barcode with address information embedded can be recorded in the data processing apparatus 115 when, for example, an electronic file is formed as print data from the client PC 102 on the recording apparatus 112 as a recorded image on paper. After being converted into raster data, it is added to a predetermined location on the raster data to form an image. The user who has distributed the image-formed paper here can read the image reading unit 110 to detect the storage location of the original electronic file from the pointer information in step 123 described above.

  In addition to the two-dimensional barcode described in this embodiment, the means for adding additional information for the same purpose is, for example, a method of adding pointer information directly to a document as a character string, a character string in a document, A method generally referred to as a digital watermark, such as a method of embedding information by modulating a character spacing and a method of embedding in a halftone image in a document, can be applied.

[Example 1]
Another embodiment for the process of assigning priority to the server to be searched will be described.

  In the previous embodiment, when giving priority to the document management server to be searched, the character string most frequently used in the document is extracted and the matching degree is used. The degree of matching of the image can be used by performing the degree of matching based on the document layout using block selection or vectorizing the image data inserted in the document.

  That is, a document layout structure can be obtained by performing block selection processing on the input image data. Also, by vectorizing the image data in the document, it is possible to digitize the characteristics of the image data. With these methods, it is possible to compare not only character strings but also layouts and images.

1 is a block diagram illustrating a configuration example of an image processing system according to the present invention. 2 is a configuration diagram of an MFP 100. FIG. It is a flowchart figure which shows the outline | summary of the whole image processing by this invention. It is a flowchart figure which shows the process by the document management server side. FIG. 6 is a flowchart showing processing on the MFP side. It is a figure which shows a block selection process. It is a figure which shows the block information with respect to each block obtained by the block selection process. 5 is a flowchart illustrating a process of decoding a two-dimensional barcode (QR code symbol) added to a document image and outputting a data character string. FIG. 4 is a diagram illustrating an example of a document 310 to which a two-dimensional barcode is added. It is a flowchart at the time of searching an electronic file from pointer information. 3 is a flowchart according to the present invention. It is a flowchart which performs the layout search of the file similar to an input file from the database. It is a figure which shows the maximum point at the time of vectorizing parts other than a character. It is a figure which shows a part of block selection process. It is a flowchart which shows a graphical object recognition process. It is a flowchart which shows a graphic element detection process. It is a figure which shows the data structure of DAOF. It is a flowchart which shows application data conversion. It is a flowchart which shows document structure tree creation. It is a figure for demonstrating a document structure tree. It is a flowchart which shows the process in which the data character string as pointer information is encoded by a two-dimensional barcode (QR code symbol: JIS X0510) 311 and added to an image.

Explanation of symbols

10 Office 13 Proxy server 20 Office 104 Internet 107 LAN
100 MFP
101 Management PC
102 Client PC (external storage means)
103 Proxy server 104 Internet 105 Database 106 Document management server 107 LAN
108 LAN
109 LAN
100 MFP

Claims (11)

  From the electronic file stored in the document management server by using the means for reading and scanning the original, the specifying means for specifying the electronic file of the original from the image information obtained by the means, and the vectorizing means Means for extracting the characteristics of the electronic file, means for extracting the characteristics of the original read by using the vectorization means, means for comparing the characteristics of the electronic file with the characteristics of the read original, and calculating the similarity, An image processing system comprising: means for assigning a priority to a document management server to be searched according to the degree of similarity; and means for performing a search in order from the document management server with the highest priority.   2. An image processing system according to claim 1, wherein said specifying means includes means for recognizing a storage location of an electronic file additionally recorded on a document.   2. The image processing system according to claim 1, wherein the specifying means includes means for searching for specific information described in the document from a file stored in the storage means.   2. The image processing system according to claim 1, wherein the vectorization means includes OCR means for OCRing characters in a document.   The image processing system according to claim 1, wherein the vectorization unit divides the document into a plurality of objects and independently vectorizes each object.   2. The image processing system according to claim 1, wherein the vectorization means includes format conversion means for converting a vectorized object into a default format that can be handled by existing document creation software.   2. The image processing system according to claim 1, wherein the means for extracting the feature performs character recognition using the OCR of the vectorization means and extracts a character string used most frequently as a feature.   2. The image processing system according to claim 1, wherein the means for extracting the feature performs character recognition using the OCR of the vectorizing means and extracts a character string having the largest font as a feature.   2. The image processing system according to claim 1, wherein said means for extracting the feature calculates what shape and size of an object is included by using the object division of said vectorization means and extracts it as a feature. .   2. The image processing system according to claim 1, wherein the means for extracting the features digitizes the image data by using the vectorizing means to digitize the features of the image itself and extract them as features.   The feature of the image data read by scanning and scanning with the electronic document in the document management server is extracted by using any one of the feature extraction means from Item 7 to Item 10, and the degree of similarity is extracted. The image processing system according to claim 1, wherein searches are performed in descending order.