JPH0728935A - Document image processor - Google Patents

Abstract

PURPOSE:To provide a document image processor which can accurately extract the areas based on the previously prepared layout information. CONSTITUTION:An area extracting part 2 extracts each area on a pertinent document image out of the document image acquired by a reading part 1. A layout identifying part 4 compares the area extracting result of the part 2 with a singular or plural layout information stored previously in a layout information storing part 3. Then the area extracting result of the part 2 is corrected based on the coincident layout information, and a character area is extracted. The characters of the extracted character area are recognized by a character recognition part 6, and this recognition result is stored in a recognizing result storing part 7. The areas other than the character areas are stored in an image data storing part 5 as the image data. Meanwhile the area extracting result is stored in the part 3 when there is no coincident layout information.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION The present invention relates to a character recognition device (OC).
The present invention relates to a document image processing device used for R), facsimile, and the like.

[0002]

2. Description of the Related Art When recognizing characters in a document including pictures, photographs, etc. in addition to characters, it is necessary to extract only the character portion from the document image. A device for processing such a document image is a document image processing device. 2 and 3
FIG. 7 is an explanatory diagram of an example of conventional document image processing. Figure 2
(A) and the form 11 to be read shown in FIG.
From 21, the straight line image shown in FIG. 2 (b) and FIG. 3 (b) is detected. Then, the frame structure is extracted from these straight line groups, the reading field is determined by matching with the frame structure information stored in advance, and the characters in this field are recognized (see Japanese Patent Laid-Open No. 2-217977).

[0003]

However, the above-mentioned conventional technique has the following problems. That is, in the above-described conventional technique, only the form having the straight line image as shown in FIG. 2B and FIG. 3B is the processing target, and the straight line image does not always exist like the general document. The document could not be processed. For example, as shown in FIG. 4, when the document written in the character area spans a plurality of columns, there is a problem that the document cannot be processed. The present invention has been made by paying attention to the above points, and even when there is no such a linear image as described above, it is possible to determine the area to be the character recognition target, so that the area extraction and the character recognition can be performed accurately. It is an object of the present invention to provide a document image processing apparatus which has been described above.

[0004]

A document image processing apparatus according to the present invention includes a plurality of or a single piece of layout information indicating the attributes of each area arranged on a document image and an area extraction unit for extracting each area of the document image. Of the layout information storage section to be stored and the layout information stored in the layout information storage section, the one that most closely matches the area extraction result by the area extraction section is extracted, and if there is no matching layout information, the area extraction is performed. The layout storage unit stores the result as new layout information in the layout information storage unit, and if there is matching layout information, the layout identification unit corrects the area extraction result according to the layout information. .

[0005]

In the document image processing apparatus of the present invention, the area extracting section extracts each area on the document image from the document image obtained by the reading section. Next, the layout identification unit compares the area extraction result with a plurality of or a single piece of layout information stored in advance in the layout information storage unit. Then, the area extraction result of the area extracting unit is corrected according to the matched layout information, and the character area is extracted.
With respect to this character area, the character recognition unit performs character recognition, and the result is stored in the recognition result storage unit. Areas other than characters are stored in the image data storage unit as image data.

[0006]

Embodiments of the present invention will now be described in detail with reference to the drawings. FIG. 1 is a block diagram of an embodiment of the document image processing apparatus of the present invention. The illustrated apparatus includes a reading unit 1, a document image processing device 10, an image data storage unit 5, a character recognition unit 6, and a recognition result storage unit 7. The reading unit 1 is composed of a CCD line sensor or the like, and electrically scans a document to be processed in the main scanning direction and mechanically scans it in the sub scanning direction to read it. The reading unit 1 is provided with, for example, a known sheet feeder (not shown). The document image processing apparatus 10 includes an area extraction unit 2, a layout information storage unit 3,
It comprises a layout identification section 4.

The area extracting unit 2 extracts an area of the image read by the reading unit 1. This process is performed as follows, for example. As shown in FIG. 4, areas such as characters and figures are automatically extracted from the binarized image data. Therefore, from this image data, as shown in FIG.
As shown in (a) and (b), a blank area having a length and width equal to or larger than a predetermined threshold value is traced to detect the area, and as shown in FIG. It is stored as a combination of. The layout information storage unit 3 is composed of a floppy disk or the like, and stores a plurality of layout information in advance. The layout information may be singular or may be empty in the initial state.

FIG. 6 shows each area of the document image, and FIG.
Shows an example of layout information corresponding to the document image of FIG. For example, in the first row of the table shown in FIG. 7, the region 61 of FIG. 6 is defined as xs = 100, ys = 300, xe = 160.
It is specified as a region divided by each straight line of 0 and ye = 1000. Then, the attribute of the area is indicated by flag to be a character (horizontal writing). The attribute is composed of, for example, 6-bit data, a bit of "1" indicates a character (horizontal writing), a bit of "2" indicates a character (vertical writing), and a bit of "4" indicates a figure. Also, the "8" bit indicates a photograph, the "16" bit indicates a ruled line, and the "32" bit indicates a table.

Therefore, in the second row of the table shown in FIG. 7, the region 62 of FIG. 6 has xs = 1650, ys = 300, and xe = 3.
It is a region partitioned by each straight line of 300 and ye = 1500, and its attribute is indicated by flag as a figure.
Also, in the third row of the table shown in FIG. 7, the area 63 in FIG.
s = 100, ys = 1200, xe = 1600, ye =
It is an area divided by 3000 straight lines, and its attributes are
The flag indicates that it is a photo. In the fourth row of the table shown in FIG. 7, the area of FIG. 6 is xs = 1650, ys = 17.
The area is defined by each straight line of 00, xe = 3300, and ye = 4600, and its attribute is indicated by a flag as a character (horizontal writing). The fifth row of the table shown in FIG. 7 is
The area of FIG. 6 is xs = 100, ys = 3100, xe =
The area is defined by each straight line of 1600 and ye = 4600, and its attribute is indicated by a flag as a character (horizontal writing).

The layout information shown in FIG. 7 may include items such as the reading order of character areas. For example, when the sentences are written in the order of regions 61, 65, and 64 in FIG. 6, by indicating this order, the character recognition can be automatically performed according to this order.

The layout identification section 4 stores a plurality of layout information stored in advance in the layout information storage section 3,
The region extraction result obtained by the region extraction unit 2 is compared. As a result of identifying the matching layout information, if it is necessary to correct the area extraction result, the area extraction result is corrected according to the layout information, and areas other than characters are stored in the image data storage unit 5 as image data. On the other hand, the character area is stored in the recognition result storage unit 7 as text data after the character recognition by the character recognition unit 6. Further, the similarity is obtained for each layout information by comparing the layout information in the layout identification unit 4 with the area extraction result. Then, the layout information having the highest degree of similarity is set as the matching layout. The area extraction result is corrected according to this layout information.

First, a certain area stored in the layout information is set as a known area, and the coordinates of the center point of this known area are obtained by the following equation. {(Xs + xe) / 2, (ys + ye) / 2} On the other hand, among the regions (extracted regions) obtained as a result of region extraction, the center point is calculated by the above formula for all the extracted regions having the same attributes as the known region. . Then, the distances between the center points of the known areas and the center points of all these extraction areas are obtained. Then, the extraction region having the smallest distance is set as the region corresponding to the known region. Note that there are extraction regions that do not correspond to any known regions shown in FIG. 8B, such as the extraction regions 71 and 75 shown in FIG. 8A. The above processing is performed for all known areas.

In these processes, the similarity between the area extraction result and the layout information is "0" in the following two cases.
(No match), and the subsequent processing is not performed. (1) When there is no extraction region having the same attribute as the known region, that is, when there is no extraction region corresponding to the known region, it is determined that they do not match. (2) When a certain extraction area corresponds to a plurality of known areas, they are not matched for the sake of similarity calculation described later.

In cases other than the above two cases (1) and (2), the degree of similarity between the area extraction result and the layout information is obtained as follows. First, the area of the overlapping portion of the known area and the corresponding extraction area is obtained. This area is
It can be easily obtained from the coordinates of both areas. Then, the degree of similarity is calculated by the following equation. Similarity = (sum of areas of overlapping portions) / (sum of areas of all known regions) × 100 The above processing is performed for all the layout information stored in the layout information storage unit 3. Then, the layout having the highest degree of similarity is set as the matching layout. However, the similarity of the matching layout has a certain value (TH
L) In the following cases, the area extraction result is stored in the layout information storage unit 3 as new layout information without matching layout. In this case, since the area extraction result is used as it is, the following correction processing is not performed.

Finally, the area extraction result is corrected according to the layout information determined to be the matching layout in the above processing. FIG. 8 is an explanatory diagram of an example of the correction process. The upper left and lower right character areas of the matching layout information are each divided into two areas in the area extraction result. Therefore, there is a need to integrate those two areas. In this case, the similarity between the extraction area corresponding to the known area and the other areas is 1
Integration is performed with the combination that is closest to 00%. Here, the area that does not correspond to the known area is an area that does not have an overlapping portion with the known area. It is necessary to integrate such a region with the extraction region corresponding to the known region. However, integration is not performed in such a combination that the known area and the extracted area do not have a one-to-one correspondence. Of the areas obtained as a result of the correction, the areas other than the characters are stored in the image data storage unit 5 as image data.

On the other hand, of the corrected area extraction results obtained by the layout identifying section 4, for the character area, the character recognizing section 6 cuts out the characters one by one, recognizes the character, and then recognizes the character. The character code obtained as a result is stored in the recognition result storage unit 7. FIG. 9 shows an outline of the processing procedure of the layout identification unit described above.

First, in step S1, an extraction area of a document image corresponding to a known area of layout information is obtained. After obtaining this correspondence, in step S2, the degree of similarity between the layout information and the document image is obtained. This similarity is obtained for all layout information. In step S3, it is determined whether or not the maximum similarity calculated in step S2 is less than or equal to a certain value. Information. On the other hand, if the maximum similarity calculated in step S2 is not less than a certain value, the layout information is set as layout information that matches the area extraction result,
In step S4, the area extraction result is corrected according to the layout information.

Next, the operation of the above-mentioned device will be described. First, the reading unit 1 reads a document image. Area extraction unit 2
Then, each area is extracted from this document image by a known method as shown in FIG. The layout information storage unit 3 stores layout information as shown in FIG. The layout identification unit 4 performs processing according to the flowchart of FIG. Comparing FIG. 8A and FIG. 8B, it is the known region 81 that has the center point closest to the center point of the extraction region 71. Therefore, the extraction area 72
The known area 81 corresponds to. Similarly, a known area 82 corresponds to the extraction area 73, and a known area 83 corresponds to the extraction area 74. Further, the known region 84 corresponds to the extraction region 75, and the known region 84 corresponds to the extraction region 76. Further, a known area 85 corresponds to the extraction area 77.

Next, the area extraction result of FIG.
The similarity of the layout information of (b) is calculated. For this reason,
First, the area of each overlapping portion of the extraction regions 71 and 72 and the known region 81 is obtained. Similarly, the areas of the respective overlapping portions of the extraction regions 73 and 74 and the known regions 82 and 83 are obtained. Also,
The area of each overlapping portion of the extraction regions 75 and 76 and the known region 84 is obtained. Further, the area of the overlapping portion of the extraction region 77 and the known region 85 is obtained. Then, the sum of the areas of these overlapping portions is obtained. On the other hand, known areas 81, 82, 83, 8
The areas of 4, 85 are calculated, and the sum of them is calculated. The sum of the areas of the known regions may be obtained in advance and included in the layout information.

Then, the degree of similarity is obtained by dividing the sum of the areas of the overlapping portions of the known area and the extraction area by the sum of the areas of the known areas. If the degree of similarity is 100%, the layout of the area extraction result completely matches the known layout. However, even if they do not completely match, if the degree of similarity is large to some extent, it can be considered that they are included in the error range of region extraction and match the layout of the layout information. In the case of FIG. 8, it is considered that the layouts match.

Therefore, the area extraction result of FIG. 8A is corrected according to FIG. 8B. That is, of the plurality of extraction areas in FIG. 8A, one corresponding to one known area in FIG. 8B is integrated. For example, the extraction area 71 is the extraction area 7
2 is integrated into one extraction area 91 shown in FIG. On the other hand, the extraction area 75 includes the extraction area 73 and the extraction area 7.
Although the attribute of the extraction area 75 is a character, the attribute of the extraction area 73 is a graphic, and the attribute of the extraction area 76 is a character, it is integrated with the extraction area 76. Unlike the case shown in FIG. 8A, the extraction area 7
Even when the attribute of 3 is also a character, the degree of similarity when integrated with the extraction area 76 is higher than the degree of similarity when integrated with the extraction area 73, so it is integrated with the extraction area 76. And
One extraction area 94 shown in FIG. 8C is used.

As a result, referring to the attribute of the layout information of FIG. 8B, the extraction area 91 corresponding to the known area 81,
The extraction area 94 corresponding to the known area 84 and the extraction area 95 corresponding to the known area 85 are respectively taken out as character areas, and characters can be cut out from these areas for character recognition. Therefore, it is not necessary to check the attributes of all the extracted areas obtained as a result of area extraction, and rapid document image processing can be performed. Also, FIG.
It is possible to prevent the extraction areas 92 and 93 shown in (c) from being erroneously recognized as character areas.

It is needless to say that the present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the matters described in the claims. For example, in the comparison between the area extraction result and the layout information, the distance between the center point of the extraction area and the center point of the known area is compared. However, the present invention is not limited to this. You may make it compare the distance of a vertex.

[0024]

As described above, according to the document image processing apparatus of the present invention, the best match of the area extraction result is automatically selected from the prepared layout information, and the area extraction result is selected accordingly. Since the image processing is modified so that the image processing is performed, the image processing can be performed quickly and surely as compared with the case where the image processing is performed as a document whose layout is unknown. As a result, the accuracy of character recognition can be improved for any document image. When comparing the layout information and the area extraction result, the correspondence relationship is obtained from the distance between the center points of the known area and the extraction area, and the similarity between the layout information and the area extraction result is calculated from the area of the overlapping area of the corresponding areas. By doing so, an accurate similarity can be calculated and appropriate image processing can be performed.

[Brief description of drawings]

FIG. 1 is a block diagram of an embodiment of a document image processing apparatus of the present invention.

FIG. 2 is an explanatory diagram of an example (No. 1) of conventional document image processing.

FIG. 3 is an explanatory diagram of an example (No. 2) of conventional document image processing.

FIG. 4 is an explanatory diagram of an example (part 1) of region extraction processing.

FIG. 5 is an explanatory diagram of an example (No. 2) of region extraction processing.

FIG. 6 is an explanatory diagram of each area of a document image.

FIG. 7 is an explanatory diagram of an example of layout information.

FIG. 8 is an explanatory diagram of an example of a correction process.

FIG. 9 is a flowchart illustrating an outline of a processing procedure of a layout identification unit.

[Explanation of symbols]

 1 Reading Section 2 Area Extraction Section 3 Layout Information Storage Section 4 Layout Identification Section 5 Image Data Storage Section 6 Character Recognition Section 7 Recognition Result Storage Section 10 Document Image Processing Device

 ─────────────────────────────────────────────────── ─── Continued Front Page (72) Inventor Makoto Torikoshi 1-7-12 Toranomon, Minato-ku, Tokyo Oki Electric Industry Co., Ltd.

Claims (2)

[Claims] 1. A region extraction unit for extracting each region of a document image, a layout information storage unit for storing a plurality or a single piece of layout information indicating attributes of each region arranged on the document image, and the layout information storage unit. Of the layout information stored in, the one that most closely matches the area extraction result by the area extraction unit is extracted, and if there is no matching layout information, the area extraction result is stored as new layout information in the layout information storage unit. A document image processing apparatus, comprising: a layout identifying unit that stores the layout information and corrects the area extraction result in accordance with the layout information when the corresponding layout information exists. 2. The layout identification unit, when comparing the layout information and the area extraction result, obtains coordinates of a center point of each area included in both, and determines the area of the layout information and the area extraction result. The area with the smallest distance between their center points is set as the corresponding area, the similarity is obtained from the area of the overlapping portion of these corresponding areas, and the layout information with the maximum similarity matches the area extraction result. A document image processing apparatus, which is characterized by setting layout information.