JP4366011B2 - Document processing apparatus and method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a document processing apparatus and method for performing digitization processing of a document, and more particularly to an image area separation processing executed in digitization processing of a document.
[0002]
[Prior art]
In recent years, with the progress of computerization of information, there is an increasing demand for storing or transmitting documents in electronic form instead of paper. In particular, due to the low price of storage media and the increase in communication bandwidth, documents to be digitized are expanding from black and white binary to full color documents.
[0003]
Document digitization here refers not only to photoelectric conversion of a paper document by a scanner or the like to convert it into image data, but also to the text, symbols, and figures that make up the document by recognizing the contents described. It is divided into areas with different properties such as photos, tables, etc., and the character part is character code information, the figure is vector data, the picture is image data, the table is structure data, etc. Refers to that.
[0004]
The first stage of electronic processing of such a document is to analyze the content written in a one-page document image and divide it into subelements of different properties such as characters, diagrams, photographs, tables, That is, it is a region division process. FIG. 25 is a diagram illustrating an example of area division.
[0005]
As an implementation example of such region division processing, “Method and Apparatus for character recognition” (Shin-Ywang et al./Canon K. K) of US Pat. No. 5,680,478 can be cited. In this example, a set of eight connected contour blocks of black pixels and four connected contour blocks of white pixels in a document image is extracted, and a character region, a picture or figure, a table, a frame is extracted from its shape, size, set state, and the like. , A characteristic area of a document such as a line is extracted. In the example of FIG. 25, character characteristic areas (blocks 1, 3, 4, 6), pictures and graphic areas (block 2), table areas (block 5), frames, and lines (7) are extracted. is doing.
[0006]
Here, the 8-connected outline block of black pixels (hereinafter referred to as black pixel block) is a collection of black pixels connected in any of eight directions from a certain black pixel as shown in FIG. Further, the 4-connected outline block of white pixels (hereinafter, white pixel block) refers to an aggregate of white pixels connected in any of four directions from a white pixel as shown in FIG.
[0007]
The above-described region division processing is premised on that the input document image is a monochrome binary because of its operation principle. Therefore, in order to perform color document region division using this technique, it is necessary to binarize the document image in advance. In general, binarization of a color image is performed by obtaining a threshold value from a luminance distribution of pixels and converting each pixel of the image into a white or black pixel with the above-described luminance threshold as a boundary.
[0008]
[Problems to be solved by the invention]
As a method for obtaining a threshold value for binarization of a color image, there are a method for obtaining a single screen and a method for obtaining each region. In the binarization method proposed in Japanese Patent Application No. 11-238581 by the present applicant, an optimum threshold value is dynamically obtained for each region according to the contents of the input document, and this is used for each region. The binarization optimal for is performed. In particular, it is possible to binarize a color image in which high-intensity characters on a low-luminance background and low-intensity characters on a high-luminance background are mixed, so that all of them are automatically converted to black characters on a white background. It is possible to obtain an optimal binary image as an input of the division process.
[0009]
FIG. 24 is a diagram for explaining a state in which region separation is performed on a document including a colored background by the previously proposed binarization method. In FIG. 24, a color document 2301 has a dark colored background area in the lower half, and light color characters are placed on it, and other than that, dark color characters are displayed on a light color background. It shall be. It can be seen that in such a document, the upper half and the lower half will be semantically separated.
[0010]
When binarization is performed on a color document such as the document 2301 by the above-described binarization method, a binary image as indicated by 2302 in FIG. 24 is generated. In the binary image 2302, the background is removed to make all white pixels, and the characters are all black pixels. At this time, if region segmentation processing is performed on the binary image 2302 as usual, the result shown in 2303 in FIG. 24 is obtained, because information on the underlying region existing in the lower half of the screen is missing. Originally, TEXT1 and TEXT2 are bonded to each other even though they should be separated into two at the center.
[0011]
That is, the character area range designation information based on the background color originally possessed by the color image is lost when binarization is performed.
[0012]
The present invention has been made in view of the above-described problems, and an object thereof is to enable region division while maintaining the distinction between regions represented by colors.
[0013]
[Means for Solving the Problems]
  In order to achieve the above object, a document processing apparatus according to an aspect of the present invention is provided.
  LuminanceDetermine multiple thresholds for binarizing an imageFirstA determination means;
  SaidFirstUsing each of the plurality of threshold values determined by the determining meansLuminanceObtaining means for obtaining a plurality of binary images by binarizing the images;
  Included in each of a plurality of binary images acquired by the acquisition meansBlack pixelConnected componentAreaOn the basis of the,Each with a different background brightnessArea location and sizeWhenTheShowGenerating means for generating area information;
  Second determination means for determining whether or not to invert the luminance value for each region indicated by the region information in the luminance image;
  For each area indicated by the area information in the luminance image, setting means for setting a threshold for binarizing each area;
  For each region indicated by the region information in the luminance image, with respect to the region determined to be inverted by the second determining unit, a corresponding region in the luminance image whose luminance value has been inverted is determined by the setting unit. A binary partial image is obtained by binarizing with a threshold value set for the area, while the luminance value is inverted for the area determined not to be inverted by the second determining means. A binary partial image acquisition unit that acquires a binary partial image by binarizing a corresponding region in the non-luminance image with a threshold value set for the region by the setting unit;
  By overwriting the binary partial image of each area acquired by the binary partial image acquisition means on the corresponding area in the binary image obtained by binarizing the entire surface of the luminance image with a predetermined threshold, the luminance image ConcerningBinarization means for obtaining a second binary image;
  Second binary image obtained by the binarization meansExtract document elements fromRegion information generated by the generating meansEach area indicated by is used as a frame element, a tree structure is obtained based on the document element and the frame element, and the luminance image is obtained based on the tree structure.Division processing means for performing area division;With
  The tree structure obtained by the division processing means is a tree structure in which document elements included in each area indicated by the area information among the extracted document elements are descendants of frame elements corresponding to the areas..
[0014]
  Also, a document processing method of the present invention for achieving the above objectIs
  FirstThe decision meansLuminanceDetermine multiple thresholds for binarizing an imageFirstA decision process;
  Acquisition meansFirstUsing each of the plurality of threshold values determined in the determination stepLuminanceAn acquisition step of acquiring a plurality of binary images by binarizing the images;
  The generation means is included in each of the plurality of binary images acquired in the acquisition step.Black pixelConnected componentAreaOn the basis of the,Each with a different background brightnessArea location and sizeWhenTheShowA generation step of generating region information;
  A second determination step in which a second determination unit determines whether to invert the luminance value for each region indicated by the region information in the luminance image;
  A setting step in which a setting unit sets a threshold for binarizing each region for each region indicated by the region information in the luminance image;
  The binary partial image acquisition means, for each region indicated by the region information in the luminance image, for the region determined to be inverted in the second determination step, the correspondence in the luminance image with the luminance value inverted. A binary partial image is obtained by binarizing the area to be processed with the threshold value set for the area in the setting step, and on the other hand, the region determined not to be inverted in the second determination step Is a binary partial image that obtains a binary partial image by binarizing a corresponding region in the luminance image that has not been inverted in luminance value with the threshold value set for the region in the setting step. Acquisition process;
  Binarization meansBy overwriting the binary partial image of each region acquired in the binary partial image acquisition step on the corresponding region in the binary image obtained by binarizing the entire surface of the luminance image with a predetermined threshold, the luminance image ConcerningA binarization step for obtaining a second binary image;
  A second binary image obtained in the binarization step by the division processing means;Extract document elements fromRegion information generated in the generating stepEach area indicated by is used as a frame element, a tree structure is obtained based on the document element and the frame element, and the luminance image is obtained based on the tree structure.A division processing step for performing area division;Have
The tree structure obtained in the division processing step is a tree structure in which document elements included in each area indicated by the area information among the extracted document elements are descendants of frame elements corresponding to the areas..
[0015]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.
[0016]
FIG. 1 is a diagram showing an overview of a document processing apparatus according to the present embodiment. In FIG. 1, reference numeral 101 denotes a computer apparatus that executes a document digitization processing program including a program for realizing processing described below with reference to the attached flowchart. The computer device 101 is accompanied by a display device 102 for displaying a situation and an image to a user, and an input device 103 configured to include a pointing device such as a keyboard and a mouse for receiving a user operation. As the display device for the display 102, a CRT, an LCD, or the like is used. A scanner device 104 optically reads a document image, digitizes it, and sends the obtained image data to the computer device 101. In this embodiment, a color scanner is used.
[0017]
FIG. 2 is a block diagram showing the configuration of the document processing apparatus according to this embodiment. In FIG. 1, reference numeral 201 denotes a CPU, which realizes various functions including electronic processing described later by executing a control program stored in a ROM 202 or a RAM 203. A ROM 202 stores various control programs executed by the CPU 201 and data. A RAM 203 stores various control programs executed by the CPU 201 and provides a work area necessary for the CPU 201 to execute various processes. Reference numeral 204 denotes an external storage device which stores a control program for realizing the processing described with reference to the attached flowchart by the CPU 101, document image data obtained by reading the image input device 104, and the like. A computer bus 205 connects the above-described components.
[0018]
FIG. 3 is a diagram showing an outline of document digitization processing by the document processing apparatus of this embodiment. In the flow of digitization processing according to the present embodiment, first, a color document to be digitized is read by the input unit 301 using the scanner 104 and stored in the external storage device 204 as image data. Next, the binarization unit 302 performs binarization processing on the document image data stored in the external storage device 204 for the subsequent area division processing. The region dividing unit 303 extracts elements such as characters, diagrams, tables, frames, lines, and the like from the binary image obtained by the binarizing unit 302 and divides them into regions. The digitized document creation unit 304 creates a digitized document by using character recognition data or using table structure data based on the attribute of each divided element. The output unit 305 stores the generated digitized document in the external storage device 204. Note that the output form in the output unit 305 is not limited to storage in the external storage device 204, and is displayed on the display 102 or output to other devices on the network via a network interface (not shown). It is also possible to output to a printer (not shown).
[0019]
Hereinafter, the operation of the binarization unit 302 shown in FIG. 3 will be described with reference to the drawings. FIG. 4 is a flowchart for explaining binarization processing according to this embodiment. FIG. 5 is a diagram showing an example of a color document image used for explaining the present embodiment. The color document image shown in FIG. 5 includes three background colors (background A501, background B502, background C503), the background A501 portion is white, the background B502 portion is yellow, and the background C503 portion is blue. And On the other hand, as for the character color, character string A504 and character string B505 are both black, and character string C506 is white.
[0020]
First, in step S401, a color document image to be processed is converted into a luminance image. Here, in general, the color format of the original image is RGB format, the luminance image is a gray scale format of 0 to 255 per pixel, and the luminance Y of each pixel is Y = the pixel values R, G, and B of the original image. Although it is determined by 0.299R + 0.587G + 0.114B, it goes without saying that other forms / conversion formulas may be used. As a result, the luminance of each part of the background color A501, background color B502, and background color C503 in FIG. Therefore, the peak of the histogram curve as shown in FIG. 6 has a certain width.
[0021]
Next, in step S402, a histogram of the luminance image data obtained in step S401 is taken. FIG. 6 is a diagram showing a histogram of luminance values obtained from the color image shown in FIG. If a histogram is obtained, a plurality of threshold values are determined from the histogram in step S403. For example, two threshold values t1 and t2 are extracted from the histogram shown in FIG.
[0022]
The plurality of threshold values t1 and t2 can be determined by the following procedure, for example. Reference is made from the luminance value 255 (white) to 0 (black) of the histogram. For example, the following conditions are set as conditions for selecting a threshold value from the referenced luminance value.
[0023]
Condition 1: In the histogram curve, a total frequency (area) included between the current reference point and a point that goes back by, for example, 10 pixels in the direction in which the luminance value on the horizontal axis increases from the reference point is a predetermined value. Greater than the first value.
[0024]
Condition 2: A rapid decrease in the frequency of the vertical axis in the histogram curve occurs (for example, “greater than a predetermined first slope” once or “greater than a predetermined second slope” twice in succession. Occurrence).
[0025]
Condition 3: A gentle decrease or increase in the histogram curve occurs (smaller than a predetermined third slope).
[0026]
In this embodiment, each of the points t1 and t2 shown in FIG. 6 satisfies “condition 2” after satisfying “condition 1” among the above conditions, and further satisfies “condition 3” thereafter. Detected as a result. The reference point may refer to the histogram curve at a predetermined interval. Moreover, although the above conditions are used in this embodiment, the determination of a threshold value is not restricted to this. For example, the case where the following conditions 4 to 6 are simultaneously satisfied may be selected as the threshold value.
[0027]
Condition 4: In the histogram curve, the total frequency (area) included between the current reference point and a point that goes back, for example, 40 pixels from the reference point in the direction in which the luminance value on the horizontal axis increases is a predetermined second value. Greater than the value of.
[0028]
Condition 5: The frequency of the vertical axis at the current reference point on the histogram curve is smaller than a predetermined third value.
[0029]
Condition 6: In the histogram curve, the histogram value at the current reference point and a point that goes back, for example, 20 pixels from the reference point in the direction of increasing luminance value on the horizontal axis is larger than a predetermined fourth value.
[0030]
In step S404, a plurality of binary images are created from each threshold value. These are temporary binary images. In this example, the binary image 701 shown in FIG. 7A is obtained by binarizing with the threshold value t1, and the binary image 702 shown in FIG. 7B is obtained by binarizing with the threshold value t2. Generated. Here, regions 703 and 704 are regions of bases B502 and C503 shown in FIG. At the threshold value t1, the background B502 and the background C503 are also black, and at the threshold t2, the background 502 is white and the background C503 is black.
[0031]
Next, in step S405, the black area in each binary image obtained in step S404 is extracted, and this is recorded as area information indicating areas having different background colors. This is a process of tracking a connected component of black pixels on a binary image and extracting a rectangular region having a certain size or more. In the example of FIG. 7, the region 703 is extracted from the binary image 701, and the region 704 is extracted from the binary image 702.
[0032]
Next, in step S406, the analysis in each area extracted in step S405 is performed. Here, the range of each region extracted in step S405 is extracted from the luminance image obtained in step S401, a histogram of luminance values is taken for each range, and is it necessary to invert the luminance value for each region from this histogram? Judgment is made and if necessary, the luminance value is inverted. Further, an optimum binarization threshold value in each region is obtained from this histogram in the same manner as in FIG. 6 (however, the predetermined value used for the judgment is different because the size of the target region is small). The obtained binarization threshold value and inversion information are output so as to accompany each area information. Whether or not inversion is necessary can be determined by obtaining an average value (average) of the histogram and its skew (skew), for example, by a calculation formula shown below.
[0033]
(average) = Σip (i),
σ²= Σ (i-av)²p (i),
(skew) = 1 / σ²Σ (i-av)^Threep (i),
However, Σip (i) represents the sum of ip (i) from i = 0 to i = 255, p (i) represents the probability density, and av represents the average value.
[0034]
When the absolute value of the skew is larger than the threshold value, it is determined that the area includes characters, and when it is determined that the area further includes characters and the skew is positive, it is determined that the image of the area should be inverted and inverted. The flag becomes yes.
[0035]
In this example, the histogram shown in FIG. 8 is obtained from the region 703, it is determined that no inversion is necessary, and the threshold value t21 is obtained from the histogram. Furthermore, the histogram shown in FIG. 9 is obtained from the region 704 and it is determined that it is necessary to invert, and the threshold t22 is obtained by inverting the histogram of FIG. 9 (FIG. 9 represents the histogram before inversion and the threshold t22. Is shown as the corresponding value in the histogram before inversion). Then, as shown in FIG. 10, region information in which a coordinate value, a binarization threshold value, and inversion information are set as one set is stored for each region.
[0036]
In this example, the area information 1001 for the area 703 is obtained from the histogram of FIG. 8 by using, as coordinate values, the left end x-coordinate l1, the right end x-coordinate r1, the upper end t1, and the lower end b1 as binarization threshold values. The threshold t21 has “no inversion” (no) as the inversion information. Similarly, the area information 1002 for the area 704 includes a threshold value obtained from the histogram of FIG. 9 using the x coordinate l2 at the left end of the rectangle, the x coordinate r2 at the right end, the t2 at the upper end, and the b2 at the lower end as the binarization threshold values. Inversion information “yes” is included as inversion information t22.
[0037]
Next, in step S407, a final binary image is generated from the luminance image obtained in step S401 based on the region information obtained in step S406. According to this example,
(1) A binary image 701 is obtained by binarizing the entire surface with a threshold value t1.
(2) The luminance image of the area 703 on the binary image 701 is processed according to the contents of the area information 1001 (in this case, binarized with the threshold value (t21)), and the obtained image is converted into the area 703 Overwrite to. As a result, a binary image 702 is obtained.
(3) For the region 704, since the corresponding region information 1002 indicates that there is inversion, the inversion processing is performed on the luminance image of the portion of the region 704, and the image obtained by binarization using the threshold value t22 is displayed in the region 704 Overwrite the part. As a result, a final binary image as shown in FIG. 11 is obtained from the color image shown in FIG.
[0038]
Next, the region dividing unit 303 performs region division using the binary image and region information obtained by the binarizing unit 302 as described above. Hereinafter, the process of the area dividing unit 303 will be described.
[0039]
FIG. 12 is a flowchart for explaining the region division processing of this embodiment. In step S1201, the region dividing unit 303 extracts document elements from the binarized document image obtained by the binarizing unit 302, and creates a tree structure representation thereof. Details of the processing in step S1201 will be described with reference to the flowchart of FIG. FIG. 13 is a flowchart for explaining the element extraction and tree structuring processing in step S1201 of FIG.
[0040]
In step S1301, all black pixel blocks are extracted from the binarized image. Note that the black pixel block is an 8-connected outline block of black pixels as described above, and is a set of black pixels having an outline formed by pixels that are in contact with each other vertically and horizontally as shown in FIG. . In the subsequent step S1302, it is determined whether or not the extracted black pixel block has a size equal to or smaller than a predetermined threshold with respect to a maximum character height and width (values obtained experimentally in advance) expected in advance. If it is below, the process advances to step S1308 to determine that the black pixel block is a character element. This is called “CHAR”.
[0041]
In step S1303, it is determined whether the extracted black pixels are vertically long or horizontally long with a certain ratio or more. If the black pixel block is vertically long or horizontally long with a certain ratio or more, it is determined as “LINE” in step S1309. In step S1304, attention is paid to the outline formed by the black pixels in the extracted black pixel block, and if the shape is a thin diagonal line, the process proceeds to step S1309, and the black pixel block is referred to as “LINE”. judge.
[0042]
In step S1305, it is checked whether the outline shape of the black pixel block is a quadrangle. FIG. 15 is a diagram illustrating an example in which (a) the outline of a black pixel block is a rectangle, and (b) the outline of a black pixel block is a non-rectangle. If it is determined in step S1305 that the outline shape of the black pixel block is not a rectangle, the process proceeds to step S1312, and the black pixel block is determined to be “PICTURE”.
[0043]
On the other hand, if the black pixel block is a square, the process proceeds to step S1306. In step S1306, four connected outline chunks of white pixels existing inside the black pixel chunk are extracted. The 4-connected outline block of white pixels is a set of white pixels having an outline formed by pixels that are in contact only in the vertical and horizontal directions as shown in FIG. Hereinafter, this set is called a white pixel block.
[0044]
In step S1307, it is determined whether or not the shape of the white pixel block extracted from the black pixel block in step S1306 is all quadrangular, and the black pixel block is filled without a gap at a predetermined interval. If the result of this determination is YES, the process advances to step S1311 to determine that the black pixel block is “FRAME”. FIG. 17 is a diagram illustrating an arrangement example of internal white pixel blocks in a frame (FRAME) and a diagram (PICTURE). According to the determination in step S1307, since (a) and (b) satisfy the determination condition in step S1307, it is determined in step S1311 that the frame is a frame (FRAME).
[0045]
In addition, the arrangement of white pixel blocks shown in FIG. 17C is that “the shape of the white pixel blocks extracted from the black pixel blocks are all square and fills the black pixel blocks without any gaps”. The condition is not satisfied, and it is determined as a diagram (PICTURE) in step S1312. Eventually, in step S1312, a black pixel block that does not meet any of the conditions up to step S1307 is set as “PICTURE”.
[0046]
In the present embodiment, an element “GROUND” that is a parent of each element is introduced. Assuming that the entire screen is a single “GROUND”, all the elements extracted so far from the image are expressed as the elements of the child. Then, each of the white pixel blocks extracted from the inside of the black pixel block determined as “FRAME” is set to “GROUND”, and further, the above-described steps S1301 to S1312 are performed inside the white pixel. Are extracted (step S1313). When “FRAME” is further extracted inside “FRAME”, this is further recursively processed as GROUND.
[0047]
At the time when all the recursive internal searches are completed, the elements extracted from the image form a tree structure. FIG. 18 is a diagram showing an example of a document image and an example of a tree structure obtained by processing this by element extraction and tree structuring in step S1201. As shown in FIG. 18A, the document image 1801 has text strings (CHAR) 1802, 1807, 1808, a frame (FRAME) 1804, and diagrams (PICTURE) 1803, 1809.
[0048]
When this document image 1801 is made into a tree structure by the above-described processing, it is as shown in FIG. GROUND 1821 indicates the entire document image 1801, and one of its elements, FRAME 1824, corresponds to a frame 1804 in the document image 1801. The frame 1804 is further divided into two frames, which are indicated as GROUND 1825 and 1826 on the tree structure, respectively.
[0049]
As described above, when the tree structure of the document image element is obtained in step S1201 of FIG. 12, the area information acquired in step S406 is applied to the tree structure in step S1202. That is, the tree structure obtained in step S301 is changed with reference to the area information recorded in the binarization process by the binarization unit 302. The processing in step S1202 will be described using the flowchart in FIG.
[0050]
In step S1901, it is checked whether area information exists. If there is no area information, this process is terminated as it is. If the area information exists, the process proceeds to step S1902, and the area is virtually considered as a set of “FRAME” and “GROUND” and is inserted into an appropriate portion of the element tree structure. In other words, a new “FRAME” corresponding to the rectangle represented by the region information is inserted at a position having “GROUND” as a parent, which contains the region, and a new “GROUND” is placed as a child of that.
[0051]
In step S 1903, among the siblings having the same parent (GROUND) as “FRAME” inserted in step S 1902, all elements located inside the area are moved to their descendants, that is, new “GROUND” children.
[0052]
FIG. 20 is a diagram illustrating an example of the tree structure changing process described above. The document image 2001 has a colored background area 2002. The binarization unit 302 binarizes the document image 2001 to generate a binary image 2010 and also generates area information 2020 corresponding to the area 2002 (FIG. 4). The region dividing unit 303 generates the tree structure 2030 having the entire image as GROUND 2031 by performing the processing described in the flowchart of FIG. 13 on the binary image 2010 (S1201). Then, the area information 2020 is applied to the tree structure 2030 to change the tree structure to obtain the tree structure 2040.
[0053]
More specifically, a frame (FRAME) corresponding to the region 2002 is inserted with GROUND 2031 as a parent, and GROUND 2041 is placed as a child thereof. Then, by arranging each element (A, B, C, figure) included in the region 2002 as a child of the GROUND 2041, the tree structure is changed.
[0054]
When the process of step S1202 is completed as described above, the process proceeds to step S1203. In step S1203, character elements are grouped to create lines and character areas. The process of step S1203 will be described using the flowchart of FIG. FIG. 21 is a flowchart for explaining the character region creation processing in step S1203.
[0055]
In step S2101, each “CHAR” is grouped together with each other having a horizontal distance within the threshold with the adjacent “CHAR”. This group is called “TEXTLINE”. However, these groupings are performed only between “CHAR” s having the same parent.
[0056]
Next, in step S2102, the “TEXTLINE” comrades that are adjacent to each other and whose vertical distance is within the threshold are grouped. The group of “TEXTLINE” obtained in this way is called “TEXT” or character area. However, these groupings are performed only between “TEXTLINE” consisting of “CHAR” having the same parent.
[0057]
With the above processing, each element of the document image is “TEXT” which is a character area, “LLINE” which is a line image part, “PICTURE” which is a picture or picture area, and “FRAME” which is a table or frame area. It is divided into.
[0058]
The grouping at the time of creating the character area in step S2103 is performed in consideration of the tree structure. For example, as shown in FIG. 22, if there is a solid line frame = “FRAME1” on the binary image, “FRAME1” Each character inside and outside must be in a different group. That is, they are grouped into different character areas such as “TEXT1” and “TEXT3”, “TEXT2” and “TEXT4”.
[0059]
Similarly, in the case of color image processing as shown in FIG. 23, even if there is no information that forms a frame on the image after binarization processing, the region obtained during binarization processing by the processing in step S1202 The information is reflected in the area division, and the characters on the color background become different character areas, and a correct area division result similar to FIG. 22 is obtained. That is, the binarization processing unit 302 obtains a binary image 2310 of the document image 2301 and area information 2320. When the binary image 2310 is divided into regions, the region information 2320 is reflected by the processing in step S1202, and a region division result is obtained as indicated by 2330.
[0060]
If the area dividing process is performed without performing the process of step S1202, the background information is not reflected, and an erroneous character area is obtained as shown in FIG.
[0061]
As described above, according to the present embodiment, when the color image is binarized and the area division processing is performed, the analysis of the area division processing is performed using the area information of the colored background stored at the time of the binarization processing. Since the contents are changed, it is possible to extract a character area that correctly reflects information lost in binarization in color image division processing, and more accurate region division processing is possible.
[0062]
According to the above embodiment, the area information stored by the binarizing unit 302 and the “FRAME” area processed by the area dividing unit 303 are limited to squares. An area such as a circle or an ellipse may be targeted. Even in this case, it is possible to extract a character area that accurately reflects information that has been lost in binarization in a color image, and it is possible to perform more accurate area division processing.
[0063]
Note that the present invention can be applied to a system including a plurality of devices (for example, a host computer, an interface device, a reader, and a printer), and a device (for example, a copying machine and a facsimile device) including a single device You may apply to.
[0064]
Another object of the present invention is to supply a storage medium (or recording medium) in which a program code of software that realizes the functions of the above-described embodiments is recorded to a system or apparatus, and the computer (or CPU or CPU) of the system or apparatus. Needless to say, this can also be achieved by the MPU) reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instruction of the program code. It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.
[0065]
Furthermore, after the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the function is determined based on the instruction of the program code. It goes without saying that the CPU or the like provided in the expansion card or the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.
[0066]
【The invention's effect】
As described above, according to the present invention, it is possible to divide a region while maintaining the distinction between regions represented by colors.
[Brief description of the drawings]
FIG. 1 is a diagram showing an overview of a document processing apparatus according to an embodiment.
FIG. 2 is a block diagram showing a configuration of a document processing apparatus according to the present embodiment.
FIG. 3 is a diagram illustrating an outline of document digitization processing by the document processing apparatus according to the embodiment;
FIG. 4 is a flowchart illustrating binarization processing according to the present embodiment.
FIG. 5 is a diagram illustrating an example of a color document image used for describing the embodiment.
6 is a diagram showing a histogram of luminance values obtained from the color image shown in FIG. 5. FIG.
7A illustrates an image obtained by binarizing the color image illustrated in FIG. 5 with the threshold value t1 illustrated in FIG. 6, and FIG. 7B illustrates an image binarized using the threshold value t2.
8 is a diagram showing a histogram of luminance values in a region 703 in FIG.
FIG. 9 is a diagram showing a histogram of luminance values in a region 704 in FIG.
FIG. 10 is a diagram showing area information.
11 is a diagram showing a final binary image of the color image shown in FIG. 5. FIG.
FIG. 12 is a flowchart illustrating area division processing according to the present embodiment.
FIG. 13 is a flowchart for describing element extraction and tree structuring processing in step S1201 of FIG. 12;
FIG. 14 is a diagram illustrating an example of 8-connected contour blocks of black pixels.
FIGS. 15A and 15B are diagrams illustrating an example in which the outline of a black pixel block is a square, and FIG. 15B is a case in which the outline of a black pixel block is a non-rectangle.
FIG. 16 is a diagram illustrating an example of four connected outline blocks of white pixels.
FIG. 17 is a diagram illustrating an arrangement example of internal white pixel blocks in a frame (FRAME) and a diagram (PICTURE).
FIG. 18 is a diagram showing an example of a document image and an example of a tree structure obtained by processing this by element extraction and tree structuring in step S1201.
FIG. 19 is a flowchart for describing tree structure change processing in step S1202 of FIG. 12;
FIG. 20 is a diagram illustrating an example of the tree structure changing process described above.
FIG. 21 is a flowchart for describing character area creation processing in step S1203;
FIG. 22 is a diagram illustrating an example of region division processing for a document having a frame region.
FIG. 23 is a diagram illustrating a state in which a color document having a colored background area is correctly divided according to the present embodiment.
FIG. 24 is a diagram illustrating a state in which region division is performed on a color document having a colored background region by a general method.
FIG. 25 is a diagram illustrating an example of region division.

Claims (9)

First determination means for determining a plurality of threshold values for binarizing the luminance image;
Obtaining means for obtaining a plurality of binary images by binarizing the luminance image using each of the plurality of threshold values determined by the first determining means;
Generation means for, based on the area of the connected component of black pixels, and generates area information in which the luminance of the background indicating the position and size of each different regions included in each of the plurality of binary images obtained by the obtaining means,
Second determination means for determining whether or not to invert the luminance value for each region indicated by the region information in the luminance image;
For each area indicated by the area information in the luminance image, setting means for setting a threshold for binarizing each area;
For each region indicated by the region information in the luminance image, with respect to the region determined to be inverted by the second determining unit, a corresponding region in the luminance image whose luminance value has been inverted is determined by the setting unit. A binary partial image is obtained by binarizing with a threshold value set for the area, while the luminance value is inverted for the area determined not to be inverted by the second determining means. A binary partial image acquisition unit that acquires a binary partial image by binarizing a corresponding region in the non-luminance image with a threshold value set for the region by the setting unit;
By overwriting the binary partial image of each area acquired by the binary partial image acquisition means on the corresponding area in the binary image obtained by binarizing the entire surface of the luminance image with a predetermined threshold, the luminance image and binarizing means for obtaining a second binary image about,
A document element is extracted from the second binary image obtained by the binarizing means, each area indicated by the area information generated by the generating means is set as a frame element, and the document element, the frame element, A division processing means for obtaining a tree structure based on the image and dividing the luminance image into regions based on the tree structure ;
The tree structure obtained by the division processing means is a tree structure in which document elements included in each area indicated by the area information among the extracted document elements are descendants of a frame element corresponding to each area. Feature document processing device. Said generating means, the region area of the connected component is greater than the predetermined size of the black pixels included in each of the plurality of binary images, said brightness of the base is determined to different regions, the luminance of the background is different areas the document processing apparatus according to generate area information indicating a position and size in claim 1, wherein the. The setting means, on the basis of the histogram of the region indicated by the region information in the luminance image, according to claim 1 or 2, characterized in that setting a threshold for binarizing the respective regions, respectively Document processing device. Said second determining means, based on the histogram of the region indicated by the region information in the luminance image, to claim 1, wherein the determining whether to perform inversion of the brightness values for the respective areas 4. The document processing apparatus according to any one of items 3. The division processing means includes
And forming means for forming a tree structure from the second binary image obtained by said binarizing means extracts said document element,
Each area indicated by the area information is inserted as a frame element in the tree structure formed by the forming means, and each document element indicated by the area information among the extracted document elements corresponds to each area. Change means for changing the tree structure by moving to the descendants of the elements of the frame to be
The document processing apparatus according to claim 1, wherein region division is performed based on the tree structure changed by the changing unit. The document processing apparatus according to the color image in any one of claims 1 to 5, wherein the obtaining further Bei converting means for converting the luminance image. Said first determining means takes a histogram of the intensity image, a document processing apparatus according to any one of claims 1 to 6, wherein determining said plurality of threshold values from the histogram. First determining means, a first determination step of plural determining a threshold for binarizing the brightness image,
An acquisition unit that acquires a plurality of binary images by binarizing the luminance image using each of the plurality of threshold values determined in the first determination step;
Generating means, based on the area of the connected component of black pixels contained multiple binary images respectively acquired by the acquisition step, and generates area information in which the luminance of the background indicating the position and size of each different region Generation process;
A second determination step in which a second determination unit determines whether to invert the luminance value for each region indicated by the region information in the luminance image;
A setting step in which a setting unit sets a threshold for binarizing each region for each region indicated by the region information in the luminance image;
The binary partial image acquisition means, for each region indicated by the region information in the luminance image, for the region determined to be inverted in the second determination step, the correspondence in the luminance image with the luminance value inverted. A binary partial image is obtained by binarizing the area to be processed with the threshold value set for the area in the setting step, and on the other hand, the region determined not to be inverted in the second determination step Is a binary partial image that obtains a binary partial image by binarizing a corresponding region in the luminance image that has not been inverted in luminance value with the threshold value set for the region in the setting step. Acquisition process;
The binarizing means overwrites the binary partial image of each area acquired in the binary partial image acquiring step on the corresponding area in the binary image obtained by binarizing the entire surface of the luminance image with a predetermined threshold. A binarization step for obtaining a second binary image related to the luminance image ;
The division processing means extracts a document element from the second binary image obtained in the binarization step, uses each region indicated by the region information generated in the generation step as a frame element, and the document element Obtaining a tree structure based on the frame elements and dividing the luminance image into regions based on the tree structure ,
The tree structure obtained in the division processing step is a tree structure in which document elements included in each area indicated by the area information among the extracted document elements are descendants of frame elements corresponding to the areas. Characteristic document processing method. Computer
First determination means for determining a plurality of threshold values for binarizing the luminance image;
An acquisition unit for acquiring a plurality of binary images by binarizing the luminance image using each of the plurality of threshold values determined by the first determination unit, and a plurality of binary images acquired by the acquisition unit, respectively. generating means based on the area of the connected component of black pixels, for generating area information indicating the position and size of each area where the luminance of the background is different contained,
Second determination means for determining whether or not to invert the luminance value for each region indicated by the region information in the luminance image;
Setting means for setting a threshold value for binarizing each area for each area indicated by the area information in the luminance image;
For each region indicated by the region information in the luminance image, with respect to the region determined to be inverted by the second determining unit, a corresponding region in the luminance image whose luminance value has been inverted is determined by the setting unit. A binary partial image is obtained by binarizing with a threshold value set for the area, while the luminance value is inverted for the area determined not to be inverted by the second determining means. A binary partial image obtaining unit that obtains a binary partial image by binarizing a corresponding region in the non-luminance image with a threshold value set for the region by the setting unit;
By overwriting the binary partial image of each area acquired by the binary partial image acquisition means on the corresponding area in the binary image obtained by binarizing the entire surface of the luminance image with a predetermined threshold, the luminance image binarizing means for obtaining a second binary image about,
A document element is extracted from the second binary image obtained by the binarizing means, each area indicated by the area information generated by the generating means is set as a frame element, and the document element, the frame element, The tree structure is obtained based on the document element included in each area indicated by the area information among the extracted document elements as a descendant of a frame element corresponding to each area. A division processing means for dividing an area of the luminance image based on the tree structure ,
A computer-readable storage medium storing a program for functioning as a computer.

Images