JP4159071B2 - Image processing method, image processing apparatus, and computer-readable recording medium storing program for realizing the processing method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing method, an image processing apparatus, and a computer-readable recording medium on which a program for realizing the processing method is recorded. More specifically, the present invention relates to a method for optically recognizing characters entered on a paper, The present invention relates to an apparatus, and more particularly, to an image processing method and apparatus for extracting written characters from a form image with high accuracy.
[0002]
[Prior art]
Conventionally, the process of extracting the entered character part from the form image is for a complete standard document, and the characters are written on the same form with the same shape, position of the character and the entry frame as a whole. I was expecting something. Therefore, the character entry position on the image having the same format as the processing target form is designated in advance by coordinates, and when the processing target image is input, alignment with the same format image is performed to enter the character in the processing target image. This was done by estimating the position and extracting the characters. For the alignment as described above, for example, what is disclosed in Japanese Patent Application Laid-Open No. 10-91783 extracts a crossing point of a line segment from an image and performs image alignment using the extracted point as a mark. It was an effective means for accurate character extraction.
[0003]
[Problems to be solved by the invention]
However, the forms that have been distributed in the past are not completely fixed, and there are many forms that are generally fixed but slightly different in shape. An example will be described with reference to FIG.
FIG. 1 is a diagram for explaining an example of a form that has been distributed in the past. As in the example shown in FIGS. 1A and 1B, both look like the same form. However, in reality, it differs in the following points.
(1) The line type (solid line / dotted line) used in the address and name fields is different.
(2) The gender and adult / child column in the number of people column are upside down.
(3) The size of the frame in the number of people column is different.
[0004]
As in the example shown in FIG. 1, when the position or size of a frame or print character changes in the image, the coordinate value of the character entry position is designated based on one image by the above-described method. However, even if characters were written on the other form, those characters could not be extracted accurately.
[0005]
The present invention has been made in consideration of the above-described circumstances, and an image processing method, an image processing apparatus, and a method for realizing the processing method capable of extracting characters from forms having slightly different shapes. The object of the present invention is to provide a computer-readable recording medium in which a program is recorded.
[0006]
[Means for Solving the Problems]
In order to achieve the above object, the invention of claim 1 is an image processing method for extracting characters in a form image, an image input step for inputting the form image, and a keyword existing in the form image. A keyword input step, a frame extraction step for extracting a frame surrounded by a ruled line in the form image, a character recognition step for recognizing characters in the frame, and a character of the recognition result and the keyword are compared. a keyword matching process, and a affiliation frame extracting step of extracting by searching belongs frame frame results the keyword and the same characters of the comparison is present, the frame extraction step, a predetermined length or more of the black run A black run extraction step for extracting the black runs, a pixel complement step for filling the black runs with black pixels when the extracted black runs are closer than a predetermined distance, and a form image in which the black pixels are embedded Circumscribing connected components of white pixels A circumscribed rectangle extracting step for extracting a shape, a rectangle integrating step for integrating the circumscribed rectangles according to the overlapping state when the circumscribed rectangles obtained by the extraction overlap, and a size of the integrated circumscribed rectangle A rectangular size examination step of comparing the size with a predetermined size and excluding those smaller than the predetermined size, and a frame position determination step of setting the coordinate value of the circumscribed rectangle obtained based on the examination as the coordinate value of the frame It is provided with .
[0007]
The invention of claim 2 is an image processing apparatus for extracting characters in a form image, wherein the image input means inputs a form image, the keyword input means inputs a keyword existing in the form image, and the form A frame extracting unit for extracting a frame surrounded by a ruled line in the image; a character recognizing unit for recognizing a character in the frame; a keyword collating unit for comparing the character of the recognition result with the keyword; and a comparison result An affiliation frame extracting means for searching and extracting an affiliation frame of a frame in which the same character as the keyword exists,
The frame extraction means includes a black run extraction means for extracting a black run having a predetermined length or more, and a pixel that fills the black run with black pixels when the extracted black runs are closer than a predetermined distance. Complementing means, circumscribed rectangle extracting means for extracting circumscribed rectangles of connected components of white pixels from the form image in which the black pixels are embedded, and circumscribed rectangles obtained by the extraction overlap each other In accordance with the examination, a rectangle integration means for integrating the circumscribed rectangles, a rectangle size examining means for comparing the size of the integrated circumscribed rectangle with a predetermined size, and excluding those smaller than the predetermined size, and with the coordinates of the circumscribed rectangles obtained Te characterized by comprising a frame position determining means for the coordinate values of the frame.
[0008]
Invention of Claim 3 is a computer-readable recording medium which recorded the program for implement | achieving each function for performing the image processing method which extracts the character in a form image with a computer, Each said function Includes an image input function for inputting a form image, a keyword input function for inputting a keyword existing in the form image, a frame extraction function for extracting a frame surrounded by a ruled line in the form image, A character recognition function for recognizing characters, a keyword collation function for comparing the recognition result characters with the keyword, and a result of comparison by searching for and belonging to a frame where the same character as the keyword exists A frame extraction function, wherein the frame extraction function is a black run extraction function that extracts black runs longer than a predetermined length, and the extracted black runs are closer than a predetermined distance. A pixel complementing function for filling the black run with black pixels, a circumscribed rectangle extracting function for extracting a circumscribed rectangle of a connected component of white pixels from a form image in which the black pixels are embedded, and a circumscribed rectangle obtained by the extraction When there is an overlap, the rectangle integration function that integrates circumscribed rectangles according to the degree of overlap, and the size of the integrated circumscribed rectangle is compared with a predetermined size, and those that are smaller than the predetermined size are excluded And a frame position determining function that uses the coordinate value of the circumscribed rectangle obtained based on the review as a coordinate value of the frame.
[0015]
DETAILED DESCRIPTION OF THE INVENTION
(Example 1)
In the example shown in FIGS. 1A and 1B, the item names are the same, although they are not complete standard forms. For example, the name is supposed to be written next to the frame printed as "Name", and the male is filled in the frame adjacent to the frame printed as "Men", regardless of the vertical or horizontal difference. There is no difference between them. Therefore, first, an item name such as “name” and “male” is found, and a frame adjacent to the item name is searched for and used as an entry frame.
[0016]
FIG. 2 is a configuration diagram for explaining an embodiment of an image processing method, an image processing apparatus, and a computer-readable recording medium on which a program for realizing the processing method according to the present invention is recorded. Are image input means, 2 is image storage means, 3 is keyword input means, 4 is keyword storage means, 5 is frame extraction means, 6 is frame information storage means, 7 is character recognition means, 8 is keyword matching means, 9 is This is a means for grouping membership.
FIG. 3 is a flowchart for explaining an embodiment of the operation of the embodiment shown in FIG.
[0017]
(1) An image to be processed is input by the image input means 1 such as a scanner and stored in the image storage means 2 (step S1).
(2) A keyword is input by the keyword input means 3 and stored in the keyword storage means 4 (step S2). This keyword represents the attribute of the character entry box. In the example shown in FIG. 1, this keyword corresponds to “name”, “male”, etc., and the attribute to be extracted is designated manually in advance.
(3) The frame extraction unit 5 performs frame extraction, and stores the information in the frame information storage unit 6 (step S3). As a frame extraction method, a method of extracting ruled lines and using the information can be considered, but in the first embodiment, it is obtained from white pixel connected components as follows.
[0018]
FIG. 4 is a block diagram for explaining an embodiment of the frame extracting means shown in FIG. 2. In the figure, 11 is a black run extracting means, 12 is a black inter-pixel complementing means, and 13 is complementary image information storage. Means 14, white pixel connected component circumscribed rectangle extraction means 15, rectangle information storage means 16, rectangle integration means 16, rectangle size examination means 17, and frame position determination means 18.
FIG. 5 is a flowchart for explaining an embodiment of the operation of the embodiment shown in FIG.
6 and 7 are diagrams showing specific examples of the embodiment shown in FIG.
[0019]
(A) The black run extraction means 11 extracts a black run longer than a predetermined value (step S3-1).
(B) A search is made for those whose distances between the runs are close to each other, and black pixels are embedded between the runs on the image by the pixel interpolation means 12 between black runs (step S3-2). This processing is performed because the connected component in the frame is connected to the outside of the frame if the ruled line is blurred when the frame position is determined by extracting the white connected component later, and the frame cannot be correctly extracted. The reason why the complement is limited to a long run is to prevent generation of false ruled lines by integrating the noises in an area where a lot of fine noise exists such as a shaded area (FIG. 6). See). The image information in which black pixels are embedded is stored in the complementary image information storage unit 13.
[0020]
(C) The circumscribed rectangle of the white pixel connected component (hereinafter also referred to as a white rectangle) is extracted by the white pixel connected component circumscribed rectangle extracting means 14 and stored in the rectangular information storage means 15 (step S3-3). A region surrounded by black pixels constituting the frame is extracted as a circumscribed rectangle here. This rectangular position is a frame candidate.
(D) Among the frame candidates, those that overlap between certain white rectangles are selected and integrated by the rectangle integration means 16 (step S3-4). As shown in FIG. 7, if the white rectangles overlap each other to some extent so that they are not extracted as separate frames when the frames are divided by characters, as shown in FIG. It is a process that integrates as a rectangle.
(E) Finally, the rectangle size reviewing unit 17 excludes a rectangle whose size is too small because the possibility of being inside the character is high (step S3-5), and the frame position determining unit 18 determines the position of the white rectangle. Is determined as the frame position (step S3-5).
[0021]
(4) Since the frame has been extracted as described above, the character recognition means 7 performs character recognition processing within the frame (step S4).
(5) Since the position of the frame is known, the character recognition process is performed only inside the frame. Next, the obtained recognition result and the registered keyword are compared by the keyword collating means 8 to search for the same one (step S5). An image region (frame region) that gives the same recognition result as the keyword is a frame in which the keyword exists. Thus, a keyword frame in which a keyword (characters identified as) exists can be extracted.
[0022]
(6) Search for a character entry frame (hereinafter referred to as an affiliation frame) belonging to (related to) the keyword frame (step S6). For example, in the example shown in FIG. 1, there are two character entry frames belonging to the frame in which the item (keyword) “adult” exists, and both of them are extracted to form a character entry frame group belonging to “adult”. . Also, since there is only one character entry box belonging to the item “name”, this one frame is extracted. The policy is to search for frames that touch the right and bottom of the frame where the keyword exists, and group them as frames with the same attribute until they touch frames of different sizes. The processing procedure is as follows.
[0023]
FIG. 8 is a diagram for explaining one embodiment of the belonging frame grouping means shown in FIG. 2, in which 21 is a reference frame setting means, 22 is an adjacent frame search means, 23 is a frame shape verification means, Reference numeral 24 denotes an in-frame keyword verification unit, 25 denotes an affiliated frame candidate information storage unit, 26 denotes an affiliated frame candidate layer number counting unit, 27 denotes an affiliated frame information storage unit, and the other operations are the same as those in the embodiment shown in FIG. The same reference numerals as those in the embodiment shown in FIG.
FIG. 9 is a flowchart for explaining an embodiment of the operation of the embodiment shown in FIG.
10 and 11 are diagrams showing specific examples of the embodiment shown in FIG.
[0024]
(A) The reference frame setting means 21 determines the frame in which the keyword exists as an initial reference frame (step S6-1).
(B) The adjacent frame searching means 22 searches for a frame in contact with the right side of the reference frame (step S6-2). Specifically, it is considered that the right end coordinate of the reference frame and the left end coordinate of the adjacent frame are close to each other.
(C) If there is no frame to contact (NO in step S6-3), the right side search for the reference frame is terminated and a lower side search is performed (step S6-10).
[0025]
(D) The frame shape verification means 23 verifies the size and position of the frame (step S6-4). When the upper and lower ends of the frame are inside the reference frame and the height difference is small, it is determined that the shape of the frame conforms to the reference, and otherwise (NO in step S6-4). ), The right side search of the reference frame is finished and the lower side search is performed (step S6-10).
(E) The in-frame keyword verification means 24 examines whether or not the adjacent search target frame is a keyword frame (step S6-5), and if it is not a keyword frame (NO in step S6-5), frame shape verification The means 23 is used to examine the difference in width between the reference frame and the adjacent search target frame (step S6-6). If the difference is large, it is determined that the adjacent search target frame is not a grouping target. (NO in step S6-6), the right side search is finished and the process proceeds to the lower side search (step S6-10).
(F) The in-frame keyword verification means 24 examines whether or not the adjacent search target frame is another keyword frame (step S6-7). If it is a keyword frame (YES in step S6-7), grouping is performed. It determines with it not being a target, finishes a right side search, and moves to a lower side search (step S6-10).
[0026]
(G) Since the above search conditions are cleared (NO in step S6-7), the adjacent search target frames are grouped and determined to belong to the relevant frame candidate of the keyword. (Step S6-8).
(H) The reference frame setting means 21 resets the grouped frames as reference frames (step S6-9), and further checks the connection relationship with the right adjacent frame. In this manner, the process is repeated until no new grouped frame is found.
(I) Subsequently, grouping is also performed on the frame below the keyword frame (step S6-10). The procedure is the same as when searching for the right side.
[0027]
(J) If grouping can be performed on both the right side and the lower side, only one of them is registered as a membership frame. As a criterion for selecting either, the affiliation frame candidate layer number counting means 26 is used, as shown in FIG. 11, the number of affiliation frames in each of the vertical and horizontal groups is counted (step S6-11), and the larger one is selected. This is selected and stored in the affiliation frame information storage means 27 (step S6-12). In the example shown in FIG. 11, since the number of layers in the vertical direction is large, the group in the vertical direction (lower side) is adopted. In the case of the same number, the horizontal direction is adopted.
(K) When all keywords have been examined (YES in step S6-13), the result is output (step S6-14), and the process is terminated.
With the above processing, the character entry area belonging to each keyword can be specified, so the image at that position is called a character image, and characters can be extracted.
[0028]
(Example 2)
FIG. 12 is a main part configuration diagram for explaining an embodiment of a computer-readable recording medium on which a program for realizing the image processing method according to the present invention is recorded, and is implemented when the present invention is realized by software. It is an example.
A general-purpose processing device including a CPU 31, a memory 32, a hard disk 33, an input device 34, a CD-ROM drive 35, a display 36, a mouse, and the like is prepared. A recording medium 37 such as a CD-ROM stores a program for realizing the processing functions and processing procedures of the image processing of the present invention. The document image to be processed is stored in the hard disk 33, for example. The CPU 31 reads out and executes a program for realizing the processing functions and procedures described above from the recording medium 37, and outputs the image processing result to the display 36 or the like.
[0029]
【The invention's effect】
As is clear from the above description, according to the present invention, characters can be extracted even from forms that are not completely fixed, and characters can be extracted correctly even if the ruled lines are somewhat blurred.
[Brief description of the drawings]
FIG. 1 is a diagram for explaining an example of a form that has been distributed in the past.
FIG. 2 is a main part configuration diagram for explaining an embodiment of an image processing method, an image processing apparatus and a computer-readable recording medium storing a program for realizing the processing method according to the present invention.
FIG. 3 is a flowchart for explaining an embodiment of the operation of the embodiment shown in FIG. 2;
4 is a block diagram for explaining an embodiment of the frame extracting means shown in FIG. 2;
FIG. 5 is a flowchart for explaining an embodiment of the operation of the embodiment shown in FIG. 4;
6 is a diagram showing a specific example of the embodiment shown in FIG.
7 is a diagram showing a specific example of the embodiment shown in FIG.
FIG. 8 is a diagram for explaining an embodiment of the belonging frame grouping means shown in FIG. 2;
FIG. 9 is a flowchart for explaining an embodiment of the operation of the embodiment shown in FIG. 8;
10 is a diagram showing a specific example of the embodiment shown in FIG.
FIG. 11 is a diagram showing a specific example of the embodiment shown in FIG. 9;
FIG. 12 is a block diagram illustrating a main part of an embodiment of a computer-readable recording medium on which a program for realizing an image processing method according to the present invention is recorded.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Image input means, 2 ... Image storage means, 3 ... Keyword input means, 4 ... Keyword storage means, 5 ... Frame extraction means, 6 ... Frame information storage means, 7 ... Character recognition means, 8 ... Keyword collation means, 9 ... belonging frame grouping means, 11 ... black run extracting means, 12 ... inter-black run pixel complementing means, 13 ... complementary image information storing means, 14 ... white pixel connected component circumscribed rectangle extracting means, 15 ... rectangular information storing means, 16 ... rectangular integration means, 17 ... rectangular size examination means, 18 ... frame position determination means, 21 ... reference frame setting means, 22 ... adjacent frame search means, 23 ... frame shape verification means, 24 ... in-frame keyword verification means, 25 ... Affiliation frame candidate information storage means, 26 ... Affiliation frame candidate layer count counting means, 27 ... Affiliation frame information storage means, 31 ... CPU, 32 ... Memory, 33 ... Hard disk, 34 ... Input device, 35 ... CD-ROM drive Breakfast, 36 ... display, 37 ... recording medium.

Claims (3)

An image processing method for extracting characters in a form image,
An image input step for inputting a form image, a keyword input step for inputting a keyword existing in the form image, a frame extraction step for extracting a frame surrounded by a ruled line in the form image, and characters in the frame a character recognition step of recognizing a keyword matching process and, affiliation frame to be extracted by searching the belonging frame frame results the keyword and the same character in comparison exists of comparing the character of the recognition result keyword An extraction process ,
The frame extraction step includes a black run extraction step for extracting a black run having a predetermined length or more, and a pixel that fills the black run with black pixels if the extracted black runs are closer than a predetermined distance. A complementing step, a circumscribed rectangle extracting step of extracting a circumscribed rectangle of a connected component of white pixels from a form image in which the black pixel is embedded, and a circumscribed rectangle obtained by the extraction overlap each other Based on the examination, a rectangle integration step for integrating the circumscribed rectangles, a rectangle size examining step for comparing the size of the integrated circumscribed rectangle with a predetermined size, and excluding those smaller than the predetermined size, and And a frame position determining step in which the coordinate value of the circumscribed rectangle obtained in this way is used as the coordinate value of the frame . An image processing apparatus that extracts characters in a form image, and is surrounded by an image input means for inputting a form image, a keyword input means for inputting a keyword existing in the form image, and a ruled line in the form image A frame extracting unit for extracting a frame, a character recognizing unit for recognizing a character in the frame, a keyword collating unit for comparing the character of the recognition result with the keyword, and a character identical to the keyword as a result of the comparison An affiliation frame extraction means for searching and extracting the affiliation frame of an existing frame;
The frame extraction means includes a black run extraction means for extracting a black run having a predetermined length or more, and a pixel that fills the black run with black pixels when the extracted black runs are closer than a predetermined distance. Complementing means, circumscribed rectangle extracting means for extracting circumscribed rectangles of connected components of white pixels from the form image in which the black pixels are embedded, and circumscribed rectangles obtained by the extraction overlap each other In accordance with the examination, a rectangle integration means for integrating the circumscribed rectangles, a rectangle size examining means for comparing the size of the integrated circumscribed rectangle with a predetermined size, and excluding those smaller than the predetermined size, and An image processing apparatus comprising: frame position determining means that uses the coordinate value of the circumscribed rectangle obtained in this way as the coordinate value of the frame . A computer-readable recording medium recording a program for realizing each function for executing an image processing method for extracting characters in a form image by a computer,
The functions include an image input function for inputting a form image, a keyword input function for inputting a keyword existing in the form image, a frame extraction function for extracting a frame surrounded by ruled lines in the form image, A character recognition function for recognizing characters in the frame; a keyword collation function for comparing the recognition result character with the keyword; and, as a result of the comparison, searching for a belonging frame of a frame in which the same character as the keyword exists An affiliation frame extraction function to extract,
In addition, the frame extraction function extracts a black run having a predetermined length or more, and if the extracted black runs are closer than a predetermined distance, the black run is filled with black pixels. A pixel complement function, a circumscribed rectangle extracting function for extracting a circumscribed rectangle of a connected component of white pixels from a form image in which the black pixel is embedded, and a circumstance of overlapping when circumscribed rectangles obtained by the extraction overlap each other A rectangle integration function for integrating the circumscribed rectangles according to the size, a rectangle size examination function for comparing the size of the integrated circumscribed rectangle with a predetermined size, and excluding those smaller than the predetermined size, and It is a frame position determination function that uses the coordinate value of the circumscribed rectangle obtained based on the coordinate value of the frame as a computer, and can be read by a computer in which a program for realizing the above functions is realized by a computer Do recording medium.

Images