JPH08221506A - Device and method for recognizing business document - Google Patents

Abstract

PURPOSE: To easily and exactly extract a rectangular frame by detecting the intersections of line segments which almost perpendicularly cross each other, picking up four points among these intersection and checking whether four point can be the apexes of the rectangular frame or not by the presence/absence of a line segment between the apexes. CONSTITUTION: The angle of a line segment, whose length is longer than a threshold value set in advance, is averaged and the inclination of an entire image is measured. Then, the calculated angle of the line segment is referred to and the line segments almost horizontal or vertical to the inclination of the image are extracted. Next, a rectangular frame extracting part 4 extracts the rectangular frame formed by the extracted horizontal and vertical line segment by using a prescribed procedure. For each extracted rectangular frame, attributes such as the connecting state and shape of four apexes, the presence/ absence of slanting lines and the kinds and linen width of four sides of the rectangular frame and the slanting lines inside it are measured. The coordinate position of the rectangular frame and the other attributes provided in this case are considered as rectangular frame data. Then, the line segments used for four sides forming the rectangular frame are considered as ruled line data.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a form document recognition apparatus and method for converting a form or document including a tabular portion into data.

[0002]

2. Description of the Related Art Generally, a form document includes ruled lines and characters forming a table, and in some cases, other types of guidelines such as underlines and dividing lines, or many types of line segments such as figures and photographs. Be done.

In a form document recognition apparatus for recognizing a form document as described above, it is important to extract ruled lines accurately from the form document. Most conventional form document recognition devices use a method for extracting ruled lines by first extracting a line segment from an input image of a form document and determining, as a ruled line, a line segment that satisfies a predetermined condition. Is taking For example, there is a method in which the length of the extracted line segment is measured and the ruled line is longer than the set threshold value.

However, in such a method, a line segment intended to be a ruled line fails to be extracted as a ruled line because it does not satisfy the above-mentioned predetermined condition, or a line segment forming a character is used. May satisfy the above-mentioned predetermined condition and may be erroneously determined as a ruled line. For example, the extracted line segment may become shorter than the threshold value due to the blurring of the form document image or the break of the line, and the ruled line extraction may fail. Further, in a complicated form document image, characters and ruled lines may be close to each other, or a character pattern may include a line segment having a length exceeding a threshold value. there were.

When performing structure recognition or character code recognition of a table portion for recognizing a form document, it is important to extract a rectangular frame (rectangular area) which is a basic constituent element of the table.
Conventionally, as a method of extracting this rectangular frame, there has been a method of extracting based on the intersection pattern detected from the image. For example, in the method disclosed in Japanese Patent Laid-Open No. 2-50774, a pattern such as a wedge shape or a T shape is extracted from an image, and a table area is extracted by analyzing and combining the patterns. Although this method can deal with some blurring and line breaks, in a complicated document image, the pattern of the character string portion may be mistaken for the intersecting pattern, and the character string pattern cannot be reliably removed. There was a drawback.

There is also a method of extracting a rectangular area using the concept of line segments and intersections of line segments. For example, Hiroyuki Naruse et al. “Structure Recognition of Form Document Using Frame Ruled Line Information” (IEICE Transactions D-II, Vol. J75-DI)
I, No. As shown in 8), it is a method of extending the extracted ruled line to positively create a rectangular area and extracting it as a rectangular frame. However, this method creates a rectangular area on the assumption that the correct ruled lines are extracted, and like the above case, when the character and the table are in contact with each other or the character part is noisy. Since it was not possible to classify the characters and the ruled lines, and the correct ruled lines could not be extracted, the correct rectangle could not be extracted.

When extracting the rectangular frame, the method of obtaining the connection state of four vertices or the shape near the vertices has a drawback that it takes a long processing time because the local pattern near the vertices is recognized in detail.

[0008]

As described above, in the conventional form document recognition device and method, the line segment extracted from the input image of the form document is determined to be the ruled line. Since I used the method, if there are various line segments in the input image due to a complicated table included in the form document, or the quality of the input image of the form document is poor, Accurate ruled lines could not be extracted, such as failure in extraction or extraction of non-ruled lines as ruled lines, resulting in incorrect extraction of rectangular frames.

Further, in the method of extracting the rectangular frame based on the intersection pattern of the line segments detected from the image, for a form document including a complicated table, the pattern of the character string portion is mistaken for the intersection pattern. I could not extract the rectangular frame accurately.

Further, in the method of extending the extracted ruled lines to positively create a rectangular area and extracting it as a rectangular frame, when the character and the table are in contact with each other or the character part is noisy, It was not possible to accurately extract the rectangular frame.

Further, in extracting the rectangular frame by extracting the rectangular frame by inspecting the connection state of the four vertices or the shape near the vertices, it takes time to process the local pattern near the vertices in detail. There was a drawback such as this.

The present invention has been made in consideration of the above circumstances, and it is possible to easily and accurately extract a rectangular frame even in a form document including a complicated table or a form document image of poor quality. It is an object to provide a recognition device and method.

It is another object of the present invention to provide a form document recognition apparatus and method capable of easily obtaining the connection state of each vertex of a rectangular frame and the shape of each vertex from a form document image.

[0014]

According to the present invention, in a form document recognition apparatus for recognizing a form document including a table composed of a plurality of rectangular frames, a predetermined threshold value or more is input from image data of the input form document. A line segment extraction means for extracting a line segment having a length, an intersection point extraction means for extracting an intersection point of the extracted line segments which are orthogonal to each other, and a rectangular shape from the set of the extracted intersection points. When four intersections having four vertices are selected, and a line segment having a predetermined threshold value or more is present in a portion corresponding to a side of the rectangle having the four intersections, the rectangle having the four intersections is selected. And a rectangular frame extracting unit that determines the rectangular frame.

Further, according to the present invention, in a form document recognition apparatus for recognizing a form including a table composed of a plurality of frames, a rectangular frame extracting means for extracting a rectangular frame from the image data of the input form document is extracted. A vertex deviation measuring means for measuring a deviation between each vertex of the rectangular frame and a line segment near each vertex of the frame forming the table in the image data, and each measured vertex and line segment. It is characterized by further comprising: vertex attribute measuring means for determining the connection state of each vertex or the shape of each vertex of the frame forming the table according to the information indicating the respective shifts.

According to the present invention, in a form document recognition method for recognizing a form document including a table composed of a plurality of rectangular frames, a line segment having a length equal to or greater than a predetermined threshold value in image data of the input form document. Is extracted, intersections of mutually orthogonal ones of the extracted line segments are extracted, four intersections which are four vertices of a rectangle are selected from the set of the extracted intersections, and the four intersections are selected. If there are extracted line segments equal to or more than a predetermined threshold value in the portion corresponding to the side of the rectangle consisting of, the rectangle consisting of the four intersections is determined as a rectangular frame.

According to the present invention, in a form document recognition method for recognizing a form including a table including a plurality of frames, a rectangular frame is extracted from image data of an input form document, and each vertex of the extracted rectangular frame is extracted. And the line segment in the vicinity of each apex of the frame constituting the table in the image data are respectively measured, and the table is obtained by the information indicating the respective misalignment between each measured apex and the line segment. It is characterized in that the connection state of each apex of the frame forming the frame or the shape of each apex is determined.

[0018]

In the present invention, line segments having a certain length or more (such as capable of forming a rectangular frame) are picked up from the line segments extracted from the form document image, and they are almost perpendicular to each other (or Detect intersections of intersecting line segments (at a predetermined angle), pick up 4 points from the obtained intersections, and check whether these 4 points can be the vertices of the rectangular frame by checking the presence of line segments between the vertices. By doing so, a rectangular frame can be obtained.

Therefore, according to the present invention, since the intersection information of the line segments is used, even if the extracted line segments are interrupted or the lines do not completely intersect, It is possible to detect the vertices of a rectangular frame. Furthermore, by checking whether a rectangular frame can be generated from the obtained intersections, it is possible to determine whether the line is a ruled line or a character in the process of extracting the rectangular frame, instead of classifying the lines before extracting the rectangular frame. It is possible to classify while judging, and there are few mistakes in line type judgment, and it is possible to accurately distinguish between ruled lines and line segments in character patterns. at the same time,
It is possible to accurately extract a rectangular frame even for a complicated document image.

Further, according to the present invention, a rectangular frame is formed from the line segments extracted from the form document image, the distance to the line segment corresponding to each vertex of the rectangular frame is measured, and each measured vertex and the line segment are measured. The connection state of the vertices and the shape of the intersection are determined by the distance between and.

Therefore, according to the present invention, by simply using the distance between the vertices and the line segment, the connected state of each vertex and the shape in the vicinity of the vertex are obtained, and thus, complicated processing is not performed, and it is possible to simply Shapes such as roundness of vertices can be determined.

[0022]

Embodiments will be described below with reference to the drawings. FIG. 1 is a block diagram showing the configuration of a form document recognition apparatus according to the first embodiment of the present invention. The form document recognition device of this embodiment includes an image input unit 1, a line segment extraction unit 2, a line segment classification unit 3, a character string frame extraction unit 5, a character string recognition unit 6, a form document creation unit 7, and an output unit 8. Composed. The line segment classification unit 3 has a rectangular frame extraction unit 4 inside.

The image input section 1 is for inputting a form document to be processed as image data, and is composed of, for example, an image scanner. The line segment extraction unit 2
Preprocessing such as binarization processing, thinning processing, and line segment connection processing is performed on the input image data, and the input image data is extracted as line segment data. For line segment data, length,
Includes end point coordinates, etc.

The line segment classifying unit 3 classifies the extracted line segments into rectangular frame data, ruled line data (including line segments constituting the rectangular frame), and character pattern data and outputs the data. The rectangular frame data is extracted by the rectangular frame extraction unit 4 in the line segment classification unit 3 in the line segment classification process.

The character string frame extraction section 5 refers to the rectangular frame data, divides the character pattern data into character strings, and measures the coordinate position and the vertical and horizontal size of each character string. Also, a character string frame circumscribing the character string is extracted.

The character recognition unit 6 converts the extracted character string into a character code by a character pattern recognition process, and outputs character code data in character string units. The form document creation unit 7
Form document data is created from ruled line data, rectangular frame data, and character code data.

The output unit 8 is for outputting form document data, and is composed of, for example, a display device or a printer. The created form document data can be stored in a magnetic storage device (not shown) or the like.

The flow of processing of this embodiment will be described below with reference to the flow chart shown in FIG. Note that FIG.
FIG. 4 shows an example of the input form, and FIG. 4 shows an example of data obtained by reading the input form shown in FIG.

[Step S1] First, the form document to be processed is optically read from the image input unit 1. The input image data is temporarily stored in an image memory (not shown) or the like.

[Step S2] The line segment extraction unit 2 performs preprocessing such as binarization processing, thinning processing, and line segment connection processing on the input image data, and all input image data is line segment data. To extract.

FIG. 5 shows a flowchart of an example of line segment extraction processing in the line segment extraction unit 2. First, the input image data is binarized at a preset threshold value (step S21). A thinning process is performed on the binarized image (step S22). A chain code is measured for the obtained thin line, and polygonal line approximation is performed (step S2).
3). Then, the feature amount measurement such as the length, the coordinates of the starting and ending points, and the angle is performed for each line segment that is approximated to the broken line (step S2).
4). Finally, referring to the above feature amount, a plurality of line segments that are close to each other and have a small angle difference are connected to each other and are output as the extracted line segment (step S2).
5).

[Step S3] The extracted line segment is classified by the line segment classification unit 3 into a ruled line or a line segment in a character pattern. In the line segment classification process, the rectangular frame extraction unit 4 in the line segment classification unit 3 extracts a rectangular frame.

FIG. 6 is a flow chart of an example of line segment classification processing in the line segment classification unit 3. First, the line segment extraction unit 2
From the line segment data extracted in (3), a line segment having a length smaller than a preset threshold value t1 is removed as a minute line segment that cannot be a ruled line (step S32).

The threshold value t2 (t2> t1) whose length is preset among the line segments whose length is equal to or longer than t1 which has not been removed
The angles of the larger line segments are averaged to measure the inclination of the entire image (step S33). The inclination may be calculated by taking a histogram of the angles of the line segments and taking the maximum value. Of course, the inclination may be obtained from all the line segments.

By referring to the obtained angle of the line segment, a line segment that is substantially horizontal or vertical to the inclination of the image is extracted (step S34). The rectangular frame extraction unit 4 extracts a rectangular frame formed by the extracted horizontal and vertical line segments by the procedure described below (step S35).

For each of the extracted rectangular frames, the connection state and shape of the four vertices, the presence / absence of diagonal lines, the four sides of the rectangular frame, and the line type and line width of the diagonal lines inside the rectangular frame are measured (step S36). ).

The coordinate position and other attributes of the rectangular frame obtained here are used as rectangular frame data (step S3).
7). As a result of the rectangular frame extraction processing in step S35,
The line segments used on the four sides forming the rectangular frame are set as ruled line data (step S40). On the other hand, as a result of the rectangular frame extraction processing in step S35, for the line segments that are not used for the four sides forming the rectangular frame, the length of the line segment is compared with a predetermined threshold value t3 (step S39). ), Data longer than the threshold value t3 is ruled line data (step S
40), other than that is regarded as a line segment of the character pattern and set as a character pattern (step S42).

As a result of the horizontal / vertical line segment extraction processing in step S34, the line segment used as an attribute (oblique line, apex shape) of the rectangular frame is set as ruled line data for the line segment which is not horizontal / vertical. (Steps S39, S40). On the other hand, the line segment not used as the attribute (oblique line, shape of vertex) of the rectangular frame is a character pattern (steps S39 and S4).
2).

Further, the line segment removed as the minute line segment in step S32 is a character pattern (step S4).
2). As described above, the line segment is classified and the rectangular frame is extracted, and the character segment data, the ruled line data, and the rectangular frame data, which are the classification results, are output from the line segment classification unit 3. Other line segments such as line segments in figures / photographs and guidelines such as underlines / divider lines will be classified as ruled lines.

By the way, in the rectangular frame extraction in step S35, a large rectangular frame composed of a plurality of rectangular frames may be extracted in an overlapping manner. For example, in FIG. 4, the rectangular frame R [3-1-4-7] includes a plurality of small rectangular frames (R [2-1-4-5] and R [3-2-5-7]). Therefore, a process of deleting a large rectangular frame from the rectangular frame data to be output is performed. Here, the rectangular frame R [a
-B-c-d] is the lower left point Pa, the upper left point Pb, the upper right point P
c, the lower right point Pd is defined as a rectangular frame having four vertices. Therefore, in the flowchart of FIG. 6, step S35
After the extraction of the rectangular frame or the attribute measurement of the rectangular frame in step S36, it is advisable to delete the overlapping rectangular frames. This processing refers to the coordinate values of the four vertices, and if the large rectangular frame includes the small rectangular frame, deletes the large rectangular frame. As described above, the overlapping rectangular frames can be erased.

FIG. 7 is a flow chart showing an example of the processing procedure of step S35 performed in the rectangular frame extracting section 4 in the line segment classifying section 3. First, the coordinates of the intersection of the vertical line segment and the horizontal line segment are calculated (step S5).
1). The intersection point obtained here does not have to be on the line segment.

Considering the case as shown in FIG. 8, the intersection point P6 to be extracted even when the intersections occur outside the line segment.
1, P62. However, when the distance Dm between the end of the line segment and the intersection is larger than a certain threshold value D (Dm>
D) is not an intersection to be extracted. The distance Dm is the maximum value Dm = MAX of the horizontal distance Dx and the vertical distance Dy.
(Dx, Dy). If there is an intersection in the line segment,
Let Di = -1.

Next, those redundant to the extracted intersection are removed (step S52). As shown in FIG. 9, when a plurality of intersections are close to one location, the processing to combine them into one point is performed. Here, the intersection having the smaller distance Dm between the intersection and the end of the line segment is used. When the distances Dm are equal, the coordinates are averaged. In the case of FIG. 9, the intersection P7
Since 1 and the intersection point P72 are very close to each other, the intersection point P71 between the line segment L71 and the line segment L71 in which the distance between the intersection point and the two line segments is small is adopted, and the intersection point P72 is deleted.

As described above, it is possible to extract an intersection point which may become the apex of the rectangular frame. Next, the extracted intersections are sorted before the rectangular frame is searched (step S53). This can increase the search speed of the rectangular frame. Sorting is done so that the dots line up from left to right on the screen. Therefore, sorting is performed by the x-coordinate value of the intersection.

After sorting, two points are picked up in order from the left point to generate the left vertical line of the rectangular frame (step S54). This process proceeds from the left to the right of the screen, and is continued until the picked-up point is not on the right side (step S55), and when the two picked-up points are exhausted, the rectangular frame search ends.

The verticality of the two picked-up points is checked (step S56). If there is no line segment, pick up the next two points. Next, it is checked whether a line segment exists between the two points (step S5).
7). If there is no line segment, pick up the next two points.

When the above two checks are cleared, it is assumed that the left vertical line of the rectangular frame has been obtained, and a horizontal line extending from this vertical line to the right is searched. Therefore, the third pickup located on the right side of this vertical line is picked up (step S58). If there are no more points to be picked up as the third point, this vertical line does not form a rectangular frame, so the next two points are picked up again to search for a new vertical line candidate (step S59).

Of the two points picked up first, the horizontality of the line connecting the upper point (smaller x coordinate) and the third point is checked (step S60). If it is not horizontal, pick up the next third point.

Next, it is checked whether a line segment exists between the two points (step S61). If there is no line segment, pick up the next third point. When the above two checks are cleared, it is assumed that the horizontal line on the upper side of the rectangular frame has been obtained, and the fourth vertex of the rectangular frame is evaluated.

First, by the vertical line on the left side and the horizontal line on the upper side, 4
Assuming most positions of the points, the fourth point pickup is performed (step S62). When there are no more points to pick up as the third point, this horizontal line does not form a rectangular frame, and therefore the next third point is picked up again in order to search for a new horizontal line candidate (step S63).

The parallelism of the two sides (vertical line and horizontal line) formed by the fourth point is checked (step S64). If they are not parallel, pick up the next 4th point. next,
It is checked whether a line segment exists between the two points (step S
65). If there is no line segment, pick up the next 4th point.

When the above two checks are cleared, all sides of the rectangular frame can be fixed. With respect to the rectangular frame thus completed, the coordinate values of the four vertices are stored as rectangular frame data (step S66).

FIG. 10 is a diagram for explaining the line segment check processing (steps S57, S61, S66) in FIG. In FIG. 10, points P81 and P82 are points at both ends of one side of the rectangular frame, that is, vertex candidate points. When checking whether a line segment exists between these two points, an area such as 80 is first generated. This area 80
Has a length corresponding to the distance between P81 and P82 and a width of n pixels. It is confirmed what percentage of the line segment extracted by the line segment extraction unit 2 exists in the region 80 in the length direction of the region 80. For example, in the area 80, the line segment L8
1, line segment L82, and line segment L83 exist. Although the line segment L83 extends outside the area, only the portion included in the area is added to the length count, and the existing percentage r is measured. When this percentage s is larger than a certain threshold value S, it is recognized that a line segment exists.

Here, the above processing will be described by taking the form of FIG. 3 as a specific example of input. When the form of FIG. 3 is input from the image input unit 1, the line segment extraction unit 2 causes the line segments L1 to L22
Is obtained.

The line segment classifying unit 3 uses the rectangular frame extracting unit 4 for the line segments L1 to L14 which are horizontal / vertical line segments.
Intersection points P1 to P15 of two line segments that vertically intersect are obtained.
As described above, when obtaining an intersection, an intersection that is not located on the line segment but intersects outside the line segment is also extracted as an intersection to be obtained. For example, it is the intersection P6 of the line segment L2 and the line segment L9, and the intersection P14 of the line segment L4 and the line segment L14.

After that, four vertices capable of generating a rectangular frame are found from the extracted intersections P1 to P7. For example,
From four points P1, P2, P4, and P5, a rectangular frame R [2-1
-4-5] is generated. In this way, 28 large and small rectangular frames can be searched.

However, as it is, the large rectangular frame (for example, the rectangular frame R [3-1-4-7]) becomes a plurality of small rectangular frames (in this case, R [2-1-4-5] and R [2-1-4-5]).
[3-2-5-7]) is included, a large rectangular frame is deleted from the rectangular frame data to be output.

In this way, eight rectangular frames including R [2-1-4-5] can be output.
Here, the line segment classification unit 3 measures the attributes of the remaining rectangular frame. The attribute items to be measured are the connection state and shape (rounded corners) of each vertex, diagonal lines in the frame, line width of ruled lines, and line type. By this processing, the rectangular frame R [2-1-4-4
It can be recognized that there is a diagonal line in [5].

Line segments (L1 to L) found as a rectangular frame
14), diagonal lines (L15) attached to the rectangular frame are classified as ruled lines, and line segments (L16 to L22) are classified as line segments in the character pattern. In this way, the extracted line segments can be accurately classified, and at the same time, the rectangular frame can be extracted.

Next, FIG. 11 shows an example of the rectangular frame attribute measuring process of step S36. First, diagonal line extraction processing (step S111) is performed to check the presence or absence of diagonal lines in the rectangular frame. Next, the line type measurement (step S112) and the line width measurement (step S113) of the four sides of the rectangular frame and the diagonal lines inside the rectangular frame are performed.

Further, the following processing is performed if necessary. The vertex measurement process (step S114) is performed to measure the distances between the four vertices and the corresponding vertical and horizontal sides. Based on the vertex distance, vertex shape determination processing (step S
115) and vertex connection state forming processing (step S116)
Do either or both. The processing order of the vertex shape determination processing and the vertex connection state shaping processing is arbitrary.

FIG. 12 is a diagram for explaining an example of the vertex shape determination processing of the rectangular frame in step S115. In FIG. 12, a rectangular frame R [102-101-103-
104] is extracted. At this time, the vertex P101
To measure the shape of the apex P101 and line segment L103
The distance Dx = a and the distance Dy = b between the vertex P101 and the line segment L101 are measured. As a result, if the condition of α <Dx <β and α <Dy <β is obtained, this vertex P1
It is determined that 01 has a rounded corner, and a 90-degree arc C101 is generated at this vertex P101. In this case, the line segment L
The arc C101 can be generated without processing the features of 105 to L107 in detail. However,
Line segments L105 to L107 that are the basis of generation of arc C101
Are classified as part of the ruled line in the line segment correspondence process of step S39 in FIG.

13 and 14 are diagrams for explaining the vertex connection state forming process of the rectangular frame in step S116 of FIG. At the vertices of the rectangle extracted by the rectangular frame extraction unit 4 described above, an intersection may exist outside one line segment as shown in FIGS. 13 and 14. here,
Distance Dx between the intersection and the line segment that does not include the intersection inside
When is smaller than the predetermined threshold value γ, it may be considered that the line is broken, and the line segment may be extended to the intersection and the line segment may be connected. For example, in FIG. 13, line segment L11
1 and the line segment L112, the intersection P111 extracted and the distance measurement results of the line segments L111 and L112, Dx = c,
Dy = -1 will be obtained. In this case, c <γ, and line breaks can be connected. In addition, FIG.
At P, which is the intersection of line segment L121 and line segment L122
As a result of distance measurement of 121 and each line segment, Dx = d, Dy =-
Get one. In this case, d> γ, which is not regarded as a break in the line, and the connection can be prevented.

As described above, the attribute measurement of the rectangular frame can be performed. [Step S4] The character string frame extraction unit 5 refers to the rectangular frame data extracted in step S5, divides the character pattern data classified from the extracted line segment into a set of character strings, and Measure the coordinate position and vertical and horizontal size.
Also, a character string frame circumscribing each character string is extracted.

[Step S5] The character recognition unit 6 converts the extracted character string into a character code by a character pattern recognition process, and outputs character code data in character string units.

[Step S6] The form document creating unit 7 creates form document data from character code data including ruled line data, rectangular frame data, and character string frame information.

[Step S7] The output unit 8 outputs the form document data. As described above, according to the present embodiment, by using the intersection information of the lines, even if the extracted line segment is interrupted, or even if the lines are not completely intersected, It is possible to detect the vertices of a rectangular frame. In addition, it is possible to classify lines by determining whether they are ruled lines or characters in the process of extracting a rectangular frame, instead of performing the classification before extracting the rectangular frame. It is possible to improve the performance, and it is possible to easily and accurately extract a rectangular frame from a complicated document image.

Further, by simply using the distance between the vertices and the line segment, the connected state of each vertex and the shape in the vicinity of the vertex are obtained, so that complicated processing is not performed, and the shape such as the roundness of the vertex can be easily obtained. Can be determined.

(Second Embodiment) FIG. 15 shows a second embodiment of the present invention.
It is a block diagram showing a configuration of a form document recognition apparatus according to the embodiment.

The form document recognition apparatus of this embodiment is simplified to have the rectangular frame data extraction function of the first embodiment. The image input unit 101, the line segment extraction unit 102, and the rectangular frame extraction unit. 104, a rectangular frame attribute measuring unit 105, and a rectangular frame data output unit 106.

The image input unit 101 is for inputting a form document to be processed as image data, and is composed of, for example, an image scanner. This corresponds to the image input unit 1 of the first embodiment.

The line segment extraction unit 102 performs preprocessing such as binarization processing, thinning processing, and line segment connection processing on the input image data, and extracts the input image data as line segment data. The line segment data includes the length, the coordinates of the end point, and the like. This corresponds to the image input unit 2 of the first embodiment.

The rectangular frame extraction unit 104 extracts rectangular frame data from the extracted line segment. This corresponds to the processing of steps S32 to S35 in FIG. 6 of the first embodiment. The rectangular frame attribute measuring unit 105 measures, for the extracted rectangular frame, attribute data such as presence / absence of diagonal lines that are features of the rectangular frame, side line width and line type, vertex shape, and connection state. This corresponds to the processing of step S36 of FIG. 6 of the first embodiment.

The rectangular frame data output unit 106 generates predetermined image format data or vector data based on the obtained rectangular frame data. Then, the generated data is output from, for example, a display device or a printer. Alternatively, it is stored in a magnetic storage device or the like.

With the above arrangement, the processing flow of this embodiment is as follows. First, the image input unit 101
Then, for example, the form document to be processed as shown in FIG. 3 is optically read and stored as image data in an image memory (not shown) or the like. The input image data is temporarily stored in an image memory (not shown) or the like.

Next, the line segment extraction unit 102 performs preprocessing on the input image data, for example, in the procedure as described with reference to FIG. 5 in the first embodiment, and converts the input image data into line segments. Extract as data.

Next, the rectangular frame extraction unit 104 uses the procedure as described with reference to steps S32 to S34 of FIG. 6 in the first embodiment to set a predetermined length that may form a rectangular frame. The horizontal / vertical line segments above the above are extracted, and a rectangular frame is extracted from the extracted horizontal / vertical line segments by the procedure described with reference to step S35 in FIG. 6 and FIG. 7, for example.

The rectangular frame attribute measuring unit 105 is, for example, as shown in FIG.
In step S36 of step S36, the presence or absence of diagonal lines that characterize the rectangular frame, the line width and line type of the side, the shape of the apex, and the like with respect to the extracted rectangular frame described above are described. Measure attribute data such as connection status. Also, one or both of the vertex shape determination process of step S115 and the vertex connection state shaping process of step S116 are performed as necessary.

As described above, it is preferable to perform the overlapping rectangular frame erasing process after the rectangular frame extracting unit 104 or the rectangular frame attribute measuring unit 105.
Finally, the rectangular frame data output unit 106 outputs the rectangular frame as image format or vector data based on the code data of the rectangular frame. As described above, according to the present embodiment, by using the intersection information of lines and lines, even if the extracted line segment is interrupted, or even if the lines are not completely intersected, the rectangle The vertices of the frame can be detected. In addition, it is possible to classify lines by determining whether they are ruled lines or characters in the process of extracting a rectangular frame, instead of performing the classification before extracting the rectangular frame. It is possible to improve the performance, and it is possible to easily and accurately extract a rectangular frame from a complicated document image.

Further, by simply using the distance between the vertices and the line segment, the connected state of each vertex and the shape in the vicinity of the vertex are obtained, so that complicated processing is not performed and the shape such as the roundness of the vertex can be easily obtained. Can be determined. Further, the present invention is not limited to the above-described embodiments, and various modifications can be carried out without departing from the scope of the invention.

[0081]

According to the present invention, a rectangle is obtained from the intersection of a line segment and a line segment in a form document image, and whether or not this rectangle is a rectangular frame of the form document is determined by the existence of the line segment between the vertices. Therefore, the vertices of the rectangular frame can be reliably detected even when the extracted line segments are interrupted or when the lines do not intersect completely. Also, the line classification
It is possible to classify while judging whether it is a ruled line or a character in the process of extracting a rectangular frame instead of performing it before extracting a rectangular frame, and it is possible to improve the performance of line type discrimination, It is possible to easily and accurately extract a rectangular frame from a complicated document image.

Further, according to the present invention, the connection state of each vertex and the shape near the vertex are obtained by using the distance between the vertex of the rectangle extracted from the form document image and the line segment near the vertex. So, without complicated processing, simply
Shapes such as roundness of vertices can be determined.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a form document recognition device according to a first embodiment of the present invention.

FIG. 2 is a flowchart showing an outline of a processing flow of the embodiment.

FIG. 3 is a diagram showing an example of input.

FIG. 4 is a diagram showing an example of data generated from read input.

FIG. 5 is a flowchart showing a flow of line segment extraction processing.

FIG. 6 is a flowchart showing the flow of line segment classification processing.

FIG. 7 is a flowchart showing the flow of rectangular frame extraction processing.

FIG. 8 is a diagram for explaining intersection measurement in rectangular frame extraction processing.

FIG. 9 is a diagram for explaining redundant intersection removal in rectangular frame extraction processing.

FIG. 10 is a diagram for explaining line segment checking in the rectangular frame extraction processing.

FIG. 11 is a flowchart showing the flow of rectangular frame attribute measurement processing.

FIG. 12 is a diagram for explaining apex shape determination processing of a rectangular frame.

FIG. 13 is a diagram for explaining a vertex connection state forming process of a rectangular frame.

FIG. 14 is a diagram for explaining a vertex connection state forming process of a rectangular frame.

FIG. 15 is a block diagram showing a configuration of a form document recognition device according to a second embodiment of the present invention.

[Explanation of symbols]

1 ... Image input section, 2 ... Line segment extraction section, 3 ... Line segment classification section, 4
... rectangular frame extraction unit, 5 ... character string frame extraction unit, 6 ... character string recognition unit, 7 ... form document creation unit, 8 ... output unit, 101 ... image input unit, 103 ... line segment extraction unit, 104 ... rectangular frame Extraction unit 105 ... Rectangular frame attribute measurement unit 106 ... Rectangular frame data output unit

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Shigeka Shimotsuji No. 1 Komukai Toshiba-cho, Sachi-ku, Kawasaki-shi, Kanagawa Incorporated, Toshiba Research and Development Center

Claims (4)

[Claims] 1. A form document recognition device for recognizing a form document including a table composed of a plurality of rectangular frames, extracting line segments having a length equal to or greater than a predetermined threshold value from image data of the input form document. Line segment extraction means, intersection point extraction means for extracting intersection points of the extracted line segments that are orthogonal to each other, and four intersection points which are four vertexes of a rectangle from the set of the extracted intersection points. If there is an extracted line segment equal to or larger than a predetermined threshold value in a portion corresponding to the side of the rectangle formed by the four intersections, the rectangle formed by determining the rectangle formed by the four intersections as a rectangular frame A form document recognition device comprising: 2. A form document recognition device for recognizing a form including a table composed of a plurality of frames, and rectangular frame extracting means for extracting a rectangular frame from image data of an input form document, and the extracted rectangular frame. Of each of the vertices and the line segment near each of the vertices of the frame forming the table in the image data, and a vertex shift measuring means for measuring the shift between each of the measured vertices and the line segment. A form document recognition apparatus comprising: vertex attribute measuring means for determining a connection state of each vertex and a shape of each vertex of a frame forming the table according to information indicating a deviation. 3. A form document recognition method for recognizing a form document including a table composed of a plurality of rectangular frames, wherein a line segment having a length equal to or greater than a predetermined threshold value is extracted from image data of the input form document. Then, the intersections of the extracted line segments that are orthogonal to each other are extracted, and four intersections that are the four vertices of the rectangle are selected from the set of the extracted intersections, and the intersections are composed of the four intersections. If there is an extracted line segment equal to or larger than a predetermined threshold value in the portion corresponding to the side of the rectangle, then 4
A method for recognizing a form document, characterized in that a rectangle composed of three intersections is determined as a rectangular frame. 4. A form document recognition method for recognizing a form including a table including a plurality of frames, wherein a rectangular frame is extracted from image data of an input form document, and each vertex of the extracted rectangular frame and the The table is constructed by measuring the deviations between the line segments in the vicinity of the respective vertices of the frame forming the table in the image data, and expressing the respective deviations between the measured vertices and the line segments. Form document recognition method, characterized in that the connection state or the shape of each vertex of the frame is determined.