JPH07105310A - Method for detecting tilt of picture and method for processing table - Google Patents

Abstract

PURPOSE:To recognize exactly characters in a table on a tilted picture by detecting a tilt of the picture including ruled lines. CONSTITUTION:A black run extract section 403 extracts a black run longer in a main and a sub scanning direction on an input picture 402, and a rectangle integration section 405 integrates black runs within a predetermined distance into a rectangle. A ruled line rectangle extract section 407 integrates rectangles within a predetermined distance to extract a rectangle surrounded by ruled lines. A tilt detection section 409 discriminates a direction of a tilt of the input picture by comparing start and end point coordinates of pairs of black runs integrated by the rectangle integration section 405 and comparing start and end point coordinates of pairs of ruled lines integrated by the ruled line rectangle integration section 407.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to the processing of images of a document or a form (these are collectively referred to as documents in this specification) input by an image input device such as an image scanner, and more particularly to a document including ruled lines. The present invention relates to detection of inclination of an image of a document and processing of a table area in the image.

[0002]

2. Description of the Related Art When an image of a document is processed in a character recognition device, if the input image is tilted, the recognition rate and the like are adversely affected. Even in a general image input device such as a copying machine, a facsimile device, and a scanner, the inclination of the input image is not preferable.

As a method of detecting the inclination of an input document image, a method is known in which character block blocks are extracted from the document image and the inclination is obtained from the arrangement relationship between the character block blocks and the characteristics such as the center. Kaihei 3-62284).

[0004]

However, Japanese Unexamined Patent Publication (Kokai) No. 3-6.
The method of No. 2284 is based on the premise that a character block can be extracted, that is, a character exists in the input image. For this reason, it is not possible to detect the inclination of the image formed only of the ruled line format data.

Therefore, it is an object of the present invention to provide an image inclination detecting method capable of detecting the inclination of an input image including ruled lines regardless of whether or not it contains characters. Another object of the present invention is to provide a table processing method capable of accurately recognizing a character in a table on an image that is tilted and input.

[0006]

In order to achieve the above-mentioned object, the invention of claim 1 extracts a black run having a length equal to or more than a certain value on an input image, and is predetermined by the extracted black run. A feature is that black runs within a distance are integrated into a rectangle that includes all of them, and the direction of the inclination of the input image is determined by the mutual positional relationship of the black runs that are integrated when the black runs are integrated. To do.

According to a second aspect of the present invention, a black run having a length equal to or longer than a certain value on the input image is extracted, and a black run within a predetermined distance by the extracted black run is included in the rectangle. , Which are within a predetermined distance of the black run, and which are within a predetermined distance, are integrated into a ruled line rectangle that includes all of them, and depending on the mutual positional relationship of the rectangles that are integrated when the rectangles are integrated. The feature is that the direction of inclination of the input image is determined.

According to a third aspect of the present invention, a black run having a length equal to or larger than a certain value on the input image is extracted, and a black run within a predetermined distance by the extracted black run is included in a rectangle including all the black runs. , The black run integrated rectangles within a predetermined distance are integrated into a ruled line rectangle that includes all of them, and the positions of the black runs are integrated with each other when the black run is integrated. The inclination direction of the input image is determined based on the relationship and the mutual positional relationship of the rectangles that are integrated when the rectangles are integrated.

According to a fourth aspect of the present invention, in the method of the first or third aspect, the inclination direction is determined by comparing the coordinates of the start point and the end point of the integrated black run.

According to a fifth aspect of the present invention, in the method of the second or third aspect, the direction of the inclination is determined by comparing the coordinates of the start point and the end point of the integrated rectangle.

The invention of claim 6 is the invention of claim 1, 3 or 4.
In the method of the invention, for a plurality of pairs of black runs to be integrated, the inclination direction is determined for each pair, the number of determinations is counted for each inclination direction, and the inclination direction with the greater number of determinations is counted. Is the result of the determination of the tilt direction by the black run.

According to a seventh aspect of the invention, in the method of the sixth aspect of the invention, the inclination is determined with respect to one longest rectangular range.

The invention of claim 8 is the invention of claim 2, 3 or 5.
In the method of the invention, for a plurality of pairs of integrated rectangles, the inclination direction is determined for each pair, the number of determinations is counted for each inclination direction, and the inclination direction with the larger number of determinations is determined. It is characterized in that the result of the determination of the inclination direction by a rectangle is used.

According to a ninth aspect of the present invention, in the method of the eighth aspect, the inclination is determined with respect to one longest ruled line rectangle.

According to a tenth aspect of the present invention, in the method of the second or third aspect, a value obtained by dividing the number of black pixels inside the ruled line rectangle by the size of the ruled line rectangle in the longitudinal direction is the width of the ruled line rectangle. It is characterized in that it is subtracted from the size in the direction and the tilt angle of the input image is obtained from the size after the subtraction and the size in the longitudinal direction of the ruled line rectangle.

According to the invention of claim 11, in the method of claim 1, the value obtained by dividing the number of black pixels in the rectangle by the size of the rectangle in the longitudinal direction is subtracted from the size of the rectangle in the lateral direction. Then, the inclination angle of the input image is obtained from the size after the subtraction and the size in the longitudinal direction of the rectangle.

According to a twelfth aspect of the present invention, the inclination direction of the input image is detected by the method according to any one of the first to ninth aspects, and the inclination angle of the input image is detected according to the tenth or eleventh aspect. An image in which the inclination of the input image is corrected based on the detected direction and angle of the detected inclination is created by the method of the invention, a ruled line rectangle is extracted from the tilt corrected image, and the extracted ruled line rectangle is extracted. It is characterized in that the frame of the table is recognized from the coordinates, and the character image inside the recognized frame is cut out from the tilt correction image for character recognition.

[0018]

According to the method of the invention of each of claims 1 to 11, the tilt direction of the image can be detected with reference to the ruled line on the input image. Further, according to the method of the invention of claim 10 or 11, The angle of inclination of the input image can also be detected. In this way, if there is a ruled line on the input image, the inclination of the input image can be accurately detected. Therefore, for example, even with a form such as a table without characters, the inclination of the image can be detected. .

The method according to the first or second aspect of the invention is based on the positional relationship of the black runs that are integrated when the black runs are integrated, or the positional relationship of the rectangles that are integrated when the integrated rectangles of the black runs are integrated. The inclination of the image is detected by using this method, but the black run rectangle integration and the rectangle integration processing are generally necessary processing steps in table processing, and are therefore rational methods that are not wasted in applying to table processing. .

According to the method of the third aspect of the present invention, in addition to the positional relationship of the black runs forming the ruled line, the positional relationship of the integrated rectangles of the black runs forming the ruled line is utilized for the tilt direction determination. . Therefore, it is possible to detect the tilt direction more accurately as compared with the method using only the positional relationship of the black runs.

According to the method of the invention of claim 4 or 5,
Based on the magnitude relationship between the coordinates of the start point and the end point of the black run or the rectangle, the tilt direction can be determined by an extremely simple rule.

According to the method of the invention of claim 6 or 8,
Since the tilt direction is determined based on the positional relationship of a plurality of pairs of black runs or black run integrated rectangles that form the ruled line, it is possible to avoid the influence of noise and detect the tilt direction in a stable and reliable manner.

According to the invention of claim 7 or 9, since the inclination direction is determined with reference to the longest ruled line on the input image, an accurate determination result can be obtained. Further, since attention is paid only to the range of one ruled line, the processing load can be reduced as compared with the case where all ruled lines are targeted.

According to the method of the tenth or eleventh aspect of the present invention, the inclination angle of the input image can be accurately detected in consideration of the black run integrated rectangle or the thickening of the integrated rectangle due to the inclination of the input image.

According to the twelfth aspect of the present invention, the inclination of the input image is detected, the inclination of the input image is corrected, the ruled lines are extracted, the frame of the table is recognized, and the character image in the frame is corrected for inclination. Since the image is cut out from the image, it is possible to accurately recognize the characters in the table on the image that are tilted and input.

[0026]

Embodiments of the present invention will be described below with reference to the drawings.

<First Embodiment> FIG. 1 is a block diagram for explaining the present embodiment. In FIG. 1, an image of a document including ruled lines is read by a binary image input unit (image scanner or the like) 101 and input as binary image data. The binary image data of the input image is stored in the binary image memory 102.

The black run extraction unit 103 scans the binary image data in the binary image memory 102 and extracts a black run having a length of a certain value or more in the main scanning direction or the sub scanning direction. Then, the extracted black run data (starting point,
The coordinates of the end point) are stored in the black run memory 104.

In the rectangular integration unit 105, the black run data in the black run memory 104 is referred to, and black runs within a predetermined distance range with respect to a long black run in the main scanning direction or the sub-scanning direction. Then, a process of integrating them into a rectangle including all of them is performed, and the data of the integrated rectangle of the black run (vertex coordinates, etc.) is stored in the rectangle memory 106. Further, a pair of integrated black run data (start point and end point coordinates) is stored in a specific area of the rectangular memory 106.

The inclination detecting unit 107 receives the data of the black run pair to be integrated from a specific area in the rectangular memory 106 during the black run integrating process by the rectangle integrating unit, and detects the data of the black run pair to be integrated. By examining the mutual positional relationship, the direction of inclination of the input image relative to the binary image input unit 101 is determined. An example of the method of determining the tilt direction will be described below.

Example 1 Method Focusing on Ruled Lines in the Main Scanning Direction When integrating black runs in the main scanning direction, as shown in FIG.
Two long black runs LH1 and LH that are integrated in the main scanning direction
Compare the X coordinates of the start or end points of 2. As shown in FIG. 2A, the X coordinate (xs of the start point of the upper black run LH1
If the relationship between 1) and the X coordinate (xs2) of the starting point of the lower black run LH2 is xs1 <xs2, it is determined as “downward right”, and as shown in FIG. X coordinate (xe1) and X coordinate (xe2) of the end point of black run LH2 are xe
If 1> xe2, it is determined to be "upward". If none of these, it is determined that there is no inclination.

Note that this determination may be performed on a plurality of pairs of black runs to be integrated, and the final determination result may be determined by, for example, the majority of the determination results of each pair. Alternatively, it is also possible to count the number of times of each of the downward sloping and upward sloping determinations and use the tilt direction that has reached the predetermined value first as the final determination result.

(Example 2) Method Focusing on Ruled Lines in Sub-scanning Direction When integrating black runs in the sub-scanning direction, as shown in FIG.
Two long black runs LV1 and LV integrated in the sub-scanning direction
Compare the Y coordinates of the start or end points of 2. As shown in FIG. 3A, the Y coordinate of the start point of the right black run LV1 (Ys
If the relationship between 1) and the Y coordinate (Ys2) of the starting point of the lower black run LV2 is Ys1 <Ys2, it is determined to be “downward right”, and as shown in FIG. 3B, the end point of the black run LV1. Y coordinate of (Ye1) and the Y coordinate of the end point of the black run LV2 (Ye2) are
If Ye1> Ye2, it is determined as “upward to the right”. If none of these, it is determined that there is no inclination.

Note that this determination may be performed on a plurality of pairs of black runs to be integrated, and the final determination result may be determined by, for example, a majority of the determination results of each pair. Alternatively, it is also possible to count the number of times of each of the downward sloping and upward sloping determinations and use the tilt direction that has reached the predetermined value first as the final determination result.

When both the integrated rectangle of black runs in the main scanning direction and the integrated rectangle of black runs in the sub-scanning direction are obtained,
It is also possible to adopt a method in which the determination result is validated when the results of the inclination determination by the black runs in the main and sub scanning directions match, and when the results do not match, the determination cannot be performed. Or Lord,
Probability of the determination result in each sub-scanning direction is obtained, and when the determination results in each direction do not match, the determination result with the higher probability is used as the final determination result.

<Embodiment 2> FIG. 4 is a block diagram for explaining the present embodiment. In FIG. 4, an image of a document including ruled lines is read by a binary image input unit (image scanner or the like) 401, and binary image data is converted into binary image data.
It is stored in the value image memory 402.

The black run extraction unit 403 scans the binary image data in the binary image memory 402 and extracts a black run having a length of a certain value or more in the main scanning direction or the sub scanning direction. Then, the extracted black run data (starting point,
The coordinates of the end point) are stored in the black run memory 404.

In the rectangular integration section 405, the black run data in the black run memory 404 is referred to, and black runs within a predetermined distance range with respect to a long black run in the main scanning direction or the sub-scanning direction. Then, a process of integrating them into a rectangle including all of them is performed, and data of the integrated rectangle of the black run (vertex coordinates, etc.) is stored in the rectangle memory 406.

By such a black run integration process in the main and sub scanning directions, most of the black runs forming the ruled line can be integrated. However, in images of documents input by an image scanner or the like, ruled lines are often broken due to ruled line intersections, ruled line fading, and the like.

In order to correct such breaks and extract the original ruled lines, in the ruled line rectangle extraction unit 407, the integrated rectangle of the main scanning direction black run and the integrated rectangle of the sub scanning direction black run in the rectangular memory 406 are respectively extracted. For this, ruled line rectangle integration is performed. That is, regarding the integrated rectangle of long black runs in the main scanning direction, the rectangles within the range of the predetermined distance are integrated into a rectangle that includes all of them (main scanning direction ruled line rectangle). Similarly, for a black run integrated rectangle that is long in the sub-scanning direction, a rectangle within a range of a predetermined distance is
It is integrated into a rectangle that includes all of them (ruled line rectangle in the sub-scanning direction). Then, the extracted ruled line rectangle data (vertex coordinates, etc.) is stored in the ruled line rectangle memory 408. In addition, the data of the rectangle to be integrated (vertex coordinates, etc.) is stored in the ruled line rectangle memory 40.
8 specific areas.

In the inclination detection unit 409, the rectangle integration unit 4
04 and the ruled line quadrature extraction unit 407, the data in the black run memory 404 and the data in the specific area in the ruled line rectangle memory 408 are referred to, and the positional relationship of the integrated black run pairs and the integrated data are combined. By examining the positional relationship of the pair of rectangles, the direction of inclination of the input image relative to the binary image input unit 401 is determined. An example of the method of determining the tilt direction will be described below.

(Example 1) Method Focusing on Ruled Lines in the Main Scanning Direction When a rectangular black run having a long main scanning direction is integrated, a pair of black runs to be integrated is the same as in the first embodiment.
The inclination direction is determined based on the magnitude relationship between the X coordinate of the start point or the end point of the upper black run and the X coordinate of the start point or the end point of the lower black run (see FIG. 2).

When the ruled line rectangles in the main scanning direction are integrated,
As shown in FIG. 5, the X of the start point or the end point of each of a pair of integrated rectangles (integrated rectangle of black run) RH1, RH2
The direction of the tilt is determined based on the magnitude relationship of the coordinates. The X coordinate (Xs1) of the start point (upper left apex) of the upper rectangle RH1 and the X coordinate (Xs2) of the start point of the lower rectangle RH2 are shown in FIG.
As shown in (a), when the relationship of Xs1 <Xs2 is satisfied, the tilt direction is determined to be “downward right”. Also, FIG.
As shown in (b), the X coordinate (xe1) of the end point (lower right vertex) of the rectangle RH1 and the X coordinate (xe2) of the end point of the rectangle RH2.
However, when Xe1> Xe2, the tilt direction is determined to be “upward to the right”. When none of the above relationships is satisfied, it is determined that there is no inclination.

Then, when the determination result by the black run and the determination result by the rectangle match, the determination result is set as the final determination result.

It should be noted that the determination by the rectangle may be performed for a plurality of pairs of integrated rectangles, and for example, the majority result of the determination results of each pair may be used as the determination result. Or right down,
It is also possible to count the number of times of each upward rising determination and use the inclination direction that has reached the predetermined value first as the determination result.

(Example 2) Method Focusing on Ruled Lines in Sub-scanning Direction When a long black run in the sub-scanning direction is integrated into a rectangle, the black run pairs to be integrated are the same as in the first embodiment.
Y coordinate of the starting point of the black run on the right side and Y of the starting point of the black run on the left side
The tilt direction is determined based on the magnitude relationship with the coordinates (Fig. 3
reference).

When integrating ruled line rectangles in the sub-scanning direction,
As shown in FIG. 6, Y of the start point or the end point of each of a pair of integrated rectangles (integrated rectangles of black runs) RV1 and RV2.
The direction of the tilt is determined based on the magnitude relationship of the coordinates. The Y coordinate (Ys1) of the starting point (upper right apex) of the right rectangle RV1 and the Y coordinate (Ys2) of the starting point of the left rectangle RV2 are shown in FIG.
As shown in (a), when the relationship of Ys1 <Ys2 is satisfied, the tilt direction is determined to be “downward right”. In addition, FIG.
As shown in (b), the Y coordinate (Ye1) of the end point (lower left apex) of the rectangle RV1 and the Y coordinate (Ye2) of the end point of the rectangle RV2.
However, when the relation of Ye1> Ye2 is satisfied, the direction of the inclination is determined as “upward to the right”. When none of the above relationships is satisfied, it is determined that there is no inclination.

Then, when the determination result by the black run and the determination result by the rectangle match, the determination result is set as the final determination result.

It should be noted that the determination by the rectangle may be performed for a plurality of pairs of integrated rectangles, and for example, the majority result of the determination results of each pair may be used as the determination result. Or right down,
It is also possible to count the number of times of each upward rising determination and use the inclination direction that has reached the predetermined value first as the determination result.

When both the main-scanning direction ruled line and the sub-scanning direction ruled line are obtained, when the result of the tilt determination focusing on the ruled lines in the main scanning direction and the sub-scanning direction matches, the determination result is validated, and there is no mismatch. In such a case, it is possible to adopt a method such as making the determination impossible. Alternatively, it is also possible to determine the certainty of the determination result in each of the main and sub-scanning directions, and when the determination results in each direction do not match, the determination result with the higher certainty is used as the final determination result. it can.

<Third Embodiment> FIG. 7 is a block diagram for explaining the present embodiment. In FIG. 7, images of documents including ruled lines are read by a binary image input unit (image scanner or the like) 701 and input as binary image data. The binary image data of the input image is stored in the binary image memory 702.

The black run extraction unit 703 scans the binary image data in the binary image memory 702 and extracts a black run having a length of a certain value or more in the main scanning direction or the sub scanning direction. Then, the extracted black run data (starting point,
The coordinates of the end point) are stored in the black run memory 704.

In the rectangular integration section 705, the black run data in the black run memory 704 is referred to, and black runs within a predetermined distance range with respect to a long black run in the main scanning direction or the sub-scanning direction. Then, a process of integrating them into a rectangle including all of them is performed, and the data of the integrated rectangle of the black run (vertex coordinates, etc.) is stored in the rectangle memory 706. In addition, a pair of integrated black run data (coordinates of start point and end point) is stored in a specific area of the rectangular memory 706.

The inclination condition determining unit 708 is configured to detect the rectangular memory 7
The longest rectangle is extracted from the integrated rectangles in the main scanning direction and the sub-scanning direction by referring to the rectangle data in 06. In the present embodiment, the rectangle is regarded as the most accurate ruled line or a ruled line representing a table on documents, and the inclination is determined by the method described below.

First, the black run extraction unit 703 executes black run extraction again within the range extracted as the longest rectangle, and the rectangle integration unit 70 for the black run extracted here.
In step 5, rectangle integration is executed.

During this rectangle integration processing, the tilt condition determination unit 7
At 08, the data of the black run pair to be integrated is read from the specific area of the rectangular memory 706, and the determination regarding the inclination condition is performed. An example of this determination condition is shown below.

(1) When the rectangle of interest is an integrated rectangle of black runs in the main scanning direction (ruled line in the main scanning direction) As shown in FIG. 2, the start or end points X of the integrated black runs LH1 and LH2 are X. Compare the coordinates. As shown in FIG. 2A, if the relationship between the X coordinate (xs1) of the starting point of the upper black run LH1 and the X coordinate (xs2) of the starting point of the lower black run LH2 is xs1 <xs2, then “down right It is determined that the above condition is met. In addition, as shown in FIG. 2B, the black run LH
1, the X coordinate of the end point of LH2 (xe1, xe2) is xe1>
If xe2, it is determined that the condition of "upward to the right" is met.

(2) When the rectangle of interest is an integrated rectangle of black runs in the sub-scanning direction (ruled line in the sub-scanning direction) As shown in FIG. 3, the start or end point Y of the integrated black runs LV1, LV2 is Y. Compare the coordinates. As shown in FIG. 3A, if the relationship between the Y coordinate (Ys1) of the starting point of the right black run LV1 and the Y coordinate (Ys2) of the starting point of the lower black run LV2 is Ys1 <Ys2, then “down right It is determined that the above condition is met. Also, the Y coordinate of each end point (Ye1, Ye2)
However, as shown in FIG. 3B, if Ye1> Ye2, it is determined that the condition of "upward to the right" is met.

Then, the tilt condition determining unit 708 determines that the condition matching number memory 70 is satisfied when the “upward right” condition is met.
Increase the value of 9 (initial value is 0) by 1 to "fall right"
When the condition is met, the value of the condition match frequency memory 709 is decreased by 1. If none of the conditions is met, the value in the condition match count memory 709 is not updated.

When the above-mentioned determination regarding the noted rectangle is completed, when the value of the condition matching number memory 709 is a positive value and the absolute value exceeds 3 in the inclination detection unit 710, the inclination direction is determined to be "upward to the right". To do. When the value of the slope condition matching count memory 709 is a negative value and the absolute value exceeds 3,
The tilt direction is determined to be “downward right”. The value is ±
If it is within the range of 3, it is determined that there is no inclination. In addition,
The reason why “no inclination” is set within the range of ± 3 is to consider the influence of noise on the input image.

<Embodiment 4> FIG. 8 is a block diagram for explaining the present embodiment. In FIG. 8, images of documents including ruled lines are read by a binary image input unit (image scanner or the like) 801, and the binary image data is stored in a binary image memory 802.

The black run extraction unit 803 scans the binary image data in the binary image memory 802, extracts a black run having a length of a certain value or more in the main scanning direction or the sub scanning direction, and extracts the extracted black. The run data (coordinates of the start point and the end point) are stored in the black run memory 804.

The rectangular integration section 805 refers to the black run data in the black run memory 804, and finds black runs within a predetermined distance range with respect to a long black run in the main scanning direction or the sub scanning direction. Then, a process of integrating them into a rectangle including all of them is performed, and the data of the integrated rectangle of the black run (vertex coordinates, etc.) is stored in the rectangle memory 806. In the ruled line rectangle extraction unit 807, the rectangular data in the rectangular memory 806 is referred to and the integrated rectangle of the black run in the main scanning direction is
Rectangles within a predetermined distance range are integrated into a rectangle that includes them (main scanning direction ruled line rectangle). Similarly, regarding the integrated rectangle of the black runs long in the sub-scanning direction, the rectangles within the range of the predetermined distance are integrated into a rectangle including all of them (a sub-scanning direction ruled line rectangle). Then, the extracted ruled line rectangle data (vertex coordinates, etc.) is stored in the ruled line rectangle memory 808.

The inclination condition determination unit 809 refers to the data in the ruled line rectangle memory 706 and extracts the longest rectangle out of the ruled line rectangles in the main scanning direction and the sub scanning direction. In the present embodiment, the rectangle is regarded as the most accurate ruled line or a ruled line representing a table on documents, and the inclination is determined by the method described below.

First, the black run extraction unit 803 executes black run extraction again within the range of the longest extracted ruled line rectangle, and the rectangular integration unit 805 for the black runs extracted here.
Then, the rectangle integration is performed, and the ruled line extraction unit 807 performs the integration of the rectangle.

During the black run integration process by the rectangle integration unit 805, the inclination condition determination unit 809 makes an inclination condition determination for the pair of integrated black runs. An example of this determination condition is shown below.

(1) When the ruled line rectangle in the main scanning direction is the object of processing As shown in FIG. 2, the X coordinates of the start points or end points of the integrated black runs LH1, LH2 are compared. As shown in FIG. 2 (a), if the relationship between the X coordinate (xs1) of the starting point of the upper black run LH1 and the X coordinate (xs2) of the starting point of the lower black run LH2 is xs1 <xs2, then “down right It is determined that the above condition is met. Further, as shown in FIG. 2B, if the X coordinate (Xe1, Xe2) of each end point is xe1> xe2, it is determined that the condition of "upward rising" is met.

(2) When the ruled line rectangle in the sub-scanning direction is the object of processing As shown in FIG. 3, the Y coordinates of the start points or end points of the integrated black runs LV1, LV2 are compared. As shown in FIG. 3A, if the relationship between the Y coordinate (Ys1) of the starting point of the right black run LV1 and the Y coordinate (Ys2) of the starting point of the lower black run LV2 is Ys1 <Ys2, then “down right It is determined that the above condition is met. Further, as shown in FIG. 3B, if the Y coordinates (Ye1, Ye2) of the respective end points are Ye1> Ye2, it is determined that the condition of "increasing to the right" is met.

Then, the inclination condition judging unit 809 increments the value of the condition-matching frequency memory 810 (initial value is 0) by 1 when the pair of black runs to be integrated meets the condition of "increasing to the right", and outputs "right". When the "falling" condition is met, the value of the condition matching frequency memory 810 is decreased by one. If none of the conditions is met, the condition match count memory 81
Do not update the value of 0.

Further, during the ruled line integration processing by the ruled line rectangle extraction unit 807, the tilt condition determination unit 809 determines the tilt condition for the pair of integrated rectangles. An example of this determination condition is shown below.

(1) When a ruled line in the main scanning direction is a processing target As shown in FIG. 5, a pair of rectangles (integrated rectangles of black runs) RH1 and RH2 to be integrated are set to the start or end points X.
The direction of inclination is determined based on the magnitude relationship of the coordinates. The X coordinate (Xs1) of the starting point (upper left apex) of the upper rectangle RH1 and the X coordinate (Xs2) of the starting point of the lower rectangle RH2 are shown in FIG.
As shown in, when the relationship of Xs1 <Xs2 is satisfied, the tilt direction is determined to be “downward right”. Further, as shown in FIG. 5B, the X coordinate (Xe1, Xe2) of each end point is X.
In the case of the relationship of e1> Xe2, it is determined that the tilt direction is “upward to the right”.

Then, the tilt condition determining unit 809 increases the value of the condition match count memory 810 (initial value is 0) by 1 when the integrated rectangular pair meets the condition of "increasing to the right", and "decreases to the right". When the condition is met, the value of the condition match number memory 810 is decreased by 1. If none of the conditions are met, the condition match count memory 810
Do not update the value of.

In this way, at the stage where the integration process for the noticed ruled line rectangle is completed, if the value of the condition match count memory 810 is a positive value and the absolute value exceeds 5, the tilt detecting section 811 determines the tilt direction. Is determined to be “upward to the right”. When the value in the tilt condition matching count memory 810 is a negative value and the absolute value exceeds 5, the tilt direction is determined to be “downward right”. Further, when the value is within the range of ± 5, it is determined that “no inclination”. It should be noted that the reason why “no inclination” is set within the range of ± 5 is to consider the influence of noise on the input image.

In this embodiment, the longest ruled line rectangle is selected, and the black run extraction, its integration, and the ruled line rectangle integration are performed again as a representative, and the inclination direction is detected by performing the condition determination at that time. However, it goes without saying that the inclination can be detected by performing the same condition determination on all ruled lines.

That is, during the black run integration process by the rectangle integration unit 805, the inclination condition determination unit 809 performs the above-described condition determination for all the pairs of black runs to be integrated, and the number of times the conditions are matched according to the result. The memory 810 is updated. Further, during the rectangle integration by the ruled line rectangle extraction unit 807, the condition determination as described above is performed for all pairs of the integrated rectangles, and the condition matching frequency memory 810 is updated according to the result. Then, the tilt detection unit 811 determines the tilt direction as described above based on the value of the condition matching frequency memory 810 at the stage when the above processing is completed.

<Embodiment 5> In each of the above-described embodiments, only the tilt direction is detected, but in this embodiment, the tilt angle is also detected.

FIG. 9 is a block diagram for explaining the present embodiment. The system configuration shown here is similar to that of the fourth embodiment, and the detection of the direction of the tilt is performed in the fourth embodiment.
Is the same as. However, the inclination direction may be detected by the same method as in the first, second, or third embodiment (in the case of the first or third embodiment, the integrated rectangle of black runs is treated as a ruled line rectangle).

Just as in the fourth embodiment, the binary image input unit 901 inputs the binary image data of a document into the binary image memory 902, and the black on the image in the main scanning direction and the sub scanning direction is long. Orchid black orchid extraction unit 903
And stores the data in the black run memory 906. The rectangular integration unit 905 integrates the black runs into rectangles, and stores the data in the rectangle memory 906. Ruled line rectangle extraction unit 9
At 07, the rectangles are integrated to extract a ruled line rectangle, and the data is stored in the ruled line rectangle memory 908. The tilt condition determination unit 909 extracts the maximum length ruled line rectangle. Then, for the range of the ruled line rectangle, black run extraction, black run integration,
The ruled line rectangle extraction is executed, and the inclination condition determination unit 909 determines the inclination condition for the black run pair and the rectangular pair, and updates the condition matching number memory 910. And
The tilt detection unit 911 determines the tilt direction from the value in the condition matching count memory 910.

In parallel with the above-described operation for detecting the tilt direction, the following processing for tilt angle detection is performed in this embodiment. FIG. 10 is an explanatory diagram of the processing.

When the black run extraction unit 903 extracts a black run by scanning the binary image in the range of the ruled line rectangle of interest, the number of black pixels inside the ruled line rectangle (pixe
l) is obtained and the value is stored in the black pixel number memory 913. The number of black pixels (pixel) may be obtained from the black run data (in the black run memory 904) included in the ruled line rectangle.

Next, in the inclination detection unit 911, the width of the ruled line (widt
h) is calculated, and the thickness of the ruled line is calculated from the calculated h and the number of black pixels by the following equation.

Thickness = pixel / width The inclination detecting unit 911 also obtains the height (height) of the ruled line from the data in the ruled line rectangle memory 908, and based on this and the thickness of the ruled line (thickness), Calculate the tilt of.

Tilt = height-thickness Then, the inclination angle (θ) is calculated by the following equation.

Tan θ = tilt / width <Sixth Embodiment> FIG. 11 is a block diagram for explaining the present embodiment. In FIG. 1, the internal structure of the broken-line frame 1001 is the same as that of the fifth embodiment shown in FIG. 9 with respect to the inclination detection of the input image, and therefore its description is omitted. Note that the portions corresponding to the respective portions in FIG. 9 are indicated by the same reference numerals.

In FIG. 11, the image correction unit 1002 is
Rotating the binary image data in the binary image memory 902 in the direction opposite to the tilt of the image by the same angle as the tilt angle in accordance with the tilt direction and angle information output from the tilt detection unit 911. The tilt was corrected by 2
Create a value image and correct it Binary image memory 1
It is stored in 003.

From the binary image data in the corrected binary image memory 1003, the black run extraction unit 903 extracts a black run having a length equal to or more than a certain value in the main scanning direction and the sub-scanning direction to obtain black. Run data to black run memory 90
Store in 4. Old data (black run data for tilt detection) in the black run memory 904 is discarded.

The rectangular integration section 905 refers to the data in the black run memory 904 and refers to the black runs in the main scanning direction and the black runs in the sub-scanning direction respectively within a predetermined distance range. Is integrated into a rectangle that includes them, and the data of the rectangle is stored in the rectangle memory 906. Old data in the rectangular memory 906 is discarded.

The ruled line rectangle extraction unit 907 refers to the data in the rectangular memory 906, and sets a predetermined distance range for each of the integrated rectangle of the black run in the main scanning direction and the integrated rectangle of the black run in the sub scanning direction. The black run in the above is integrated into a rectangle that includes all of them, and the data of the integrated rectangle (ruled line rectangle) is stored in the ruled line rectangle memory 908. Old data in the ruled line rectangle memory 908 is discarded. In this way, the ruled line rectangle on the image in which the inclination of the documents is corrected is extracted.

Next, the frame recognition unit 1004 recognizes the frame of the table (rectangular area surrounded by vertical and horizontal ruled lines) based on the data of the ruled line rectangle in the ruled line rectangle memory 908, and coordinates of each frame (pair The corner vertex coordinates) are stored in the frame coordinate memory 1005. Then, based on the frame coordinates, the in-frame character extraction unit 100
The image of the character inside each frame is cut out from the corrected binary image in the corrected binary image memory 1003 by 6,
The cut-out character image is passed to the character recognizing unit 1007 to be recognized. Thus, it is possible to accurately recognize the characters in the table on the document that are input with a tilt.

[0090]

As described above, according to the present invention, the following effects can be obtained.

(1) According to the methods of the inventions of claims 1 to 11, the tilt direction of the image can be detected with reference to the ruled line on the input image. Furthermore, according to the method of the invention of claim 10 or 11, the angle of inclination of the input image can also be detected. In this way, since the method uses ruled lines as a reference, it is possible to detect the inclination of the image even in a form such as a table without characters.

(2) The method of the invention of claim 1 or 2 is
Image tilt is detected based on the positional relationship of the black runs that are integrated when the black runs are integrated, or the positional relationship of the rectangles that are integrated when the integrated rectangles of the black runs are integrated. Also, since the rectangle integration processing is a processing step that is generally necessary for table processing, it is a rational method that can be applied to table processing without waste.

(3) According to the method of the third aspect of the present invention, in addition to the positional relationship of the black runs forming the ruled line, the positional relationship of the integrated rectangles of the black runs forming the ruled line is utilized to determine the tilt direction. To use. Therefore, it is possible to detect the tilt direction more accurately as compared with the method using only the positional relationship of the black runs.

(4) According to the method of the invention of claim 4 or 5, the tilt direction can be determined by an extremely simple rule based on the magnitude relationship between the coordinates of the start point and the end point of the black run or the rectangle.

(5) According to the method of the invention of claim 6 or 8, since the tilt direction is determined based on the positional relationship of a plurality of pairs of black runs or black run integrated rectangles that form a ruled line, the influence of noise is exerted. It is possible to detect the tilt direction in a stable and reliable manner by avoiding such problems.

(6) According to the invention of claim 7 or 9,
Since the inclination direction is determined based on the longest ruled line on the input image, an accurate determination result can be obtained. Further, since attention is paid only to the range of one ruled line, the processing load can be reduced as compared with the case where all ruled lines are targeted.

(7) According to the method of the tenth or eleventh aspect of the present invention, the inclination angle of the input image is accurately detected in consideration of the black run integrated rectangle or the thickening of the integrated rectangle due to the inclination of the input image. You can

According to the twelfth aspect of the present invention, the inclination of the input image is detected, the inclination of the input image is corrected, the ruled lines are extracted, the frame of the table is recognized, and the character image in the frame is corrected for inclination. Since the image is cut out from the image, it is possible to accurately recognize the characters in the table on the image that is input with a tilt.

[Brief description of drawings]

FIG. 1 is a block diagram for explaining a first embodiment of the present invention.

FIG. 2A shows an example of the positional relationship of pairs of black runs that form a ruled line in the main scanning direction on an image that is inclined to the lower right. (B) An example of the positional relationship of a pair of black runs forming a ruled line in the main scanning direction on an image inclined to the upper right is shown.

FIG. 3A shows an example of the positional relationship of pairs of black runs that form a ruled line in the sub-scanning direction on an image that is inclined to the lower right. (B) An example of the positional relationship of a pair of black runs forming a ruled line in the sub-scanning direction on an image inclined to the right is shown.

FIG. 4 is a block diagram for explaining a second embodiment of the present invention.

FIG. 5A shows an example of the positional relationship of a pair of rectangles forming a ruled line in the main scanning direction on an image inclined to the lower right. (B) An example of the positional relationship of the pair of rectangles forming the ruled line in the main scanning direction on the image tilted to the right is shown.

FIG. 6A shows an example of the positional relationship of a pair of rectangles forming a ruled line in the sub-scanning direction on an image inclined to the lower right. (B) An example of the positional relationship of a pair of rectangles forming a ruled line in the sub-scanning direction on an image tilted to the right is shown.

FIG. 7 is a block diagram for explaining a third embodiment of the present invention.

FIG. 8 is a block diagram for explaining a fourth embodiment of the present invention.

FIG. 9 is a block diagram for explaining a fifth embodiment of the present invention.

FIG. 10 is an explanatory diagram of detecting an inclination angle in the case of a ruled line in the main scanning direction.

FIG. 11 is a block diagram for explaining a sixth embodiment of the present invention.

[Explanation of symbols]

101, 401, 701, 801, 901 Binary image input unit 102, 402, 702, 802, 902 Binary image memory 103, 403, 703, 803, 903 Black run extraction unit 104, 404, 704, 804, 904 Black Run memory 105, 405, 705, 805, 905 Rectangle integration section 106, 406, 706, 806, 906 Rectangle memory 107, 409, 710, 811, 911 Tilt detection section 407, 807, 907 Ruled line rectangle extraction section 408, 808, 908 Ruled-line rectangle memory 809,909 Inclination condition determination unit 810,910 Condition matching number memory 1002 Image correction unit 1003 Corrected binary image memory 1004 Frame recognition unit 1005 Frame coordinate memory 1006 Frame character extraction unit 1007 Character recognition unit LH Main scanning direction Ruled line black run LV side run Black run of ruled lines RH Black integrated run rectangle of ruled lines in the main scanning direction RV Black run integrated rectangle of ruled lines in the sub-scanning direction

Claims (12)

[Claims] 1. A black run having a length equal to or greater than a certain value on an input image is extracted, and black runs within a predetermined distance of the extracted black runs are integrated into a rectangle including all of them, and An image inclination detection method, characterized in that the direction of inclination of an input image is determined based on the mutual positional relationship of the black runs that are integrated when the black runs are integrated. 2. A black run having a length equal to or greater than a certain value on an input image is extracted, and black runs within a predetermined distance of the extracted black runs are integrated into a rectangle including all of them to obtain a black run. Rectangle that integrates runs and is within a predetermined distance is integrated into a ruled line rectangle that includes all of them, and the inclination of the input image is determined by the mutual positional relationship of the rectangles that are integrated when the rectangles are integrated. An image inclination detection method characterized by determining the direction of 3. A black run having a length equal to or greater than a certain value on an input image is extracted, and black runs within a predetermined distance of the extracted black runs are integrated into a rectangle including all of them to obtain a black run. The run integrated rectangles within a predetermined distance are integrated into a ruled line rectangle that includes all of them, and the mutual positional relationship of the black runs integrated when the black runs are integrated, and An image tilt detection method characterized in that the direction of tilt of an input image is determined based on the mutual positional relationship of the rectangles that are integrated when the rectangles are integrated. 4. The image inclination detection method according to claim 1, wherein the inclination direction is determined by comparing the coordinates of the start point and the end point of the integrated black run. 5. The image tilt detection method according to claim 2, wherein the tilt direction is determined by comparing the coordinates of the start point and the end point of the integrated rectangle. 6. For multiple pairs of integrated black runs,
The tilt direction is determined for each pair, the number of times of judgment is counted for each direction of tilt, and the tilt direction with the larger number of times of judgment is used as the judgment result of the tilt direction by the black run. The image inclination detection method according to claim 1, 3 or 4. 7. The image inclination detecting method according to claim 6, wherein the inclination is determined with respect to one longest rectangular area on the input image. 8. The inclination direction is determined for each pair of a plurality of integrated rectangular pairs, the number of determinations is counted for each inclination direction, and the inclination direction having the larger number of determinations is the rectangle. The image inclination detecting method according to claim 2, 3 or 5, wherein the determination result of the inclination direction is obtained. 9. The image inclination detecting method according to claim 8, wherein the inclination is determined with respect to the range of one longest ruled line rectangle on the input image. 10. A value obtained by dividing the number of black pixels inside the ruled line rectangle by the size of the ruled line rectangle in the longitudinal direction is subtracted from the size of the ruled line rectangle in the lateral direction, and the size after the subtraction and the ruled line rectangle. The image inclination detection method according to claim 2, wherein the inclination angle of the input image is obtained from the size of the input image in the longitudinal direction. 11. A value obtained by dividing the number of black pixels inside a rectangle by the size in the longitudinal direction of the rectangle is subtracted from the size in the lateral direction of the rectangle, and the size after the subtraction and the length in the longitudinal direction of the rectangle are subtracted. The image inclination detecting method according to claim 1, wherein the inclination angle of the input image is obtained from the size. 12. The method according to claim 1, wherein the tilt direction of the input image is detected, and the tilt angle of the input image is detected by the method according to claim 10. An image in which the tilt of the input image is corrected based on the detected tilt direction and angle, a ruled line rectangle is extracted from the tilt corrected image, and a table frame is formed from the extracted ruled line rectangle coordinates. A table processing method, which comprises recognizing and slicing a character image inside the recognized frame from the tilt correction image for character recognition.