CN102592124A - Geometrical correction method, device and binocular stereoscopic vision system of text image - Google Patents

Abstract

The invention discloses a text image geometric correction method, device and binocular stereo vision system, belonging to the field of image processing. Including: build a binocular stereo vision system, obtain the respective internal parameters and mutual external parameters of the two lenses in the binocular stereo vision system, and obtain the respective projection matrices of the two lenses; image collected by the two lenses in the binocular stereo vision system Carry out stereo matching after feature extraction respectively, and purify the obtained matching points; calculate the three-dimensional coordinates of the corresponding points in the text image according to the obtained matching points, and judge the text trend direction in the text image; fit the text trend according to the text trend direction Curve, the text trend curve is discretely unfolded and then reconstructed to obtain a corrected text image. The invention is not affected by the specific content and typesetting of the text, and is especially suitable for geometric distortion correction of text with complex layout, and can be used for processing various types of text images, greatly improving the applicable range of geometric distortion correction.

Description

Text image geometric correction method, device and binocular stereo vision system

technical field

The invention belongs to the field of image processing, and relates to an image processing method and device, in particular to a text image geometric correction method, device and binocular stereo vision system.

the

Background technique

There are two main ways to collect text images: scanning the text with a scanner; shooting with various cameras and cameras. When scanning with a scanner, it is usually necessary to disassemble or flatten the manuscript before scanning, which is generally not used. If the manuscript is directly scanned on a scanner or directly photographed by a camera or camera, due to factors such as the spine and thickness of the manuscript, it will inevitably cause distortion such as bending of the manuscript itself, which not only affects the appearance of the image, but also affects the subsequent OCR recognition application. Therefore, it is very important to perform geometric correction on the captured text image.

At present, the general text image geometric correction methods are mainly divided into two categories: one is to set some known reference points on the image, and perform geometric correction according to the mapping relationship of the reference points before and after distortion. The general implementation process of this method is as follows, Place a template on the text to be photographed, take a template image, obtain known reference points on the template image, and establish a mapping relationship with the reference points in the original template, and finally perform geometric correction according to the mapping relationship; the other method directly Correction is carried out by analyzing the characteristics of the text image itself, but this method cannot deal with general non-specific distorted images, and corrects according to some characteristics of such images obtained through processing.

There are two main purposes of text collection, one is for data preservation, usually for some precious data; the other is for OCR recognition. In general, in the process of image data collection, text images of book pages are collected as far as possible from horizontal and vertical angles, which is beneficial to subsequent processing.

For the first type of method, a certain template is attached to the document for scanning or shooting, which is inconvenient for both data preservation and OCR recognition. According to a known maze pattern template, Google uses a system including two cameras and infrared rays to solve the problem of book page bending. The method is to take infrared images from two infrared cameras from different angles, and use these images to Known stereoscopic imaging technologies are combined to obtain a 3D corresponding effect of the pattern. The pattern falls on the surface of the page, and the 3D figure according to the pattern corresponds to the 3D surface of the page, thereby flattening the page. This method overcomes the inconvenience of placing the template in the shooting process, but the method is complicated in system and high in cost.

For the second type of method, the correction is mainly based on the direction of the text line in the text as reference information. This method is largely affected by the text content and typesetting. For complex typesetting, there are not only text, tables, but also lace, Text images for illustrations, it is difficult to obtain good correction results. Chinese patent CN1804861A solves the geometric distortion problem of a certain type of text image by blacking out the run length of the image, dividing the segment on the run length map, and correcting the irregular part with the regular part in the segment. However, this type of method often requires preprocessing such as layout analysis and binarization first. The results of these preprocessing directly affect the quality of distortion correction, and this type of method mainly relies on the information of text lines. In the vertical case, nothing can be done.

the

Contents of the invention

The technical problem to be solved by the present invention is to propose a geometric correction method, device and binocular stereo vision system of a text image, which uses binocular stereo vision combined with curve fitting to reconstruct the text image, and has a good correction effect for complex images. Therefore, the visual effect of the image is effectively improved, which is beneficial to subsequent OCR processing and recognition.

The invention discloses a geometric correction method of a text image, comprising the following steps:

Step 1: Build a binocular stereo vision system, obtain the respective internal parameters and mutual external parameters of the two lenses in the binocular stereo vision system after system calibration, and obtain the respective projection matrices of the two lenses;

Step 2: Perform feature extraction on the images collected by the two lenses in the calibrated binocular stereo vision system, perform stereo matching according to the extracted image features, and purify the obtained matching points;

Step 3: Calculate the three-dimensional coordinates of the corresponding points in the text image according to the purified matching points and the internal parameters, external parameters and projection matrix obtained after calibration, so as to determine the direction of the text trend in the text image;

Step 4: Fit the text trend curve according to the obtained text trend direction, unfold the text trend curve discretely and perform image reconstruction to obtain a corrected text image.

In the first step, the binocular stereo vision system includes two identical first lenses and second lenses; the first lens is used to capture the text image to be corrected; the field of view of the second lens includes the text image captured by the first lens.

During the feature extraction in the step 2, a scale invariant feature transformation method is used for feature extraction.

When the matching points obtained in the step 2 are purified, the angles of the matching point lines are counted, 180 ° is divided into n intervals, the number of all matching point line angles falling into each interval is counted, and the points are taken The angle interval with the largest number is used as the range of matching point connection angles for preliminary purification, and then it is set according to the average distance between the matching points after preliminary purification, if

为当前匹配点的距离，

为匹配点间的平均距离，则当前匹配点为提纯得到的匹配点。 Among them, T is the set threshold,

is the distance of the current matching point,

is the average distance between matching points, and the current matching point is the purified matching point.

The interval n is 36, and the set threshold T is 0.1.

In the step 3, judging the text trend direction in the text image includes the following steps:

1. According to the set sub-band width, each matching point is divided into sub-bands in the horizontal and vertical directions;

2. Calculate the dispersion of the sub-band in the corresponding direction according to the coordinates of the corresponding direction of the matching point in each sub-band;

3. In the horizontal and vertical directions, take the sub-band with the largest product of the corresponding dispersion and the number of matching points, and calculate the change degree of the distance between the text image and the lens in the horizontal and vertical directions of each matching point in the two sub-bands, respectively. The direction corresponding to the greater degree of change is taken as the direction of the text trend.

When fitting the text trend according to the text trend direction obtained in the step 4, select the X direction or Y direction coordinate point in the three-dimensional coordinates of the corresponding point in the text image according to the text trend direction, and perform curve fitting with the Z direction coordinate point, Get the text trend curve.

When the text trend curve is discretely expanded in the step 4, the intersection point of the horizontal line and the text trend curve is used as the base point to calculate the curve length of each discrete point relative to the base point.

During the reconstruction, perform backward mapping according to the target position obtained by the discrete expansion of the text trend curve, calculate the position of each coordinate point in the original image in the text image corresponding to the expansion plane, and reconstruct by using bilinear interpolation to obtain the geometric Corrected text image.

When unfolding the text trend curve, the discrete step length of the curve is determined according to the three-dimensional space coordinates of the matching point and the corresponding image coordinates of the matching point, and the length of the curve of each discrete point relative to the base point is calculated with the base point as the starting point.

The invention also discloses a binocular stereoscopic vision system, which includes a first lens and a second lens, and the first lens and the second lens are fixed side by side on a horizontal bracket; the first lens is used for shooting images to be corrected The text image of the second camera; the field of view of the second camera includes the text image captured by the first camera.

The invention also discloses a geometric correction device for text images, which includes the following modules:

Building module: build a binocular stereo vision system, obtain the internal parameters and mutual external parameters of the two lenses in the binocular stereo vision system after system calibration, and obtain the respective projection matrices of the two lenses;

Extraction module: perform feature extraction on the images collected by the two lenses in the calibrated binocular stereo vision system, perform stereo matching according to the extracted image features, and purify the obtained matching points;

Trend module: Calculate the three-dimensional coordinates of the corresponding points in the text image according to the purified matching points and the internal parameters, external parameters and projection matrix obtained after calibration, so as to judge the text trend direction in the text image;

Reconstruction module: fit the text trend curve according to the obtained text trend direction, and reconstruct the text trend curve after discrete expansion, so as to obtain the corrected text image.

The invention discloses a text image geometric correction method, a device and a binocular stereo vision system, which make full use of the three-dimensional information of the text, predict, fit and expand the text direction, and directly perform the collected text without using a template image. The text image is processed, only two common cameras are used, the operation is convenient and the cost is low, and it is not limited by the degree of text bending and the direction of horizontal and vertical bending; Typesetting effects, especially for geometric distortion correction of text with complex layouts, can be used to process various types of text images, greatly improving the scope of application of geometric distortion correction.

the

Description of drawings

Fig. 1 is the flowchart of the geometric correction method of text image of the present invention;

Fig. 2 is the schematic diagram of the binocular stereo vision system in the geometric correction method of text image of the present invention;

Fig. 3 is the three-dimensional space point cloud in the geometric correction method of text image of the present invention;

Fig. 4a is the distorted image of the mixed arrangement of text formulas in the geometric correction method of the text image of the present invention;

Fig. 4b is the front-view image of the distorted image mixed with text formulas in the geometric correction method of the text image of the present invention;

Fig. 5 is the flowchart of text trend direction determination in the geometric correction method of text image of the present invention;

Fig. 6 is a schematic diagram of the subbands in the geometric correction method of the text image of the present invention;

Fig. 7a is a flipped view of the front-view image of the distorted image in the geometric correction method of the text image of the present invention;

Fig. 7b is a schematic diagram of the curve fitting result of the front-view image of the distorted image and the base point of the curve expansion in the geometric correction method of the text image of the present invention;

Fig. 8 is a schematic diagram of curve expansion and reconstruction in the geometric correction method of text image in the present invention;

Fig. 9 is a distorted image result diagram of mixed arrangement of text formulas in the geometric correction method of text images of the present invention;

Fig. 10a is a schematic diagram of a plain text text image before correction in the geometric correction method of a text image in the present invention;

Fig. 10b is a schematic diagram of a corrected plain text text image in the geometric correction method of a text image in the present invention;

Fig. 11a is a schematic diagram of horizontal and vertical mixed text images before correction in the geometric correction method of text images of the present invention;

Fig. 11b is a schematic diagram of a corrected horizontal and vertical mixed text image in the geometric correction method of the text image of the present invention;

Fig. 12a is a schematic diagram of a text image mixed with text and pictures before correction in the geometric correction method of the text image of the present invention;

Fig. 12b is a schematic diagram of text images after correction in the geometric correction method of text images in the present invention;

Fig. 13a is a schematic diagram of a text image containing many formulas before correction in the geometric correction method of a text image in the present invention;

Fig. 13b is a schematic diagram of a corrected text image containing many formulas in the geometric correction method of the text image of the present invention;

Fig. 14a is a schematic diagram of a vertical handwritten text image before correction in the geometric correction method of a text image in the present invention;

Fig. 14b is a schematic diagram of a corrected vertical handwritten text image in the geometric correction method for a text image of the present invention.

the

Detailed ways

In order to illustrate the technical solution and content of the present invention more clearly, the present invention will be further described in detail below in conjunction with the accompanying drawings.

According to the characteristics of text distortion, there are two main forms of distortion: one is caused by the shooting angle of the lens, such as perspective distortion; the other is caused by the geometric structure change of the text itself due to the existence of the text spine and the change of thickness distortion. Due to the existence of the spine of the text and the change of the thickness, the surface structure of the text has changed. This change is reflected in the text image. Has a certain regularity. After the book page is photographed, the text image formed has continuity and consistency from top to bottom (or from left to right) when it is bent, based on this consistency law, the present invention uses a curve fitting method to characterize the text The trend changes, and on this basis, the curve is expanded to eliminate the geometric distortion of the text image.

The present invention utilizes computer stereo vision technology to shoot text through the lenses of two cameras, perform feature extraction on the two images and then match them, and reconstruct the three-dimensional coordinates of the corresponding points on the text image after obtaining stereo matching points, and carry out the process according to the three-dimensional coordinates Text bending trend judgment and curve fitting, the fitting curve is unfolded and the text image is reconstructed, realizing the geometric distortion correction of the image.

The geometric correction method of the text image disclosed by the present invention, the flow chart is as shown in Figure 1, and the specific processing steps are as follows:

Step 1: Build a binocular stereo vision system. After system calibration, obtain the respective internal parameters and mutual external parameters of the two lenses in the binocular stereo vision system, and obtain the respective projection matrices of the two lenses.

An embodiment of the binocular stereo vision system built by the present invention is shown in Figure 2, the binocular stereo vision system includes two identical first lenses and second lenses; the first lens is used to take text images to be corrected; the second lens The field of view of the second camera includes the text image captured by the first camera. The first lens and the second lens are arranged side by side on a horizontal bracket C, and respectively shoot the text to be corrected, and it is required that the text image to be corrected should be within the shooting field of view of the two lenses as much as possible.

After the positions of the two lenses in the binocular stereo vision system are fixed, the two lenses are calibrated, and the internal parameters of the two lenses and the relative positional relationship between the two lenses are obtained, and the projection matrix of the two lenses is further obtained for the subsequent calculation of the object point. The three-dimensional coordinates are prepared.

The system calibration is to establish the relationship between the pixel position of the image captured by the two lenses and the position of the scene point, and calculate the model parameters of the lens from the image coordinates and world coordinates of the known feature points according to the lens model, so as to determine the image coordinate system of the camera and the three-dimensional The correspondence between the reference coordinate systems of objects in space, only when the lens is correctly calibrated, can the actual position of the corresponding object in three-dimensional space be calculated according to the two-dimensional coordinates in the image plane.

The camera lenses used in the present invention all meet the pinhole camera model, the lens is calibrated by using the Zhang Zhengyou calibration method, and the internal and external parameters of the camera are calculated according to multiple plane template images. Several commonly used coordinate systems in this embodiment are as follows:

在图像平面的左上角，每一像素的坐标分别是该像素在数组中的列数和行数，所以

是以像素为单位的图像坐标系的坐标。 (1) Image coordinate system : Defined by the digital image captured by the lens, the origin

In the upper left corner of the image plane, the coordinates of each pixel are the number of columns and rows of the pixel in the array, respectively, so

are coordinates in the image coordinate system in pixels.

：以物理单位表示图像坐标，

为原点，

轴与

轴分别与，

轴平行，设

在

，

坐标系中的坐标为

，每一个像素在

轴与

轴方向上的物理尺寸为

，

, 则图像中任意一个象素在两个坐标系下的坐标关系可表示为下面的公式： (2) Imaging plane coordinate system

: image coordinates expressed in physical units,

as the origin,

axis with

axis respectively with,

axis parallel to

exist

,

The coordinates in the coordinate system are

, each pixel at

axis with

The physical dimension in the axial direction is

,

, then the coordinate relationship of any pixel in the image under the two coordinate systems can be expressed as the following formula:

                     

               

分别为

轴的尺度因子。 in,

respectively

The scale factor of the axes.

_：

点称为镜头光心，

轴和

轴与图像的轴与轴平行，

轴为与图像平面垂直的镜头光轴。光轴与图像平面的交点，由点

与

，

，

轴组成的直角坐标系成为相机坐标系。镜头光心到图像平面的距离

为镜头的有效焦距。空间一点

经镜头成像后其像面坐标和镜头坐标关系为： (3) Camera coordinate system

_:

The point is called the optical center of the lens,

axis and

axes and images axis with axis parallel,

axis is the optical axis of the lens perpendicular to the image plane. The intersection of the optical axis and the image plane, given by the point

and

,

,

The Cartesian coordinate system composed of axes becomes the camera coordinate system. The distance from the optical center of the lens to the image plane

is the effective focal length of the lens . a little space

After imaging through the lens, the relationship between the image plane coordinates and the lens coordinates is:

That is, the relationship between the imaging plane coordinate system and the camera coordinate system can be written as:

：客观世界的绝对坐标。相机坐标系与相机坐标系之间的关系可以用旋转矩阵与平移向量来描述。对于空间中的某一点

在世界坐标系和相机坐标系下的齐次坐标如果分别是

与

，存在如下关系： (4) World coordinate system

: The absolute coordinates of the objective world. The relationship between the camera coordinate system and the camera coordinate system can use the rotation matrix with translation vector to describe. for a point in space

If the homogeneous coordinates in the world coordinate system and the camera coordinate system are

and

, there is the following relationship:

                    

                    

为

正交单位矩阵；

为平移向量；；

为

矩阵。 in,

for

Orthogonal identity matrix;

for translation vector; ;

for

matrix.

和旋转变换矩阵

中的参数称为外部参数。外部参数有六个，相应于

的用欧拉角表示的侧倾角

、俯仰角

、旋转角

，以及相应于平移矢量的三个分量

。于是旋转矩阵

可以表示为： The translation of the object point from the world coordinate system to the camera coordinate system

and the rotation transformation matrix

The parameters in are called extrinsic parameters. There are six external parameters, corresponding to

The roll angle expressed in terms of Euler angles

,Pitch angle

, rotation angle

, and corresponding to the translation vector three components of

. Then the rotation matrix

It can be expressed as:

_{The relationship between transforming from the world coordinate system to the image coordinate system is as follows:}

Zhang Zhengyou's planar calibration method avoids the disadvantages of high equipment requirements and cumbersome operation of traditional calibration methods, and has high precision and good robustness. In the embodiment of the present invention, its main steps are:

a: Print a checkerboard pattern and stick it on the plane as a calibration board. The size of the checkerboard is 20*20, and the side length of the checkerboard is 0.067m;

b: Move the calibration board, each lens takes 15 images (not less than 3) from different angles;

c: Detect all the corner points in each image, and replace the corner points of the image with the center points of the four corner points of the checkerboard;

d: Without considering the radial distortion, using the orthogonality of the rotation matrix, the five internal parameters and external parameters of the lens are obtained by solving the linear equation;

e: Using the principle of minimum back-projection error to optimize the internal and external parameters.

，其齐次坐标与像素坐标之间通过如下的针孔相机模型相关联： The specific implementation process of calculating internal and external parameters by using the detected corner point center point is as follows: for any target point on the 2D calibration board

, whose homogeneous coordinates are associated with pixel coordinates through the following pinhole camera model:

点在世界坐标系中的齐次坐标，为与齐次世界坐标系

有关的比例缩放因子。与

分别为从世界坐标系到相机坐标系的旋转矩阵和平移向量，即镜头的外参数。

为

镜头内参数矩阵，其具体形式如下所示： in, Respectively

The homogeneous coordinates of the point in the world coordinate system, for the homogeneous world coordinate system

The relevant scaling factor. and

are the rotation matrix and translation vector from the world coordinate system to the camera coordinate system, respectively, that is, the extrinsic parameters of the lens.

for

The lens intrinsic parameter matrix, its specific form is as follows:

，

为主点（又称作像面中心）的像素坐标，

分别为有效焦距在象素坐标系

方向上的尺度因子，

为倾斜因子，表示像素坐标系中两个主坐标轴之间的角度。在理想情况下，两轴正交，。为在镜头标定过程中求解方便，将主点位置与有效焦距分离开来，分别用独立的矩阵表示，其中

均为可逆矩阵。 in,

,

The pixel coordinates of the principal point (also known as the center of the image plane),

are the effective focal length in the pixel coordinate system

The scale factor in the direction,

is the tilt factor, representing the angle between the two main coordinate axes in the pixel coordinate system. Ideally, the two axes are orthogonal, . In order to facilitate the solution in the lens calibration process, the principal point position and the effective focal length are separated, and an independent matrix is used respectively said, among them

both reversible matrix.

轴的方向与此平面垂直，则标定板上目标点的世界坐标简化为

，将其代入并进行整理后得到下式： During the calibration process, the origin of the world coordinate system is selected on the plane of the calibration plate, and

The direction of the axis is perpendicular to this plane, then the target point on the calibration board The world coordinates of simplifies to

, after substituting it and sorting it out, the following formula is obtained:

表示

点在标定平面上的齐次坐标，

即为2D单应矩阵，它完全由镜头的内、外参数矩阵决定： in,

express

The homogeneous coordinates of the point on the calibration plane,

It is the 2D homography matrix, which is completely determined by the internal and external parameter matrix of the lens:

分别表示旋转矩阵

的前两列向量。因此，对于标定板上的任意目标点，单应矩阵

为不变量。

使实际图像坐标

与根据上述关系计算出的图像坐标

之间残差最小，目标函数为： in,

represent the rotation matrix, respectively

The first two columns of the vector. Therefore, for any target point on the calibration board, the homography matrix

as an invariant.

make the actual image coordinates

with the image coordinates calculated from the above relationship

The residual between is the smallest, and the objective function is:

以后，可以对镜头的内外参数矩阵进行标定，通过逐步优化来建立对于镜头内、外参数矩阵的精确估计。 Solve the 2D homography matrix

In the future, the internal and external parameter matrix of the lens can be calibrated, and an accurate estimation of the internal and external parameter matrix of the lens can be established through gradual optimization.

In this embodiment, the internal and external parameters of the first lens and the second lens are obtained through lens calibration, and the stereo camera in the Jean-Yves Bouguet camera calibration toolbox is used for calibration, and the internal parameters and calibration results of the above two lenses are imported, namely The external parameters of the second lens relative to the first lens in the binocular system can be obtained, the parameters are the rotation and translation parameters of the second lens relative to the first lens, R is a 3×3 matrix, and T is a 3×1 matrix.

，第二镜头相对于第一镜头的外部参数矩阵为

。 In this embodiment, after stereo vision calibration, the internal parameters of the first lens and the second lens are obtained as follows: and

, the extrinsic parameter matrix of the second lens relative to the first lens is

.

From this, the respective projection matrices M1 and M2 of the two lenses A and B can be obtained: M1 and M2 are both 3×4 matrices.

、

,

This step obtains the projection matrix M1 of the first lens and the projection matrix M2 of the second lens. So far, the positions and focal lengths of the first lens and the second lens have been fixed and cannot be changed.

Step 2: Perform feature extraction on the images collected by the two lenses in the calibrated binocular stereo vision system, and perform stereo matching according to the extracted image features.

Two text images IA and IB in different positions are collected through the binocular stereo vision system, where IA is the distorted image to be corrected, and the scale invariant feature transformation (Scale Invariant Feature Transform, hereinafter referred to as SIFT) method is used to carry out the two image features Extract and preliminary matches.

The SIFT feature is a local feature of the image. This feature remains invariant to scale scaling, rotation, and brightness changes, and also maintains a certain degree of stability against viewing angle changes, affine transformations, and noise.

The SIFT method is mainly divided into two parts: feature extraction and feature matching. The specific implementation steps are as follows:

1) Establishment of Gaussian scale space and Gaussian difference scale space: In this embodiment, there is almost no scale change in the two-lens images, so only Gaussian functions of 4 scales are convolved with the input image to establish a Gaussian scale space, and 3 Gaussians are obtained Difference kernel and image convolution generate Gaussian difference scale-space (Difference of Gaussian scale-space, referred to as DOG space);

2) Spatial extreme point detection: In this embodiment, a total of 3 scales of DOG space are generated. For a certain detection point at the second scale in DOG space, 8 adjacent points of the same scale as well as its upper and lower neighbors The 9×2 points corresponding to the first scale and the third scale are compared with a total of 8+18=26 points. If the detection point is the maximum or minimum value among the 26 points, the detection point is a local extremum point;

3) Accurate positioning of extreme points: by fitting a three-dimensional quadratic function to accurately determine the position and scale of local extreme points, and at the same time remove extreme points with low contrast and unstable edge response points. In this embodiment, The contrast threshold is set to 0.06, and the extreme points whose contrast is lower than the threshold are eliminated, otherwise they are retained;

4) Determine the direction parameter of the feature point: use the gradient direction distribution characteristics of the neighborhood pixels of the feature point to specify the direction parameter for each feature point, and rotate the coordinates to this direction, so that the operator has rotation invariance. In this embodiment, the 360° is divided into 10 interval statistical feature point directions;

5) Feature descriptor generation: First, the coordinate axis is rotated to the direction of the feature point to ensure rotation invariance, and then the feature descriptor is generated centering on the feature point. In this embodiment, a 16×16 window is taken with a certain feature point as the center, and a Gaussian weighted range is taken. The closer to the feature point, the greater the contribution of the gradient direction information of the pixel, and the greater the weight value. In each 4×4 Calculate the gradient direction histogram of 8 directions on the small block, count the cumulative value of each gradient direction, and form a seed point, a total of 4×4=16 seed points, each seed point has 8 direction vector information, that is, generate The characteristic descriptor of the feature point 4×4×8=128 dimensions;

6) Feature matching: SIFT feature extraction is performed on both IA and IB to obtain two sets of feature points. For a feature point in IA, find the first two feature points in IB with the closest Euclidean distance, if the value of the closest distance/second closest distance is less than a certain threshold, then accept this pair of matching points, otherwise eliminate the pair Matching points, the threshold is taken as 0.2 in this embodiment.

The initial stereo matching points of the two text images have been found. However, due to the high similarity of the content of the text, such as text, no matter what features and matching algorithms are used, wrong matching will inevitably occur. Matching points were further screened and purified.

The present invention uses a statistical method to eliminate mismatches for initial matching points. The connection line of each pair of matching points forms a certain angle with the horizontal line passing through the origin O of the image, and the connection direction of each pair of correct matching points is roughly the same. If it is a wrong match, the angle of the matching connection line will produce a large difference.

转换到

区间，将180°划分为一定数量的区间，本实施例中，取区间数为36，每5°作为一个区间，统计所有匹配点连线角度落入各个角度区间的点个数，取点个数最多的角度区间作为正确匹配连线的角度范围，根据这个范围，将明显的错配点去除。 Connect the matching points between the left and right images, and the angle range of the connection is , the specific method of angle statistics of connecting lines is as follows: the interval

convert to

Interval, 180° is divided into a certain number of intervals. In this embodiment, the number of intervals is taken as 36, and every 5° is used as an interval to count the number of points where the angles of all matching point lines fall into each angle interval. The angle interval with the largest number is used as the angle range of the correct matching line, and the obvious mismatch points are removed according to this range.

Due to the elimination of the angle, there may still be a mismatch between the angle and the correct angle, which will have adverse effects in the subsequent 3D space point positioning and curve fitting process, so it is necessary to ensure that the matching accuracy is high enough. When the point pairs are correctly matched, except that the connection angles are roughly the same, the mutual distance between the point pairs should not have too much difference. Therefore, the present invention further constrains the matching points obtained by the preliminary purification of angle constraints. to refine the matching points.

When purifying the obtained matching points, it is set according to the average distance between matching points, if

为当前匹配点的距离，

为匹配点间的平均距离，则当前匹配点为提纯得到的匹配点。本发明的实施例中，设定的阈值T取0.1。其匹配及提纯结果中正确匹配点连线的角度基本一致，且距离也基本一致，因此采用角度加距离约束的方法可以有效去除错误的匹配，提高匹配的精度。 Among them, T is the set threshold,

is the distance of the current matching point,

is the average distance between matching points, and the current matching point is the purified matching point. In the embodiment of the present invention, the set threshold T is 0.1. In the matching and purification results, the angles of the correct matching points are basically the same, and the distances are also basically the same. Therefore, the method of angle plus distance constraints can effectively remove wrong matches and improve the matching accuracy.

Step 3: Calculate the three-dimensional coordinates of the corresponding points in the text image based on the purified matching points and the internal parameters, external parameters and projection matrix obtained after calibration, so as to determine the direction of the text trend in the text image.

The lens obtains a two-dimensional image of a three-dimensional space object through perspective transformation, and there is a certain correspondence between the points in the image and the points on the actual object. The so-called three-dimensional reconstruction process of binocular stereo vision: that is, two images obtained by shooting a point in space with two lenses from different orientations, and deduce the position coordinates of the actual space point according to the corresponding relationship. According to the lens calibration completed in step 1 and the image stereo matching obtained in step 2, the three-dimensional coordinates corresponding to the matching points can be obtained.

，通过步骤二的立体匹配已经获得对应点

，根据步骤一已经获得的两镜头的投影矩阵M1，M2，则有： Under the pinhole camera model, it is assumed that the image points of any point P in space on the two lenses C1 and C2 are respectively

, the corresponding points have been obtained through the stereo matching in step 2

, according to the two-lens projection matrices M1 and M2 obtained in step 1, there are:

The three-dimensional coordinates of the corresponding points extracted in the text image can be obtained from the above formula, and the obtained three-dimensional point cloud in discrete space is shown in Figure 3. Through the three-dimensional coordinates of the corresponding points, the polynomial least square method is used for surface fitting, and the actual three-dimensional surface model of the text in the space can be obtained.

The method of unfolding or projecting by surface fitting is an idea to solve the geometric distortion of text, but observing the regularity and particularity of geometric deformation of text, curve fitting can simplify complex problems and transform the surface problem of three-dimensional space into Curve problems in two-dimensional space and avoid uncertain deformation and distortion in the process of surface fitting and unfolding.

By selecting two specific coordinate axes for curve fitting, you can obtain the curve trend of a certain angle view of the text. For the example of text distortion shown in Figure 4a, its front view is shown in Figure 4b. It can be seen that the front view of the text is a curve, and the distance between the text surface and the camera lens changes along the X direction by observing the distance between the text surface and the camera lens, so in this embodiment, X is selected. , Z two coordinates for fitting.

Since the text is usually opened in two situations when the camera collects data, left and right and up and down, the deformation direction of the text may be horizontal or vertical. In order to determine the two coordinates used for curve fitting, it is first necessary to determine the direction of the text trend.

坐标与

，

坐标的关系，若为横向走势，则

坐标与

坐标相关，若为纵向走势，则

坐标与

坐标相关。文本走势方向判定的处理流程图如图5所示，主要分为以下几步： When judging the direction of the text trend in the text image, due to the existence of the spine and the thickness of the text, the height of the text surface changes, and the collected image also produces a bending deformation. This bending deformation is the same as the text in the space coordinate system. There is a direct relationship between the distance, so to fit the text trend curve, as long as it is determined in the three-dimensional coordinate system,

coordinates with

,

The relationship between the coordinates, if it is a horizontal trend, then

coordinates with

Coordinates are related, if it is a vertical trend, then

coordinates with

Coordinates are related. The processing flow chart of text trend direction determination is shown in Figure 5, which is mainly divided into the following steps:

Step 301 : Divide each matching point into sub-bands in horizontal and vertical directions according to the set sub-band width; set the sub-band width, which is specifically set according to the size of the image. In the embodiment of the present invention, the sub-band width is set to 200 pixels, that is, every 200 pixels are divided into a sub-band. The width and height of the image are divided into several intervals according to the width of the sub-band, and one interval is a sub-band.

，u代表像素所在的行，v代表像素所在的列。对于横向子带，u相差不大于200个像素的匹配点落入同一个子带内，在该子带内匹配点的v坐标较均匀散布于子带各个区域才能最好的反映文本的走势变化。首先统计各个子带内匹配点的个数，并计算该子带内对应的v坐标的离散度，离散度

公式如下： Step 302: Calculate the dispersion of the sub-band in the corresponding direction according to the coordinates of the corresponding direction of the matching point in each sub-band. For sub-bands used to determine the text trend, the matching points in it should be distributed as evenly as possible in each area of the entire sub-band, so as to fully reflect the trend of the text. The sub-band positioning method is described below by taking horizontally divided sub-bands as an example. . The coordinates of the matching point in image IA are

, u represents the row where the pixel is located, and v represents the column where the pixel is located. For horizontal sub-bands, the matching points whose u difference is not more than 200 pixels fall into the same sub-band, and the v-coordinates of the matching points in this sub-band are more evenly distributed in each area of the sub-band to best reflect the trend of the text. First count the number of matching points in each subband, and calculate the dispersion of the corresponding v coordinates in the subband, the dispersion

The formula is as follows:

为每个点对应的坐标，

为对应坐标的均值，n为该子带内的匹配点个数。同理，对于纵向子带，则v相差不大于200个像素的匹配点落入同一个子带，在该子带内，计算u坐标的离散度。 in,

For the coordinates corresponding to each point,

is the mean value of the corresponding coordinates, and n is the number of matching points in the subband. Similarly, for vertical sub-bands, the matching points whose v difference is not greater than 200 pixels fall into the same sub-band, and within this sub-band, the dispersion of u coordinates is calculated.

Step 303: Take the sub-band with the largest product of the corresponding dispersion and the number of matching points in the horizontal and vertical directions, respectively, and calculate the change degree of the distance between the text image and the camera in the horizontal and vertical directions of each matching point in the two sub-bands , take the direction corresponding to the greater degree of change as the direction of the text trend.

作为评判的准则，取所有子带中对应

最大的子带作为后续文本走势判定的子带。图4a所示实施例的横纵方向对应子带如图6所示，横向框内的子带即为定位到的横向子带，纵向框所示的子带即为定位到的纵向子带。 It is not only required for the judgment of the text trend that the data dispersion in the sub-band should be as large as possible, that is, as scattered as possible, and the points in the sub-band should not be too few, so a compromise between the two is taken.

As a criterion for judging, take the corresponding

The largest sub-band is used as the sub-band for subsequent text trend determination. The corresponding sub-bands in the horizontal and vertical directions of the embodiment shown in FIG. 4a are shown in FIG. 6 , the sub-bands in the horizontal frame are the positioned horizontal sub-bands, and the sub-bands shown in the vertical frame are the positioned vertical sub-bands.

坐标沿两个方向的变化程度。用横向子带的数据点，计算三维坐标沿

坐标方向的变化程度；同样用纵向子带的数据点，计算三维坐标

沿

坐标方向的变化程度。取变化较剧烈方向的坐标与

坐标作为拟合的两个变量。 After obtaining the subbands in the horizontal and vertical directions, calculate them separately

How much the coordinates change in both directions. Use the data points of the horizontal subband to calculate the three-dimensional coordinates along

The degree of change in the coordinate direction; also use the data points of the longitudinal sub-band to calculate the three-dimensional coordinates

along

The degree of change in the coordinate direction. Take the coordinates in the direction of the more drastic change and

coordinates as the two variables of the fit.

沿

坐标方向的变化程度，用VarX表示，纵向子带反映的是

沿

坐标方向的变化程度，用VarY表示。以定位到的子带内第一个匹配点作为参考点，其对应坐标为

。以横向子带为例，每个匹配点对应的

相对于

的变化为sumX，v相对于

的变化为sumV，该子带内匹配点个数为n，则横向的变化程度为： The lateral subbands reflect the

along

The degree of change in the coordinate direction is represented by VarX, and the vertical sub-band reflects

along

The degree of change in the coordinate direction, expressed in VarY. Taking the first matching point in the located subband as the reference point, its corresponding coordinates are

. Taking the horizontal sub-band as an example, each matching point corresponds to

compared to

The change of sumX,v relative to

The change of is sumV, and the number of matching points in this subband is n, then the degree of horizontal change is:

Similarly, the degree of vertical change is:

沿

坐标方向的变化，

、

作为曲线拟合的两个变量；否则

沿

坐标方向的变化，

、

作为曲线拟合的两个变量。 If VarX>VarY, then

along

The change in the direction of the coordinates,

,

as two variables for the curve fit; otherwise

along

The change in the direction of the coordinates,

,

as two variables for the curve fit.

沿

方向变化较剧烈，因此

、

作为曲线拟合的两个变量，标志变量flag=1。 The specific embodiment shown in Figure 4a,

along

Direction changes are more drastic, so

,

As the two variables of curve fitting, flag variable flag=1.

Step 4: Obtain two variables for curve fitting according to the obtained text trend direction, fit the text trend curve, reconstruct the text image according to the discrete expansion of the text trend curve, and realize the geometric correction of the text image.

The coordinates of the three-dimensional space point obtained in step 3 and the direction of the text trend can be used to fit the curve, and the coordinate point in the X direction or the Y direction in the three-dimensional coordinates of the corresponding point in the text image is selected according to the direction of the text trend, and the coordinate point in the Z direction Perform curve fitting to obtain the text trend curve. Step 3 has obtained the three-dimensional space point cloud. If the effective information reflecting the text structure and bending can be extracted from the discrete three-dimensional space point cloud, the text can be flattened according to this law to achieve the purpose of geometric distortion correction.

变化，采用曲线拟合的方法得到描述文本走势曲线的函数

。拟合方法采用多项式拟合，其公式如下所示，公式中n为拟合的阶次。  In the embodiment shown in Figure 4a, flag=1, that is, the text trend is Coordinates along

change, using the method of curve fitting to obtain the function describing the text trend curve

. The fitting method adopts polynomial fitting, and its formula is shown below, where n is the order of fitting.

为走势曲线函数表达式的系数；求得多项式系数

即可拟合出文本走势曲线，图4a所示实施例的走势曲线镜像翻转后如图7a所示。拟合后得到的函数

对应的曲线如图7b所示，从图7b与图7a的文本前视图曲线比较可以看出，图7b所示的拟合曲线反映了文本书页的走势，即反映了文本面离镜头距离

沿

的变化规律。 in,

is the coefficient of the function expression of the trend curve; obtain the polynomial coefficient

The text trend curve can be fitted, and the trend curve of the embodiment shown in FIG. 4a is mirror-inverted, as shown in FIG. 7a. The function obtained after fitting

The corresponding curve is shown in Figure 7b. From the comparison of the text front view curve in Figure 7b and Figure 7a, it can be seen that the fitting curve shown in Figure 7b reflects the trend of the text page, that is, it reflects the distance between the text surface and the camera

along

change rule.

The geometric distortion correction of the text image is actually to eliminate the distance between the text and the camera lens caused by the change of the thickness of the text and the presence of the spine, and further cause distortion. As long as the change of this distance is eliminated, the purpose of eliminating distortion can be achieved .

变化，

，则曲线展开基点为曲线上的一点，如图7b经过

的竖直线与曲线交点所示为展开基点；若为纵向走势文本，即

沿

变化

，则

处为曲线展开基点。 In order to obtain a flattened text image, this method expands the curve reflecting the trend of the text into a straight line, that is, flattens the curved text page into a plane. Before the curve is unfolded, the base point of the curve expansion needs to be determined, and the intersection point of the horizontal line and the text trend curve is used as the base point to calculate the curve length of each discrete point relative to the base point. The present invention uses a pre-designated method to determine the base point of curve expansion, and specifies the intersection point of the optical axis of the lens and the text surface as the base point of expansion. For the embodiment shown in Figure 4a, it is the text of the horizontal trend, that is along

Variety,

, then the base point of the curve expansion is a point on the curve , as shown in Figure 7b after

The intersection of the vertical line and the curve is the expansion base point; if it is a vertical trend text, that is

along

Variety

,but

is the base point of curve expansion.

，事实上拟合的曲线为一堆离散的点。曲线展开即求曲线上每个点相对于基点的曲线长度，求取曲线的长度采用积分的思想，曲线离散化后求出每个离散点对应的曲线长度。 Curve fitting obtains the coefficients describing the function expression of the trend curve

, in fact the fitted curve is a bunch of discrete points. Curve expansion is to calculate the length of each point on the curve relative to the base point. The idea of integral is used to calculate the length of the curve. After the curve is discretized, the length of the curve corresponding to each discrete point is calculated.

，根据获取匹配点的三维坐标，可得三维坐标

的最大范围

及对应的图像坐标v的变化范围

，将空间中的

坐标量化为每个像素的变化量，步长

，为了提高曲线展开的精度，可以将步长缩小一定倍数进行曲线的离散展开。本发明实施例中，取离散步长为

，以基点为起点，计算与基点距离为

的每个离散点对应的曲线长度

。 According to the three-dimensional space coordinates of the matching points and the image coordinates corresponding to the matching points, the discrete step length of the curve is determined, and the base point is used as the starting point to calculate the length of the curve of each discrete point relative to the base point. The embodiment shown in Figure 4a, the text trend curve that obtains is

, according to the three-dimensional coordinates of the matching points, the three-dimensional coordinates can be obtained

The maximum range of

And the range of change of the corresponding image coordinate v

, the space in

The coordinates are quantized as the amount of change for each pixel, the step size

, in order to improve the accuracy of curve expansion, the step size can be reduced by a certain multiple for discrete expansion of the curve. In the embodiment of the present invention, the discrete step length is taken as

, taking the base point as the starting point, calculate the distance from the base point as

The length of the curve corresponding to each discrete point of

.

，且已知文本曲面展开后所在的平面为

，可求得该点在展开平面上的三维坐标，根据曲线离散展开可知

，由此可得该三维点在原曲面上的坐标为

，结合第一镜头的投影矩阵M1可求得目标图像上的

点对应在原图像上的坐标为

，通过双线性插值即可重建出畸变校正图像。如图4a所示实施例的畸变校正结果如图9所示，从图中可以看出，畸变的文字行基本被拉直。 After obtaining the curve discrete result, perform backward mapping according to the target position obtained from the discrete expansion of the text trend curve, calculate the position of each coordinate point in the original image in the text image corresponding to the expansion plane, and reconstruct it by bilinear interpolation, and obtain Geometrically corrected text image. Backward mapping is used to avoid holes and overlaps in the reconstructed image, and bilinear interpolation is used to avoid noise and image quality degradation introduced by non-integer mapping. Its reconstruction process is shown in Figure 8, for the matching points in the target image

, and the plane where the text surface is unfolded is known as

, the three-dimensional coordinates of the point on the unfolded plane can be obtained , according to the discrete expansion of the curve, we can know that

, so the coordinates of the 3D point on the original surface can be obtained as

, combined with the projection matrix M1 of the first lens, the

The coordinates corresponding to the point on the original image are

, the distortion-corrected image can be reconstructed by bilinear interpolation. The distortion correction result of the embodiment shown in FIG. 4 a is shown in FIG. 9 , and it can be seen from the figure that the distorted character lines are basically straightened.

The above specific implementation methods are mainly described for images taken roughly horizontally and vertically, and the three-dimensional space information of the text is obtained by processing the color or grayscale text image.

In other embodiments, Fig. 10a, Fig. 11a, Fig. 12a, Fig. 13a and Fig. 14a are processed by the method described in the present invention, and the processing results as Fig. 10b, Fig. 11b, Fig. 12b, Fig. 13b and Fig. 14b are obtained .

As shown in Figure 10a and Figure 10b, it can be seen from the figures that this method has a good correction effect on the plain text image with serious bending distortion, and the curved text lines are straightened.

As shown in Figure 11a and Figure 11b, it can be seen from the figures that this method also has a good correction effect on complex typesetting text images with mixed horizontal and vertical layouts. The correction process is only related to the text trend, but not to whether the text layout is horizontal or vertical. irrelevant.

As shown in Figure 12a and Figure 12b, it can be seen from the figures that this method also has an ideal correction effect on text images with pictures and texts, column divisions, etc., as long as the text content has favorable and sufficient features, no matter it is text or pictures. Perform better distortion correction.

As shown in Figure 13a and Figure 13b: there are fewer texts, fewer features, and more formulas in the figure. The general correction method based on text content may fail. It can be seen from the figure that the correction effect of this method is good.

As shown in Figure 14a and Figure 14b, it can be seen from the figures that the text is a vertical trend text and is a handwritten text, and the correction by this method is not affected by the specific content of the text, and also achieves a good correction effect. The upper part of the image is outside the overlapping part of the two shots and is in a vertical direction, so the text outside the overlapping area will not be expanded.

The invention also discloses a binocular stereo vision system, which includes a horizontal bracket, a first lens, and a second lens, and the first lens and the second lens are fixed side by side on the horizontal bracket; the first lens is used for Taking the text image to be corrected; the field of view of the second camera includes the text image taken by the first camera.

The invention also discloses a geometric correction device for text images, which includes the following modules:

Building module: build a binocular stereo vision system, obtain the internal parameters and mutual external parameters of the two lenses in the binocular stereo vision system after system calibration, and obtain the respective projection matrices of the two lenses;

Extraction module: perform feature extraction on the images collected by the two lenses in the calibrated binocular stereo vision system, perform stereo matching according to the extracted image features, and purify the obtained matching points;

Trend module: Calculate the three-dimensional coordinates of the corresponding points in the text image according to the purified matching points and the internal parameters, external parameters and projection matrix obtained after calibration, so as to judge the text trend direction in the text image;

Reconstruction module: fit the text trend curve according to the obtained text trend direction, and reconstruct the text trend curve after discrete expansion, so as to obtain the corrected text image.

The text image geometric correction method, device and binocular stereo vision system disclosed in the present invention do not need a template image, and directly process the collected text image, only two common cameras are used, the operation is convenient and the cost is low, The present invention is also suitable for attaching a template to a text image, and by correspondingly processing the template image, a trend curve is obtained and the image to be corrected is unfolded according to the curve, and the purpose of distortion correction can also be achieved; in addition, the geometric correction method adopted in the present invention It does not depend on the direction of the text line, whether it is horizontal or vertical, it can be corrected, and the correction process has no direct relationship with the image content form. as much information as possible.

The present invention makes full use of the three-dimensional text information obtained based on features, combines the curve fitting and unfolding method, reconstructs the image through backward mapping and bilinear interpolation, removes the geometric distortion of the image, and obtains the corrected text image, without the need for processing Preprocessing such as layout analysis and binarization does not require template images to obtain the corresponding corresponding relationship, and has the advantages of being not affected by the text content form and distortion degree and having a wide range of applications.

It can be seen from this embodiment that the text image geometric distortion correction technology of the present invention can deal with text distortions in various content and typesetting forms, significantly improving image quality, and in the process of OCR recognition, distortion correction makes it difficult to perform layout analysis , The line-segmented text can be processed smoothly, and the entire text image is uniformly corrected to avoid the adverse effects of distortions such as pictures, lace, formulas, and charts on OCR.

Claims (12)

1. A geometric correction method of a text image is characterized by comprising the following steps:

the method comprises the following steps: building a binocular stereo vision system, obtaining respective internal parameters and mutual external parameters of two lenses in the binocular stereo vision system after system calibration, and solving respective projection matrixes of the two lenses;

step two: respectively extracting the characteristics of the images collected by the two lenses in the calibrated binocular stereo vision system, carrying out stereo matching according to the extracted image characteristics, and purifying the obtained matching points;

step three: calculating the three-dimensional coordinates of corresponding points in the text image according to the matching points obtained by purification and the internal parameters, external parameters and projection matrix obtained after calibration, thereby judging the text trend direction in the text image;

step four: and fitting a text trend curve according to the obtained text trend direction, and reconstructing the text trend curve after discretely expanding the text trend curve so as to obtain a corrected text image.

2. The method of claim 1, wherein: the binocular stereoscopic vision system in the first step comprises a first lens and a second lens which are the same; the first lens is used for shooting a text image to be corrected; the field of view of the second lens includes the text image captured by the first lens.

3. The method of claim 1, wherein: and in the second step, during feature extraction, a scale-invariant feature transformation method is adopted for feature extraction.

4. The method of claim 1, wherein: when the obtained matching points are purified in the second step, dividing 180 degrees into n intervals, counting the connecting line angles of the matching points, taking the interval with the largest number of matching points to carry out primary purification on the matching points, then setting according to the average distance between the matching points after the primary purification, and if so, carrying out the primary purification on the matching points

Wherein T is a set threshold value,

is the distance between the current matching points,

and if the distance is the average distance between the matching points, the current matching point is the matching point obtained by purification.

5. The method of claim 4, wherein: the interval n is 36, and the set threshold T is 0.1.

6. The method of claim 2, wherein: and judging the trend direction of the text in the text image in the third step, which comprises the following steps:

respectively carrying out sub-band division on each matching point in the horizontal direction and the vertical direction according to the set sub-band width;

calculating the dispersion of the sub-band in the corresponding direction according to the coordinates of the corresponding direction of the matching point in each sub-band;

and respectively taking the sub-bands with the maximum product of the corresponding dispersion and the number of the matching points in the horizontal and vertical directions, respectively calculating the change degree of the text image in the horizontal and vertical directions from the first lens distance of each matching point in the two sub-bands, and taking the direction with the larger change degree as the text tendency direction.

7. The method of claim 6, wherein: and in the fourth step, when the text trend is fitted according to the obtained text trend direction, selecting an X-direction or Y-direction coordinate point in the three-dimensional coordinates of the corresponding point in the text image according to the text trend direction, and performing curve fitting with the Z-direction coordinate point to obtain a text trend curve.

8. The method of claim 1, wherein: and in the fourth step, when the text trend curve is discretely expanded, calculating the curve length of each discrete point relative to the base point by taking the intersection point of the horizontal line and the text trend curve as the base point.

9. The method of claim 1, wherein: when the text trend curve is discretely expanded and then reconstructed, backward mapping is carried out according to a target position obtained by the discrete expansion of the text trend curve, the position of each coordinate point in the text image corresponding to the expansion plane in the original image is calculated, and the reconstruction is carried out in a bilinear interpolation mode, so that the text image after geometric correction is obtained.

10. The method of claim 8, wherein: when the text trend curve is expanded, the discrete step length of the curve is determined according to the three-dimensional space coordinates of the matching points and the image coordinates corresponding to the matching points, and the curve length of each discrete point relative to the base point is calculated by taking the base point as a starting point.

11. The utility model provides a binocular stereoscopic vision system, includes first camera lens, second camera lens, its characterized in that: the first lens and the second lens are fixed on the horizontal bracket in parallel; the first lens is used for shooting a text image to be corrected; the field of view of the second lens includes a text image captured by the first lens.

12. A geometric correction device for text images is characterized by comprising the following modules:

building a module: building a binocular stereo vision system, obtaining respective internal parameters and mutual external parameters of two lenses in the binocular stereo vision system after system calibration, and solving respective projection matrixes of the two lenses;

an extraction module: respectively extracting the characteristics of the images collected by the two lenses in the calibrated binocular stereo vision system, carrying out stereo matching according to the extracted image characteristics, and purifying the obtained matching points;

a trend module: calculating the three-dimensional coordinates of corresponding points in the text image according to the matching points obtained by purification and the internal parameters, external parameters and projection matrix obtained after calibration, thereby judging the text trend direction in the text image;

a reconstruction module: and fitting a text trend curve according to the obtained text trend direction, and reconstructing the text trend curve after discretely expanding the text trend curve so as to obtain a corrected text image.

Images