CN108198219A - Error compensation method for camera calibration parameters for photogrammetry - Google Patents

Abstract

The invention discloses an error compensation method for camera calibration parameters used in photogrammetry, comprising: S1, measuring and calculating the offset relative to the reference plane in the z-axis direction of the feature points of the plane target; S2, calibrated The initial value of the camera parameter without error compensation; S3. According to the initial value, it is obtained that the offset of the z-axis direction of the feature point of the planar target relative to the reference plane causes the corresponding feature point on the corresponding image of the planar target The offset of the pixel coordinates; S4. Correct the pixel coordinates on the image corresponding to the feature points on the planar target through the offset correction process; S5. Use the corrected image coordinates to re-calibrate the camera parameters. Compared with the prior art, the technical solution of the present invention has the advantages that the high-precision target processing costs are high, and the pixel coordinate error compensation method derived from the uneven plane target can improve the accuracy of the calibration result without increasing the cost.

Description

Error Compensation Method for Camera Calibration Parameters Used in Photogrammetry

technical field

The invention relates to the technical field of photogrammetry, in particular to an error compensation method for camera calibration parameters used in photogrammetry. It can be specifically applied to compensate the camera calibration parameter error caused by the flatness error of the planar target application.

Background technique

Photogrammetry is widely used in artificial intelligence, vision measurement and robotics. Camera calibration is a key step in photogrammetry, and the accuracy of calibration results directly affects the accuracy of measurement results. Planar targets are widely used because of their simple processing and stable calibration algorithms. In order to further improve the accuracy of calibration results, it is a method to improve the accuracy of calibration results by considering the errors existing in each link in the calibration process and performing error compensation. Among them, the machining accuracy of the planar calibration target has a particularly important influence on the calibration results. Therefore, improving target processing accuracy or performing additional target processing error compensation are methods to improve the accuracy of calibration results from two perspectives.

At present, improving the calibration accuracy of planar targets is mainly carried out from two aspects: improving the manufacturing accuracy of the calibration target and improving the extraction accuracy of target feature points. For example, in the patent application number CN201310140445.5 submitted by Wu Hongjie et al., a method for compensating the calibration error of multi-dimensional feature cameras is proposed, which mainly compensates the center error of the light target by calculating the x-axis and y-axis deviation of feature points. This solution cannot compensate for z-axis deviation.

In addition, in the patent application number CN201210035098.5 submitted by Zhou Fuqiang et al., a method for optimizing parameters of a visual measurement camera is proposed. This method corrects the distortion of the image coordinates of the feature points, calculates the projection ray equation connecting the projection center and the feature points of the undistorted image; compares the three-dimensional coordinates of the feature points in the camera coordinate system with the intersection point of the projection ray and the corresponding target plane, and uses The distance between the two is used as the objective function to optimize the camera parameters. This solution cannot compensate for the error caused by the unevenness of the planar target.

Furthermore, in the patent No. CN201510408436.9 proposed by Zhao Jian, a compensation method for image parameters is proposed. This method only calibrates the target for the planar checkerboard, and the method calculates the root mean square error between the three-dimensional distance between the adjacent feature points of the calibration image and the standard distance by taking the minimum distance between the grid points as the standard distance, and then The image feature points of each test image are back-projected to the target plane to form an intersection point, and the root mean square error of the distance between it and all spatial feature points is calculated. This scheme can only find the root mean square error, so the error compensation is not precise.

Or, in the patent application number CN201610608257.4 proposed by Zhang Fumin et al., a robot online error compensation method for a photography system is proposed. This method uses a two-dimensional inclinometer to measure the angle and attitude of the robot end in two directions, combined with an angle data with high precision calculated by the robot itself, it can realize the measurement of the three-dimensional attitude of the robot end, and perform data fusion with the attitude data measured by the camera system , After comparison, the compensation value is obtained, and the industrial robot is controlled to compensate the error. In this scheme, the posture data is directly calibrated and the robot's own solution is compared and compensated, and it is inherently difficult to compensate the error caused by the unevenness of the planar target.

It can be seen from this that there is still a lack of an effective compensation method for the offset in the Z-axis direction of the uneven target in the prior art, which needs to be improved urgently.

Contents of the invention

In view of the above-mentioned problems in the prior art, the object of the present invention is to provide an error compensation method for camera calibration parameters used in photogrammetry that can further improve the accuracy of camera calibration results.

In order to achieve the above purpose, an embodiment of the present invention provides an error compensation method for camera calibration parameters used in photogrammetry, including:

S1. Measure and calculate the offset relative to the reference plane in the z-axis direction of the feature points of the planar target;

S2. Calibrate the initial value of the camera parameters without error compensation;

S3. According to the initial value, it is obtained that the offset of the z-axis direction of the feature point of the planar target relative to the reference plane causes the offset of the pixel coordinate of the corresponding feature point on the corresponding image of the planar target;

S4. Correct the pixel coordinates on the image corresponding to the feature points on the planar target through an offset correction process;

S5. Perform camera parameter calibration again with the corrected image coordinates.

Preferably, step S1 includes: first setting the z-axis coordinate of any one of the feature points on the planar target to be 0, and secondly measuring the relationship between the other feature points on the planar target and any one of the feature points The coordinate value of the z-axis direction of the above measurement process, the coordinates of the characteristic points measured by the above measurement process are (x _n , y _n , z _n ), and the equation of the reference plane ax+by+z+c=0 is obtained by the least square method, so The aforementioned offset Δz _n =z _n +(ax _n +by _n +c).

Preferably, the offset correction process in step S4 includes: obtaining a homography matrix based on the initial value and the offset The pixel coordinates in the image corresponding to the feature points are (u _n , v _n ), and the corrected pixel coordinates are (u′ _n , v′ _n ), wherein the pixel coordinate compensation formula is u′ _n =u _n - h ₃ Δz _n and v' _n = v _n -h ₇ Δz _n .

Preferably, after step S5, steps S4 and S5 are repeated n times, and n is an integer greater than 0.

Compared with the prior art, the technical solution of the present invention has the advantages that the high-precision target processing costs are high, and the pixel coordinate error compensation method derived from the uneven plane target can improve the accuracy of the calibration result without increasing the cost. This scheme uses the least square method when calculating the offset in the z-axis direction of the uneven target, which makes the result of the offset more reliable and improves the calculation accuracy of the image coordinate compensation process. The optimization process of this scheme requires iteration, which is conducive to further improving the accuracy of the calibration results, and the number of iterations can be set according to the measurement accuracy requirements, and the algorithm has high practicability.

Description of drawings

Fig. 1 is the basic flowchart of the error compensation method of the camera calibration parameter that is used for photogrammetry of the present invention;

Fig. 2 is the algorithm flowchart of an embodiment of the error compensation method for camera calibration parameters used in photogrammetry of the present invention;

Fig. 3 is the mathematical model presented by image points and spatial points of the error compensation method for camera calibration parameters used in photogrammetry of the present invention;

FIG. 4 is an initial parameter calibration process of the error compensation method for camera calibration parameters used in photogrammetry according to the present invention.

Detailed ways

In order to enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

These and other characteristics of the invention will become apparent from the following description of preferred forms of embodiment given as non-limiting examples with reference to the accompanying drawings.

It should also be understood that while the invention has been described with reference to a few specific examples, those skilled in the art can certainly implement many other equivalent forms of the invention, which have the features described in the claims and thus lie within the scope of the present invention. within the limited scope of protection.

Specific embodiments of the present invention are hereinafter described with reference to the accompanying drawings; however, it should be understood that the disclosed embodiments are merely examples of the invention, which may be embodied in various ways. Well-known and/or repeated functions and structures are not described in detail to ascertain the true intention based on the user's historical operations, and to avoid obscuring the present invention with unnecessary or redundant detail. Therefore, specific structural and functional details of the invention herein are not intended to be limiting, but merely as a basis for the claims and as a representative basis for teaching one skilled in the art to variously employ the present invention in virtually any suitable detailed structure. invention.

This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," which may refer to the same or one or more of the different embodiments.

The main purpose of the present invention is to optimize the calibration parameters by compensating the deviation of the pixel points on the corresponding image caused by the unevenness of the plane target through an iterative method. First, the initial value of the camera parameters needs to be calibrated, and then the offset in the z-axis direction of the feature points of the plane target is measured. Use the least squares method to find the offset of the feature point of the plane target in the z-axis direction relative to the reference plane, and then deduce the offset calculation formula of the pixel coordinates of the corresponding feature point on the image according to the camera parameter mathematical model, and correct the feature After the offset of the pixel coordinates, re-calibrate the camera parameters, get the new calibration result as the initial value and repeat the above steps, and the result after n iterations is the final calibration result. Fig. 1 is shown as the basic flowchart of the present invention, as shown in the figure, a kind of error compensation method of the camera calibration parameter that the present invention proposes is used for photogrammetry, comprises:

S1. Measure and calculate the offset relative to the reference plane in the z-axis direction of the feature points of the planar target;

S2. Calibrate the initial value of the camera parameters without error compensation;

S3. According to the initial value, it is obtained that the offset of the z-axis direction of the feature point of the planar target relative to the reference plane causes the offset of the pixel coordinate of the corresponding feature point on the corresponding image of the planar target;

S4. Correct the pixel coordinates on the image corresponding to the feature points on the planar target through an offset correction process;

S5. Perform camera parameter calibration again with the corrected image coordinates.

In the embodiment of the present invention, the initial values of the camera parameters include: internal camera parameters, and rotation matrices, translation vectors and scaling factors of the planar target relative to the two camera coordinate systems in different attitudes. The initial parameters are obtained by Zhang Zhengyou's calibration method.

The method for solving the offset of the feature point of the planar target relative to the reference plane in the z-axis direction is as follows: firstly, assuming that any point on the planar target is a coordinate circle point, measure the z-axis coordinate values at other points, and according to the description on the target The measured feature point coordinates are (x _n , y _n , z _n ), and the least square method is used to obtain the reference plane equation Ax+By+z+C=0, and the offset Δz _n =z _n +( Ax _n +By _n +C).

The formula for solving the offset of the pixel coordinates of the corresponding feature points on the image deduced by this scheme is as follows: firstly, the initial value of the calibration result is calculated according to Zhang Zhengyou’s calibration method, and the homography matrix is calculated according to the initial value

The pixel coordinates in the image corresponding to the feature points are (u _n , v _n ), and the corrected pixel coordinates are (u′ _n , v′ _n ), where u′ _n = _un -h ₃ Δz _n , so Said v' _n =v _n -h ₇ Δz _n .

Taking the pixel coordinate value obtained from the above solution after error compensation as the initial known parameters combined with the plane coordinates of the feature points on the target, Zhang Zhengyou’s calibration formula is used to solve the compensated camera parameters.

Calculate the new homography matrix with the second compensated camera parameters as the initial value, according to the offset calculation formula of the pixel coordinates of the corresponding feature points on the above image combined with the new homography matrix value to solve the homography matrix on the image The pixel coordinates after the offset compensation of the corresponding feature point pixel coordinates are recalibrated and the camera parameters are repeated n times.

The camera parameters obtained after n times of optimization are used as the final calibration camera parameters, which are generally repeated more than 4 times.

The advantage of this scheme compared with other methods is that the processing cost of high-precision targets is high, and the pixel coordinate error compensation method derived from this scheme can improve the accuracy of calibration results without increasing the cost. This scheme uses the least square method when calculating the offset in the z-axis direction of the uneven target, which makes the result of the offset more reliable and improves the calculation accuracy of the image coordinate compensation process. The optimization process of this scheme requires iteration, which is conducive to further improving the accuracy of the calibration results, and the number of iterations can be set according to the measurement accuracy requirements, and the algorithm has high practicability.

Please also refer to Figure 2, the flow chart of the error compensation algorithm, and the specific implementation method of each step is as follows.

The first step assumes that a certain feature point on the target is the coordinate origin, and measure the z-axis coordinate values of other points. The x-axis and y-axis coordinates of the feature point are known. The measured feature point coordinates are (x _n , y _n , z _n ), representing the three-dimensional coordinates of the nth feature point, and there are N feature points in total.

Let the datum plane equation be Ax+By+z+C=0, where A, B, and C are unknown quantities, and the least square method solution process is as follows:

Let ε=Ax+By+z+C

Minimize the residual:

∑ε ² ＝∑(Ax+By+z+C) ² 1.1

1.4 It can be seen from formula 1.2-1.4:

Putting formula 1.5 into formula 1.2-1.4 above can get:

N∑xz-∑x∑z＝-A(∑x ² -∑x∑x)-B(N∑xy-∑x∑y) 1.6

z′ ₁ =N∑xz-∑x∑z 1.7

a ₁ =∑x ² -∑x∑x 1.8

b ₁ =N∑xy-∑x∑y 1.9

N∑yz-∑y∑z＝-A(∑xy-∑x∑y)-B(N∑y ² -∑y∑y) 1.10

z′ ₂ =N∑yz-∑y∑z 1.11

a ₂ =∑xy-∑x∑y 1.12

b ₂ =N∑y ² -∑y∑y 1.13

-z′ ₁ ＝Aa ₁ +Bb ₁ 1.14

-z' ₂ =Aa ₂ +Bb ₂ 1.15

In the third step, according to the obtained reference plane Ax+By+z+C=0, the offset Δz _n in the z-axis direction of the feature point of the plane target is calculated as:

Δz _n =z _n +(Ax _n +By _n +C). 1.19

The fourth step is to calibrate the target through the plane checkerboard or the plane target of the circular mark point, and write the calibration program in OpenCV according to the calibration principle of Zhang Zhengyou. The initial parameter calibration process is shown in Figure 2. The binocular camera is used to capture multiple images of the planar target at the same time, and the initial parameters of the camera are calculated from the position coordinate relationship of the feature points on the image and the calibration target. The mathematical model of the relationship between the pixel coordinates of the reaction image and the coordinates of the feature points on the target in Zhang Zhengyou’s scheme is as follows:

According to Zhang Zhengyou’s plane target camera parameter model, the relationship between the coordinates (u, v) of the pixel point on the image and the coordinates (x, y, z) of the corresponding space point is:

Where s is a scale factor, H is a homography matrix, H=A[R|T] where R is a rotation matrix, T is a translation vector, and A is a camera intrinsic parameter. Assuming that the points on the target are in a plane, the above formula can be transformed into:

The fifth step is to set the number of optimization iterations to N (N>0), and N represents the number of repetitions of the optimization iteration process, which is used for the optimization program to finally determine whether to complete the optimization output result. The number of iteration flag bit n is automatically incremented by 1 for each iteration. After each iteration, the size relationship between the iteration flag bit n and N is judged. When n is greater than or equal to the number of iterations N set by the initial user, the iteratively optimized camera is output. parameters as the final result.

The sixth step is to derive the corrected pixel coordinates according to the camera parameter mathematical model. The specific derivation process is:

As shown in Figure 3, it represents the relationship between the pixel coordinates and the target coordinate system. In Zhang Zhengyou’s model, the target is assumed to be a plane target, but the actual target’s z-axis coordinate value is not zero due to the unevenness of the plane target. The above steps calculate The offset of the target z-axis direction relative to the reference plane is shown. In Fig. 4, it is assumed that the planar target has three-dimensional coordinates, and the target is imaged and projected on the image plane through the camera lens. According to the imaging model, there is a certain mathematical relationship between the point on the target and the point on the corresponding image. In the figure, R and T represent the rotation matrix and translation vector between the target coordinate system and the image plane coordinate system, respectively. The projection point of the feature point A on the target on the image plane is A'.

For an uneven planar target, the correspondence between the coordinates (x, y, z) of the feature points on it and the coordinates (u _e , v _e ) of the pixels on the image with errors can be expressed as:

Among them, the homography matrix and scale factor are the initial values calibrated by the uneven target through Zhang Zhengyou's calibration method. From formulas 1.21 and 1.22 above, we can get:

u=h ₁ x+h ₂ y 1.23

v=h ₅ x+h ₆ y 1.24

u _e =h ₁ x+h ₂ y+h ₃ z 1.25

v _e =h ₅ x+h ₆ y+h ₇ z 1.26

Where z is Δz _n , and the pixel coordinates in the image corresponding to the nth feature point are (u _n , v _n ), and the corrected pixel coordinates are (u′ _n , v′ _n ), it can be known from formulas 1.23-1.26 :

u′ _n ＝u _n -h ₃ Δz _n 1.27

v' _n =v _n -h ₇ Δz _n 1.28

Substitute 1.19 into 1.27 and 1.28 to get:

u′ _n ＝u _n -h ₃ (z _n +(Ax _n +By _n +C)) 1.29

v′ _n ＝v _n -h ₇ (z _n +(Ax _n +By _n +C)) 1.30

In the seventh step, the corrected pixel coordinates (u′ _n , v′ _n ) and the coordinates (x, y) of the point on the corresponding plane target are substituted into the equation as known conditions:

According to Zhang Zhengyou's planar target calibration method, the internal and external parameters of the camera are solved, where s is the scale factor, A is the internal reference, R is the rotation matrix, and T is the translation vector.

In the eighth step, each time the optimization is solved, the iteration flag n is automatically incremented by one, and n represents that the iterative optimization has been performed n times.

The ninth step is to judge the relationship between the iteration flag bit n and the number of optimization iterations set to N. If n<N, take the nth compensated camera parameters obtained in the eighth step as the initial value and return to the sixth step and repeat again. Calculate from step six to step eight. Taking the camera parameters obtained after error compensation as the initial value, resubstituting the corrected pixel coordinates (u′ _n , v′ _n ) and the coordinates (x, y) of the point on the corresponding plane target into the above formula, and solving New camera parameters. Then judge. Repeating this step n times will improve the precision of the optimization result. Until n is greater than or equal to N, the result after N times of optimization is used as the final calibration camera parameter. That is, the compensation of the camera calibration parameter error caused by the flatness error of the planar target is completed.

Various operations or functions are described herein, which may be implemented or defined as software codes or instructions. Such content may be directly executable ("object" or "executable" form) source code or differential code ("delta" or "patch" code). A software implementation of the embodiments described herein may be provided via an article of manufacture having code or instructions stored therein, or via a method of operating a communication interface to send data via the communication interface. A machine or computer-readable storage medium can cause the machine to perform the described functions or operations and includes any mechanism for storing information in a form accessible by the machine (e.g., computing device, electronic system, etc.), such as recordable/non-recordable media (eg, read only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, etc.). A communication interface includes any mechanism that interfaces to any of hardwired, wireless, optical, etc. media to communicate with another device, such as a memory bus interface, processor bus interface, Internet connection, disk controller, and the like. The communication interface may be configured by providing configuration parameters and/or sending signals to prepare the communication interface to provide data signals describing the software content. The communication interface may be accessed via one or more commands or signals sent to the communication interface.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims (4)

1. a kind of error compensating method for photogrammetric camera calibration parameter, including：

S1, the offset of the relative datum plane in z-axis direction for measuring and calculating the plane target drone characteristic point；

S2, the initial value for not adding camera parameter under error compensation is calibrated；

S3, according to the initial value, show that the offset of the relative datum plane in the z-axis direction of the plane target drone characteristic point is drawn Play the offset of character pair point pixel coordinate in the plane target drone correspondence image；

S4, pass through the pixel coordinate in the characteristic point correspondence image described in offset correction course corrections on plane target drone；

S5, camera parameter calibration is re-started with the image coordinate after the correction.

2. it to be used for the error compensating method of photogrammetric camera calibration parameter as described in claim 1, which is characterized in that step Rapid S1, including：The z-axis direction coordinate of any one of characteristic point on the plane target drone is set first as 0, next is measured The z-axis direction coordinate value of the relatively any one of characteristic point of other described characteristic points, above-mentioned to measure on the plane target drone The feature point coordinates that journey is measured is (x_n,y_n,z_n), obtain the datum plane equation ax+by+z+c=0 with least square method, The offset Δ z_n=z_n+(ax_n+by_n+c)。

3. it to be used for the error compensating method of photogrammetric camera calibration parameter as described in claim 1, which is characterized in that step The offset correction processes in rapid S4, including：According to the initial value and the offset, homography matrix is obtainedPixel coordinate in the corresponding image of the characteristic point is (u_n, v_n), the pixel after correction Coordinate is (u '_n, v '_n), wherein the pixel coordinate compensation formula is u '_n=u_n-h₃Δz_nWith v '_n=v_n-h₇Δz_n。

4. it to be used for the error compensating method of photogrammetric camera calibration parameter as described in claim 1, which is characterized in that step Rapid S5 and then S4 and S5 step n times are repeated, the n is the integer more than 0.