CN106971408A - A kind of camera marking method based on space-time conversion thought - Google Patents

Abstract

The invention relates to a camera calibration method based on the idea of time-space conversion, which belongs to the field of computer vision detection and image detection, and relates to a camera calibration method based on the idea of time-space conversion. In this method, two turntables positioned at a certain angle to each other and a laser installed on one of the turntables form a three-dimensional turntable device. The laser is controlled by the independent motion of the two turntables to sweep the laser point on the wall, and then passes through time and space. According to the transformation idea, obtain the relational expression between the sweeping distance and the rotation time of the turntable, use the camera to shoot the laser point image at the corresponding time point, obtain a large number of corresponding two-dimensional space feature points, and then according to the principle of the pinhole imaging model, respectively get Intrinsic and extrinsic parameters of the camera. The method does not need to occupy too much space when performing calibration, and the speed of projecting target points is also very fast, which saves calibration time, and thus can effectively adapt to the environment of the processing site.

Description

A camera calibration method based on the idea of time-space transformation

technical field

The invention belongs to the field of computer vision detection and image detection, and relates to a camera calibration method based on the idea of time-space conversion.

Background technique

In computer vision and image detection applications, in order to determine the relationship between the three-dimensional geometric position of a point on the surface of a space object and its two-dimensional corresponding point in the image, it is necessary to establish a geometric model of camera imaging, and these geometric model parameters are camera parameters. Under most conditions, these parameters must be obtained through experiments and calculations. The process of solving the parameters is camera calibration. Therefore, the calibration of camera parameters is a very critical link in visual inspection. The accuracy of the calibration results and the stability of the algorithm directly affect the accuracy of the results produced by the camera. Therefore, simplifying camera calibration and improving calibration accuracy are the focus of current research work. However, the existing commonly used calibration methods almost all require a calibration plate or other stereoscopic reference objects as targets to assist calibration, and then obtain relevant parameters of the camera through multiple groups of cameras. However, when performing on-site measurement, due to the complex environment, the time and space for calibration are extremely limited, and the movement of the measured object will make it difficult for conventional calibration methods to complete the calibration quickly and effectively, resulting in extended measurement periods, reduced real-time measurement and visual The accuracy of the measurement decreases.

At present, the commonly used calibration method is the calibration method based on the plane calibration board proposed by Zhang Zhengyou in the article "A Flexible New Technique for CameraCalibration". This method is a method between traditional calibration and self-calibration. It only needs the camera to A calibration board takes multiple pictures from different directions, and the camera is calibrated through the correspondence between each feature point on the calibration board and the image point of its image plane, that is, the homography matrix of each image. Applications. This method has high precision, but it needs a calibration plate to assist in calibration. The calibration accuracy depends on the accuracy of the calibration plate, and the calibration plate is expensive. At the same time, the algorithm of Zhang’s calibration method is complicated, the calibration takes a long time, and it needs to occupy a lot of space during calibration. It is inconvenient to measure on site, so it is not suitable for on-line adjustment of camera parameters.

Contents of the invention

The technical problem to be solved by the present invention is to overcome the shortcomings of the existing calibration methods, and to invent a camera calibration method based on the idea of time-space conversion for the lack of effective camera calibration methods in industrial measurement sites. This method uses laser scanning on the wall plane to obtain a large number of position feature point information, and calibrates the parameters of the camera according to the obtained feature points, and obtains the coordinates of the principal point including the optical axis passing through the image plane, the equivalent focal length, and the rotation of the camera. Matrix and the translation matrix between cameras and other parameters. The method does not need to occupy too much space when performing calibration, and the speed of projecting target points is also very fast, which saves calibration time, and thus can effectively adapt to the environment of the processing site.

The technical solution adopted by the present invention is a camera calibration method based on the idea of time-space conversion. In this method, two turntables positioned at a certain angle and a laser installed on one of the turntables form a three-dimensional turntable device. The two turntables are independent of each other. The motion controls the laser to make the laser point sweep on the wall, and then through the transformation of time and space, the relationship between the sweeping distance and the turntable rotation time is obtained, and the camera is used to shoot the laser point image at the corresponding time point to obtain a large number of The corresponding two-dimensional space feature points, and then according to the principle of the pinhole imaging model, the internal parameters and external parameters of the camera are respectively obtained. The specific steps of the method are as follows:

Step 1: Assembly of Stereoscopic Turntable Device

In the three-dimensional turntable device, the upper and lower turntables 1 and 2 are positioned and installed at a certain angle to each other, the laser 3 is fixed on one of the turntables, and the laser 3 is controlled by the independent motion of the two turntables to realize the arbitrary position of the laser 3 on the wall 6 plane. Project the laser point at the place, and use the camera 4 to shoot the image of the laser target point at the same time, and obtain an image containing the feature point information of the laser target point;

Step 2: Use the camera to take pictures of the feature points on the wall multiple times in time-sharing to obtain multiple sets of image information data.

Use the three-dimensional turntable equipment to project multiple sets of laser target points on the target wall, and shoot multiple times at the same time. In each set of images captured, the actual size of the feature points in the images captured at each time point is obtained through the space-time conversion relationship.

Formula (1) is the space-time conversion relation derived according to the accompanying drawing 2, wherein: L is the actual size of the feature point and the initial point, as can be seen from the accompanying drawing 2, the unknown quantity is composed of the rotating laser target The imaginary circle of is obtained by the integral of the velocity projection on the wall, its velocity is w r , and the projection angle is α-θ, so it is necessary to record and measure other related variables r, θ, L, α, w, t; where r is The distance from the initial point to the laser, θ is the angle between the speed of the initial point and L, α is the angle between the speed of the feature point and L, w is the rotation angular velocity of the turntable, and t is the rotation time of the turntable. Using the known parameters into the formula (1), the actual size of the feature point and the initial point in the calibration process can be solved, that is, the relationship between a specific time point and the position of the spatial feature point; according to multiple sets of projection point images taken by cameras at different times , the position information of the feature points in each group of camera calibration can be solved.

Take any feature point on the wall as the coordinate origin and the wall plane as the X O Y plane to establish a three-dimensional coordinate system in space, which is called the world coordinate system; since the actual size of each feature point on the wall is known, each feature point on the wall is in the world coordinate system The X-axis coordinates and Y-axis coordinates below are known. When the wall is approximately an ideal plane, the Z-axis coordinate is zero; when the wall is not an ideal plane, the feature points obtained by laser scanning have three-dimensional information, and the Z-axis direction information of each feature point can be determined by polar coordinates (L, β) Representation, wherein the solution of the parameters in the polar coordinates can be obtained according to the following formula according to the accompanying drawing 3:

β=90°-α-arccos((L ² +r ² -r ₁ ² )/2rL) (3)

As shown in formula (2), the point on the non-ideal plane wall is assumed to be an arbitrary feature point, and its polar coordinates are (L, β), as shown in Figure 3, first use the cosine of the measured length r and r ₁ Theorem, obtain the polar diameter L of the coordinates, and then obtain the polar coordinate polar angle β through the geometric relationship of the feature triangle; finally, according to the projection of the polar coordinates of the feature points on the XOY plane, obtain the X-axis coordinates and Y-axis of the feature points coordinates to obtain the geometric feature information of the feature points.

Step 3: Establish an optimization model to optimize calibration parameters

The camera calibration adopts the classic pinhole imaging model, and the geometric feature information obtained in step 2 is processed and brought into the following pinhole model to obtain multiple sets of equations simultaneously, and then the internal parameters of the camera and the camera coordinate system can be solved With the rotation matrix and translation vector of the world coordinate system, the relevant internal parameters of the camera and the external parameters of the calibration site are obtained. The expression of the small hole model is as follows

Wherein, (X _w , Y _w , Z _w , 1) ^T is the homogeneous coordinate of the space point in the world coordinate system, and (u, v, 1) ^T is the corresponding image pixel pixel coordinate system o ₀ uv Homogeneous coordinates, α _x =f/dx is the scale factor on the u axis in the ο ₀ uv coordinate system, α _y =f/dy is the scale factor on the v axis in the coordinate system ο ₀ uv, f is the focal length of the camera lens, d _x and d _y are the horizontal and vertical physical dimensions of the pixel respectively, (u ₀ , v ₀ ) are the principal point coordinates, ρ _c is the proportional coefficient, K is the internal parameter matrix of the camera, [R|t] is the external Parameter matrix, where R is the rotation matrix and t is the translation vector.

After that, use the internal parameters of the camera, the rotation matrix and translation vector of the camera coordinate system and the world coordinate system to solve the reprojection coordinates of all feature points on the target wall The specific formula is as follows:

Among them, r _ij is the element on the i-th row and j-th column of the rotation matrix R, the translation vector t=(t ₁ ,t ₂ ,t ₃ ) ^T , f _x is the horizontal scale factor of the camera, and f _y is the vertical scale of the camera factor, ρ ₀ is the abscissa of the main point in the pixel coordinate system, λ ₀ is the ordinate of the main point in the pixel coordinate system, (X _W , Y _W , Z _W ) is the coordinate of the feature point in the world coordinate system .

According to the known distortion coefficient, the reprojected image point coordinates obtained by the actual shooting Corrected to the corresponding ideal image point coordinates (u _n , v _n ); the establishment of an optimization model minimizes the deviation between the reprojected image point coordinates and the ideal image point coordinates through iteration, and the objective optimization function is:

The LM nonlinear optimization algorithm is used to change the Hessian matrix into a symmetric positive definite matrix for solution. When the deviation is the smallest, the corresponding parameters are the optimized computer vision system camera parameters.

The beneficial effect of the invention is that the calibration method does not need to use high-precision equipment such as auxiliary calibration plates to ensure calibration accuracy, does not need to occupy space when performing calibration, and the speed of projecting target points is also very fast, saving calibration time. Therefore, it can effectively adapt to the processing site environment, effectively improve the effect of on-site calibration, and improve the calibration efficiency.

Description of drawings

Fig. 1 is a schematic diagram of the device of the camera calibration method based on the idea of time-space conversion, in which 1-up turntable, 2-down turntable, 3-laser, 4-camera, 5-bracket, 6-wall.

Figure 2 is a schematic diagram of the inference of space-time conversion. Among them, L is the actual size of the feature point and the initial point, r is the distance from the initial point to the laser, θ is the angle between the initial point speed and L, α is the angle between the initial feature point speed and L, and w is the rotation of the turntable Angular velocity, t is the rotation time of the turntable.

Fig. 3 is a schematic diagram of the non-ideal plane wall space-time transformation inference. Among them, L is the actual size of the feature point and the initial point, r is the distance from the initial point to the laser, point P is any point on the non-ideal plane wall, r ₁ is the distance from any point to the laser, θ is the speed of the initial point and L , β is the angle between any point and the ideal wall plane, α is the angle between the velocity of the initial feature point and L, w is the rotation angular velocity of the turntable, and t is the rotation time of the turntable.

detailed description

The implementation of the present invention will be described in detail below in conjunction with the accompanying drawings and technical solutions.

Fig. 1 is the device diagram of camera calibration method, and the concrete steps of method are as follows:

Step 1: Install the stereoscopic turntable equipment

As shown in Figure 1, the three-dimensional turntable equipment is positioned at a certain angle by the upper turntable 1 and the lower turntable 2. When in use, the laser 3 is fixed on the upper turntable 1. Through the respective movements of the two turntables, the laser 3 is positioned on the plane of the wall 6. The purpose of projecting a laser point anywhere.

Step 2: Use the camera 4 to photograph the feature points on the wall 6 multiple times in time-sharing to obtain multiple sets of image information data. Perform data processing on the captured images. In each group of captured images, because the formula (1) is the time-space conversion relation derived according to the accompanying drawing 2, this formula can solve the time and space characteristics of the turntable rotation during the calibration process. The positional relationship of the points, the camera captures multiple sets of projected point images at different times, and under the condition that the above parameters are known, the position information of the feature points in each set of camera calibration can be solved.

Take any feature point on the wall 6 as the coordinate origin, and take the wall plane as the XOY plane to establish a three-dimensional coordinate system in space, which is called the world coordinate system; since the actual size of each feature point on the wall is known, each feature point on the wall 6 is in the world. The X-axis coordinates and Y-axis coordinates in the coordinate system are known. When the wall is approximately an ideal plane, the Z-axis coordinate is zero. When the wall is not an ideal plane, the X-axis coordinates and Y-axis coordinates of each feature point on the wall in the world coordinate system, as well as the polar coordinates of each feature point Z-axis can also be deduced according to the accompanying drawing 2. If the wall is not an ideal plane, the relevant parameters of the XOY plane cannot be obtained according to the above formula, and the polar coordinates (L, β) of the Z axis of each feature point need to be obtained first, and the formula (2) and (3) are used to solve the problem. According to each feature point The polar coordinates of the feature point are projected on the XOY plane to obtain the X-axis coordinates and Y-axis coordinates of the feature point, and all the feature information of the feature point is obtained.

Step 3: Establish an optimization model to optimize calibration parameters

Camera calibration adopts the classic pinhole imaging model, the model expression is as formula (4), where (X _w , Y _w , Z _w , 1) ^T is the homogeneous coordinate of the space point in the world coordinate system, (u , v, 1) ^T is the homogeneous coordinate in the pixel coordinate system o ₀ uv of the corresponding image point, obtain multiple sets of parameters (X _w , Y _w , Z _w , 1) ^T and (u, v, 1) ^T , and then by combining with the pinhole model formula (3) to form a sufficient number of equations, the internal and external parameters of the camera can be solved, including: α _x ＝f/ dx is the scale factor on the u axis in the ο ₀ uv coordinate system, α _y = f/dy is the scale factor on the v axis in the coordinate system ο ₀ uv, f is the focal length of the camera lens, and d _x and d _y are pixels respectively The horizontal and vertical physical dimensions of , (u ₀ , v ₀ ) are the principal point coordinates, ρ _c is the proportional coefficient, K is the internal parameter matrix of the camera, [R|t] is the external parameter matrix of the camera, and R is the rotation matrix , t is the translation vector.

Substituting the previously obtained internal and external parameters of the camera into formula (4) to obtain the actual coordinates of reprojection of all feature points on the target wall Reprojected image point coordinates obtained due to actual shooting There is a certain distortion, which needs to be corrected to the ideal image point coordinates (u _n , v _n ) according to the known distortion parameters; then an optimization model is established to minimize the deviation between the reprojected image point coordinates and the ideal image point coordinates through iteration , and its objective optimization function is:

The LM nonlinear optimization algorithm is used to change the Hessian matrix into a symmetric positive definite matrix for solution. When the deviation is the smallest, the corresponding internal and external parameters are the final optimized computer vision system camera parameters.

Claims (1)

1. a kind of camera marking method based on space-time conversion thought, it is characterized in that, this method by two at a certain angle The three-dimensional rotation table device of turntable and the laser constitution of a turntable installed therein being mutually located, it is mutually only by two turntables Vertical motion control laser, make laser spots be scanned on wall, then passage time and space conversion idea, obtain and scan The relational expression of distance and turntable rotational time, is shot using video camera at correspondence time point to laser dot image, obtains big Corresponding two-dimensional space characteristic point is measured, then according to national forest park in Xiaokeng principle, the intrinsic parameter and outer ginseng of video camera are drawn respectively Number, method is comprised the following steps that：

Step 1：The assembling of three-dimensional rotation table device

In three-dimensional rotation table device, laser (3) is fixed therein by the mutual location and installation at a certain angle of upper and lower turntable (1,2) On one turntable, by the separate motion control laser (3) of two turntables, realize laser (3) in wall (6) plane Any place projection laser spots, while shot using video camera (4) to laser target dot image, obtain comprising laser target spot spy Levy the image of an information；

Step 2：Characteristic point on wall is repeatedly shot using video camera (4) timesharing, multiple series of images information data is obtained；In laser spots Repeatedly shot while projection, in each group of image of shooting, Each point in time is obtained by space-time conversion relational expression and shot Characteristic point actual size in image；Solve the position relationship of calibration process intermediate station rotational time and space characteristics point, when dally Changing relational expression is：

$L=\underset{0}{\overset{t}{\∫}}wrcos(\mathrm{\α}-\mathrm{\θ})dt---\left(1\right)$

Wherein：L is characterized actual size a little with initial point, and r is distance of the initial point to laser, and θ is initial spot speed and L Angle, α is characterized spot speed and L angle, and w is the angular velocity of rotation of turntable, and t is the rotational time of turntable；

Video camera does not shoot multigroup projection dot image in the same time, under the conditions of known to above-mentioned each parameter, each group of solution is taken the photograph The positional information of characteristic point in camera calibration；

Using any characteristic point on wall (6) as the origin of coordinates, using wall (6) plane as XOY plane, space multistory coordinate is set up System, referred to as world coordinate system；Due to each characteristic point actual size on wall, it is known that so each characteristic point is in the world on wall Known to X-axis coordinate and Y-axis coordinate under coordinate system；When metope is approximately ideal plane, Z axis coordinate is zero；When metope is not During ideal plane, the characteristic point that laser is scanned has three-dimensional information, and each characteristic point Z axis is represented by polar coordinates (L, θ), wherein The solution formula of parameter in polar coordinates：

$L=\sqrt{r^2+r_1^2-2r.r_{1}cos\left(wt\right)}---\left(2\right)$

β=90 °-α-arccos ((L²+r²-r₁ ²)/2rL) (3)

According to the polar coordinates of each characteristic point, projected on XOY plane, obtain the X-axis coordinate and Y-axis coordinate of characteristic point, Obtain all characteristic informations of characteristic point；

Step 3：Set up Optimized model optimization calibrating parameters

Camera calibration is using classical national forest park in Xiaokeng, and the expression formula of the model is as follows：

Wherein, (X_w,Y_w,Z_w, 1)^TThe homogeneous coordinates for being spatial point in world coordinate system, (u, v, 1)^TFor corresponding image picture point Pixel coordinate system o₀Homogeneous coordinates in uv, α_x=f/dx is o₀Scale factor in uv coordinate systems on u axles, α_y=f/dy is seat Mark system o₀Scale factor in uv on v axles, f is camera lens focal length, d_xWith d_yRespectively horizontal, the vertical physical size of pixel, (u₀,v₀) it is main point coordinates, ρ_cFor proportionality coefficient；If K is intrinsic parameters of the camera matrix, [R | t] is the external parameter of video camera Matrix, wherein, R is spin matrix, and t is translation vector；Intrinsic parameters of the camera includes principal point coordinate (u₀,v₀), scale factor α_x、α_y, coefficient of radial distortion k₁、k₂With tangential distortion coefficient p₁、p₂；Video camera external parameter is camera coordinate system relative to generation The orientation of boundary's coordinate system, including spin matrix R and translation vector T；

Scaling board is solved using the spin matrix and translation vector of the intrinsic parameter, camera coordinate system and world coordinate system of video camera Upper all characteristic point re-projection coordinatesSpecific algorithm is as follows：

$\left\{\begin{array}{c}\hat{u}=\frac{\left(f_x{r}_{11}+u_0{r}_{31}\right)X_W+\left(f_x{r}_{12}+u_0{r}_{32}\right)Y_W+\left(f_x{r}_{13}+u_0{r}_{33}\right)Z_W+f_x{t}_1+u_0{t}_3}{r_{31}{X}_W+r_{32}{Y}_W+r_{33}{Z}_W+t_3}\\ \hat{v}=\frac{\left(f_y{r}_{21}+v_0{r}_{31}\right)X_W+\left(f_y{r}_{22}+v_0{r}_{32}\right)Y_W+\left(f_y{r}_{23}+v_0{r}_{33}\right)Z_W+f_y{t}_2+v_0{t}_3}{r_{31}{X}_W+r_{32}{Y}_W+r_{33}{Z}_W+t_3}\end{array}\right.---\left(5\right)$

Wherein, r_ijFor the element on spin matrix R the i-th row, jth row, translation vector t=(t₁,t₂,t₃)^T, f_xIt is horizontal for video camera To scale factor, f_yFor video camera vertical scaling factor, ρ₀For abscissa of the principal point under pixel coordinate system, λ₀It is principal point in picture Ordinate under plain coordinate system, (X_W,Y_W,Z_W) it is characterized the coordinate a little under world coordinate system；

According to known distortion factor, the picpointed coordinate (u ' that actual photographed is obtained_n,v′_n) it is corrected to corresponding ideal image point coordinate (u_n,v_n)；Set up deviation of the Optimized model by iteration minimization re-projection picpointed coordinate and ideal image point coordinate, objective optimization Function is：

$\min \left(\underset{n=1}{\overset{m}{\mathrm{\Σ}}}\right({\left(u_n-{\hat{u}}_n\right)}^2+{\left(v_n-{\hat{v}}_n\right)}^2\left)\right)---\left(6\right)$

Using LM nonlinear optimization algorithms, it is changed into symmetric positive definite matrix by Hessian gusts and solves, the corresponding parameter when deviation is minimum Computer vision system camera parameters after as optimizing.