CN107689065A - A kind of GPS binocular cameras demarcation and spatial point method for reconstructing - Google Patents

Abstract

The present invention proposes a GPS binocular camera calibration and spatial point reconstruction method, using a GPS receiving antenna instead of a 2D or 3D target for parameter calibration of the binocular camera, and moving the GPS receiving antenna to any position in the common field of view of the two cameras , to obtain the longitude and latitude coordinate information of the centroid of the GPS receiving antenna; and the binocular camera takes multiple images containing the GPS receiving antenna, using the projection matrix relationship between the three-dimensional coordinates of the space object and the two-dimensional coordinates in the image plane, according to the camera imaging model Based on the calibration principle of the binocular camera, the parameters of the binocular camera are calculated, and the coordinates of the spatial points are accurately reconstructed. The invention can alleviate the problem of dependence on high-precision targets in the calibration process of the binocular camera, and the reconstruction coordinates of the obtained spatial points have high precision.

Description

A GPS binocular camera calibration and spatial point reconstruction method

technical field

The invention belongs to the technical field of computer vision processing, and in particular relates to a GPS binocular camera calibration and spatial point reconstruction method.

Background technique

At present, due to the advantages of fast speed, high precision, no contact, and automation, computer vision has been widely used in the fields of 3D object recognition, 3D pose reconstruction of space objects, and robot guidance. As an important part of computer vision research, binocular camera calibration technology has a great influence on the subsequent research work. Therefore, improving the calibration accuracy of binocular cameras has become an important research topic. The calibration of the camera is to calculate the precise camera imaging model parameters by establishing the correspondence between the two-dimensional projection point coordinates of the object on the image plane and the corresponding three-dimensional space coordinates in space. However, in many occasions, the internal geometric parameters and optical parameters of the camera, as well as the positional relationship between different cameras do not need to be solved specifically, only the mapping between the coordinates of the two-dimensional projection point of the object on the image plane and its corresponding spatial coordinates needs to be established relation.

The Global Positioning System (Global Position System, GPS) uses satellites to achieve navigation and positioning of targets through timing and ranging. In practical applications, GPS receivers obtain basic positioning and navigation data by receiving standard GPS navigation messages. Since GPS has the characteristics of all-round, all-weather, real-time data, and high precision in the navigation and positioning of moving targets, it has been widely used in many fields.

Considering that compared with the traditional 2D or 3D calibration target, GPS can provide higher precision spatial coordinate system coordinates, this invention proposes a novel GPS-based binocular camera calibration and spatial point reconstruction method, using GPS instead of traditional The 2D or 3D target in the calibration method is used to accurately calibrate the binocular camera, and use the calibrated binocular camera to reconstruct high-precision three-dimensional coordinates of any point in space.

Contents of the invention

The purpose of the present invention is to propose a GPS binocular camera calibration and spatial point reconstruction method, the method uses the precise coordinates of the GPS receiving antenna centroid to replace the traditional 2D or 3D calibration target, to perform binocular camera parameter calibration and three-dimensional arbitrary point in space Reconstruction solves the problem of dependence on high-precision targets in binocular camera calibration, improves the accuracy of target calibration, and reduces the relative distance error of spatial point reconstruction.

In order to solve the above technical problems, the present invention provides a GPS binocular camera calibration and spatial point reconstruction method, using the method shown in the formula (1) to calculate the projection matrix M _j (j=1) required in the binocular camera calibration process ,2), so as to realize the calibration of binocular camera parameters,

In formula (1), is the projection matrix required in the calibration process of binocular camera parameters, Z _c1 and Z _c2 are scale factors, is the element in row a and column b of M _j (j=1,2) matrix;

In formula (1), is the homogeneous coordinate representation in the world coordinate system of the centroid of the GPS receiving antenna at different positions;

In formula (1), is the homogeneous image coordinate representation corresponding to the centroid of the GPS receiving antenna in the two sets of images captured by the binocular camera, and n is the number of images captured by each camera;

In formula (1), j represents binocular cameras 1, 2.

Use the method shown in formula (2) to calculate the coordinates of the spatial reconstruction point P (X, Y, Z), so as to realize the reconstruction of spatial point coordinates,

In formula (2), The projection matrix obtained by calibration of binocular camera parameters, Z _c1 and Z _c2 are scale factors, is the element in row a and column b of M _j (j=1,2) matrix;

In formula (2), Respectively represent the homogeneous image coordinates corresponding to the centroid of the GPS receiving antenna in the two images captured by the binocular camera;

In formula (2), Homogeneous representation of coordinates of point P(X,Y,Z) for spatial reconstruction.

The homogeneous coordinates of the world coordinate system at different positions of the centroid of the GPS receiving antenna Calculated by the following method:

Using formula (3), through the conversion between the GPS navigation coordinate system (B, L, H) and the world coordinate system, the homogeneous coordinates of the world coordinate system with the centroid of the GPS receiving antenna at different positions are solved

In formula (3), (B, L, H) is the longitude, latitude and height coordinate information in the GPS navigation coordinate system WGS-84 coordinate system, N is the radius of curvature of the ellipsoid, and E is the first eccentricity of the ellipsoid.

The homogeneous image coordinates corresponding to the centroid of the GPS receiving antenna in the captured image Calculated by Harris corner detection method.

Compared with the prior art, the present invention has the remarkable advantage that the accuracy of the traditional binocular camera calibration method often depends on the 2D or 3D target used in the calibration process, such as the two-step calibration method based on radial constraints proposed by R.Tsai . During the calibration process, the 2D or 3D targets used usually need to be precisely processed, and the targets are required to cover the entire field of view as much as possible. However, in practical applications, large and high-precision 2D or 3D targets are difficult to process and maintain, and the use of small targets for calibration will cause huge calibration errors due to the uneven distribution of accuracy in the field of view, which will affect subsequent work. Since GPS can provide more accurate spatial coordinates and can move arbitrarily, and the moving range covers the entire field of view, there is no need to rely on 2D or 3D targets during the calibration process. The present invention utilizes the projection matrix relationship between the three-dimensional coordinates of the space object and the two-dimensional coordinates in the image plane, places the GPS at any position in the field of view, and shoots multiple images containing GPS by the binocular camera. According to the camera imaging model and the binocular The principle of the target is determined, and the parameters of the camera are calculated at the same time.

Description of drawings

Fig. 1 is a flow chart of the method of the present invention.

Figure 2 is the camera imaging model.

Figure 3 is a binocular measurement model.

detailed description

One, basic thought of the present invention

The present invention proposes a GPS binocular camera calibration and spatial point reconstruction method, its basic principle is:

First, obtain the longitude, latitude and height coordinate information of its centroid in the WGS-84 coordinate system through the GPS receiving antenna, and at the same time use binocular cameras to collect images of the centroid of the GPS receiving antenna at different positions;

Then, coordinate conversion is performed according to the relationship between the WGS-84 coordinate system and the world coordinate system to obtain the coordinate information in the world coordinate system, and the coordinates of the centroid of the GPS receiving antenna in the image are obtained by using the Harris corner detection algorithm;

Then, according to the camera imaging model, the camera parameter matrix is solved to realize the calibration of the internal and external parameters of the camera;

Finally, according to the binocular measurement model composed of binocular cameras, the coordinates of any point in space are solved, and the spatial point reconstruction of binocular cameras is realized.

2. The concept of camera imaging model

The four coordinate systems included in the binocular camera calibration process are the world coordinate system O _w -X _w Y _w Z _w , the camera coordinate system O _c -X _c Y _c Z _c , the image coordinate system o _c -xy and the computer Image coordinate system o-uv.

According to the ideal camera imaging model, the transformation relationship between the camera coordinate system O _c -X _c Y _c Z _c and the image coordinate system o-uv can be obtained as:

Among them, α _x , α _y are scale factors on the u-axis and v-axis respectively, or called normalized focal lengths on the u-axis and v-axis; s is the scale factor, A is the internal reference matrix of the camera, and I is the identity matrix.

The transformation relationship between the camera coordinate system O _c -X _c Y _c Z _c and the world coordinate system O _w -X _w Y _w Z _w is:

Among them, R is a 3×3 rotation matrix; t is a 3×1 translation matrix.

From formula (1) and formula (2), the transformation relationship between the image coordinate system o-uv and the world coordinate system O _w -X _w Y _w Z _w can be obtained as:

Among them, M=A[R|t] is the camera projection matrix, A is the camera internal reference matrix, and [R|t] is the camera external reference matrix. The binocular camera calibration process is the process of solving the camera projection matrix M.

The ideal camera imaging model can be found in the literature (Wu Dan. Research on Camera Calibration Algorithms in Computer Vision [D]. Huazhong University of Science and Technology, 2014.).

3. The concept of GPS coordinate system conversion

The GPS used in the present invention is Beidou Xingtong GPS, and is in RTK working mode, takes WGS-84 coordinate system (WorldGeodetic System) as navigation coordinate system, and this coordinate system is expressed with latitude B, longitude L, altitude H, and its content is contained in In the GPS navigation message, the output format is ASCII code.

After obtaining the information of the GPS navigation coordinate system (B, L, H), it needs to be converted to the earth's Cartesian coordinate system first. Let any point P on the earth's surface be P _G (B, L, H) in the GPS navigation coordinate system, and P _E (X _E , Y _E , Z _E ) in the earth's Cartesian coordinate system. Then the relationship between the GPS navigation coordinate system and the earth's Cartesian coordinate system is:

In formula (4), N is the radius of curvature of the ellipsoid, and E is the first eccentricity of the ellipsoid. Suppose the long radius of the earth is a=6378137m, and the short radius is b=6356752m, then:

After obtaining the earth's Cartesian coordinate system, it must be transformed into the world coordinate system P(X,Y,Z). The world coordinate system takes the GPS receiving antenna main station O as the coordinate origin. Let the GPS receiving antenna master station O be P _E0 (X _E0 , Y _E0 , Z _E0 ) in the earth’s rectangular coordinate system, and the GPS receiving antenna slave station A be in the earth’s rectangular coordinate system be P _E (X _E , Y _E , Z _E ), then the coordinates of A relative to O are:

The world geodetic coordinate system of the protocol is detailed in the literature (Li Zhiyuan. Application Research of GPS in Target Tracking Coordinate Transformation [J]. Computer Programming Skills and Maintenance, 2012(19):7-8.).

4. The concept of camera parameter calibration based on GPS

The camera imaging model (3) is expressed in the following form:

In formula (7), is the projection matrix required in the calibration process of binocular camera parameters, Z _c1 and Z _c2 are scale factors, is the element in row a and column b of M _j (j=1,2) matrix; is the homogeneous coordinate representation in the world coordinate system of the centroid of the GPS receiving antenna at different positions; is the homogeneous image coordinate representation of the centroid of the GPS receiving antenna in the two sets of images captured by the binocular camera, n is the number of images captured by each camera; j represents the binocular camera 1, 2.

Expanding formula (7), we get:

Eliminate Z _cj in formula (8), (j=1,2) to get:

For n known world coordinate system coordinates and the corresponding image coordinate system coordinates Each element in the M _j (j=1,2) matrix can be solved by using the direct linear transformation (DLT) method, namely:

In formula (10) So as to get about M _j (j=1,2) matrix 2n linear equations of elements. make:

Then formula (10) can be rewritten as:

K _j m _j =U _j (12)

Using the least square method, m _j in formula (12) can be obtained, namely:

That is, the calibration of the binocular camera is completed.

The direct linear transformation (DLT) method can be found in the literature (Ge Baozhen, Li Xiaojie, Qiu Shi. Camera distortion correction based on direct linear transformation of coplanar points[J]. China Laser, 2010(2):488-494.).

The least squares method can be found in the literature (Jia Xiaoyong, Xu Chuansheng, Bai Xin. The establishment of the least squares method and its thinking method [J]. Journal of Northwest University: Natural Science Edition, 2006,36(3):507-511.) .

5. The concept of spatial point reconstruction

Assume that any point P(X,Y,Z) in the space is point p ₁ (u ₁ ,v ₁ ), p ₂ (u ₂ ,v ₂ ) on the images captured by two cameras C ₁ and C ₂ , the camera The projection matrices of C ₁ and C ₂ are M ₁ and M ₂ respectively, then:

Eliminate Z _c1 and Z _c2 in formula (14), and get four linear equations about (X, Y, Z):

To write formula (15) in matrix form, we have:

The coordinates (X, Y, Z) of the spatial reconstruction point P can be obtained by using the least square method.

Compare the calculated spatial reconstruction point P coordinates (X, Y, Z) with the actual GPS measured point P coordinates (X _P , Y _P , Z _P ), and analyze the spatial reconstruction points in the X, Y, and Z directions respectively. On the error dX, dY, dZ, there are:

From formula (17), the relative distance error err of the spatial reconstruction point can be obtained as:

Six, carry out a flow process of the inventive method

Step 1: Place the GPS receiving antenna master station O and slave station A, connect the radio station, GPS and computer, make the GPS work in RTK (Real-time kinematic, carrier phase difference technology) working mode, record the latitude and longitude height of the GPS receiving antenna master station O Information P _G0 (B ₀ ,L ₀ ,H ₀ );

Step 2: Move the GPS receiving antenna from station A to different positions, and try to make the moving range cover the entire range of the field of view, and use two cameras C ₁ and C ₂ to collect images of GPS slave station A at different positions in the field of view And record the latitude, longitude and height information corresponding to slave station A

Step 3: According to the formula (4) ~ (6), the longitude and latitude height information Convert to world coordinate system coordinates P _i (X _i ,Y _i ,Z _i )(i=1,2,...,n);

Step 4: Collected images Perform image preprocessing such as binarization and smoothing filtering;

Step 5: Use the Harris corner detection algorithm to extract the image The centroid coordinates of the GPS receiving antenna in The Harris corner detection algorithm can be found in the literature [Chen Baifan, Cai Zixing. Harris corner detection based on scale space theory [J]. Journal of Central South University: Natural Science Edition, 2005, 36(5): 751-754].

Step 6: Calculate the projection matrices M ₁ , M ₂ of the two cameras C ₁ and C ₂ according to formulas (7)-(13).

Step 7: Calculate the coordinates (X, Y, Z) of the spatial reconstruction point P according to formulas (14)~(16), and compare them with the actual GPS measurement

The coordinates (X _p , Y _p , Z _p ) are compared, and the spatial reconstruction point error is analyzed according to formulas (17)-(18).

Beneficial effect of the present invention can further illustrate by following experiment:

This experiment uses BDStar GPS, the camera is Basler acA640-90gc, the CCD size is 4.88mm×3.66mm, the resolution is 658×492, and the focal length of the lens is F=12mm.

In the experiment, according to the specific steps of the method of the present invention, the position of the master station O of the GPS receiving antenna is fixed, and its latitude, longitude and height information are recorded. Move the GPS receiving antenna slave station A within the common field of view of the _two cameras, and at the same time take its images in _C1 and C2 respectively, and record the corresponding GPS centroid coordinate information. Calculate the M ₁ and M ₂ matrices of the binocular camera according to formulas (7)-(13). The calculated M ₁ and M ₂ matrices are:

According to formulas (14) ~ (16), the coordinates of the space reconstruction point P are calculated as P = (-2.3413, -1.6226, 2.2023).

According to formulas (17)-(18), the relative distance error err=0.52% of spatial point reconstruction is obtained.

Claims (4)

1. a GPS binocular camera calibration and spatial point reconstruction method, it is characterized in that, use the required projection matrix M _j (j=1,2 in the method calculation binocular camera parameter calibration process as shown in formula (1) ), so as to realize the calibration of binocular camera parameters, In formula (1), is the projection matrix required in the calibration process of binocular camera parameters, j=1,2 means camera one and camera two in binocular camera, Z _c1 and Z _c2 are scale factors, is the element in row a and column b in matrix M _j , a=1,2,3, b=1,2,3,4; In formula (1), i=1,2,...,n is the homogeneous coordinate representation of the centroid of the GPS receiving antenna in the world coordinate system at different positions, and n is the number of images taken by each camera; In formula (1), j=1,2; i=1,2,...,n are the homogeneous image coordinates corresponding to the centroid of the GPS receiving antenna in the two sets of images captured by the binocular camera. 2. GPS binocular camera calibration and spatial point reconstruction method as claimed in claim 1, it is characterized in that, use the method as shown in formula (2) to calculate spatial reconstruction point P (X, Y, Z) coordinate, thereby realize spatial point coordinate reconstruction, In formula (2), with is the projection matrix of camera 1 and camera 2 obtained through binocular camera parameter calibration, Z _c1 and Z _c2 are scaling factors, is the element in row a and column b in matrix M _j ; In formula (2), Respectively represent the homogeneous image coordinates corresponding to the centroid of the GPS receiving antenna in the two images captured by the binocular camera; In formula (2), Homogeneous representation of coordinates of point P(X,Y,Z) for spatial reconstruction. 3. GPS binocular camera calibration and spatial point reconstruction method as claimed in claim 2, is characterized in that, the homogeneous coordinates of the world coordinate system at different positions of the GPS receiving antenna centroid Calculated by the following method: Using formula (3), through the conversion between the GPS navigation coordinate system (B, L, H) and the world coordinate system, the homogeneous coordinates of the world coordinate system with the centroid of the GPS receiving antenna at different positions are solved In formula (3), (B, L, H) is the longitude, latitude and height coordinate information in the GPS navigation coordinate system, N is the radius of curvature of the ellipsoid, and E is the first eccentricity of the ellipsoid. The homogeneous image coordinates corresponding to the centroid of the GPS receiving antenna in the captured image Calculated by Harris corner detection method. 4. GPS binocular camera calibration and spatial point reconstruction method as claimed in claim 3, is characterized in that, First, obtain the longitude, latitude and height coordinate information of its centroid in the GPS navigation coordinate system through the GPS receiving antenna, and simultaneously use binocular cameras to collect images of the centroid of the GPS receiving antenna at different positions; Then, coordinate conversion is performed according to the relationship between the GPS navigation coordinate system and the world coordinate system to obtain the coordinate information in the world coordinate system, and the Harris corner detection algorithm is used to obtain the coordinates of the centroid of the GPS receiving antenna in the image; according to the camera imaging model Solve the camera parameter matrix to realize the calibration of the internal and external parameters of the camera; Finally, according to the binocular measurement model composed of binocular cameras, the coordinates of any point in space are solved, and the spatial point reconstruction of binocular cameras is realized.