CN102034238B - Multi-camera system calibrating method based on optical imaging probe and visual graph structure - Google Patents

Abstract

A calibration method for a multi-camera system based on an optical imaging probe and a visual map structure. The optical imaging probe is used to calibrate each camera independently to obtain the initial value of the internal parameters and distortion parameters of each camera; two-by-two calibration for multiple cameras, The basic matrix, epipolar constraint, rotation matrix and translation vector between two cameras with overlapping fields of view are obtained by linear estimation method; the connection relationship between multiple cameras is established according to graph theory and visual graph structure, and the shortest path method is used to estimate each The initial value of the rotation and translation vector of each camera relative to the reference camera; the sparse bundle adjustment algorithm is used to optimize the overall internal and external parameters of all cameras and the three-dimensional marker point set of the collected optical imaging probe, and obtain high-precision calibration results. The calibration process of the present invention is simple, from local to global, from coarse to fine, realizes high-precision and robust calibration, and is suitable for the calibration of multi-camera systems with different measurement ranges and different distribution structures.

Description

Multi-camera system calibration method based on optical imaging measuring head and visual graph structure

Technical Field

The invention belongs to the technical field of vision measurement, and relates to a multi-camera system calibration method based on an optical imaging measuring head and a vision graph structure.

Background

The three-dimensional measurement is carried out by using a multi-camera system, and the calibration of the internal parameters and distortion parameters of each camera, the relative poses among the cameras and the pose parameters of a reference coordinate system must be completed firstly. The robustness and the precision of the calibration directly determine the measurement precision of the multi-camera system, so the robustness and the high-precision calibration are the premise and the condition for realizing the high-precision calibration of the multi-camera system.

Camera calibration has long been known and many effective methods have been available over decades. In recent years, due to the unique and obvious characteristics of a multi-camera system on large-scale and large-range measurement, the multi-camera system is widely paid attention to domestic and foreign research, and particularly, the high efficiency and high precision of the multi-camera system become a research hotspot. However, some of the conventional multi-camera calibration methods use electronic theodolites to establish a world coordinate system, thereby increasing the cost and complexity of the system; some adopt three-dimensional, two-dimensional, one-dimensional or 0-dimensional calibration objects, and directly utilize the pose relationship between the internal parameters of a single camera and the cameras to complete the estimation of the calibration parameters of the multi-camera system, so that on one hand, the calibration objects are required to be visible in all the camera fields, on the other hand, the calibration results are influenced by noise, calculation errors, initial parameter errors and the like, and the precision is low; although some methods also adopt a binding adjustment optimization method in the final stage of calibration, the acquisition process of the three-dimensional calibration point set is complex, and extra points need to be discarded, so that the efficiency is low and the robustness is poor.

Disclosure of Invention

The invention aims to provide a multi-camera system calibration method based on an optical imaging measuring head and a visual diagram structure, aiming at the defects or shortcomings of the existing calibration technology. The method can simultaneously complete the internal and external parameter calibration of a single camera and the calibration of a multi-camera system only by one calibrated optical imaging measuring head, is simple, does not need the optical imaging measuring head to be visible in all the camera fields, only needs two or more cameras to be visible in the field, and is suitable for various measurement fields. And the calibration of the multi-camera system is completed from coarse to fine by adopting a visual diagram structure and binding adjustment optimization, so that the robustness and high precision of the result are ensured.

In order to achieve the purpose, the invention adopts the technical scheme that:

step 1, establishing hardware:

by adopting a plurality of distributed cameras and an optical imaging measuring head, all the cameras do not require overlapped view field spaces, but require at least two cameras to have overlapped view fields:

and 2, determining the spatial position of each camera, determining one or more spatial positioning points for each camera, and positioning the optical imaging measuring head within the field of view of the calibrated camera according to the principle of spatial positioning point selection, namely in the calibration process of a single camera or multiple cameras. During calibration of a single camera, firstly, a reference positioning block (1) is installed on a corresponding positioning point, an optical imaging measuring head is placed in the field range of each camera and aligned with the reference positioning block (1), the optical imaging measuring head is rotated by taking a measuring tip (2) of the optical imaging measuring head as a circle center, seven mark points (5) on an optical imaging measuring head target body 4 concentrically rotate by taking the measuring tip (2) as the circle center, and the camera acquires a plurality of images of the optical imaging measuring head in different postures; secondly, extracting the central coordinate of each LED sub-pixel in each image by adopting a threshold segmentation and ellipse fitting algorithm, and calculating the central coordinate of each mark point 5 by adopting a gravity center method; finally, recovering the internal parameter matrix and distortion parameters of each camera according to the concentric circle rotation relation of the mark points 5 and the calibrated space distance invariant relation;

step 3, reselecting and positioning the space positioning point, adjusting the posture of the optical imaging measuring head, and ensuring that the optical imaging measuring head is positioned in the overlapped field of view of every two or more cameras; in the overlapped view field, rotating the optical imaging measuring head, simultaneously acquiring the LED images of the mark points of the optical imaging measuring head by the cameras with overlapped view fields, recovering the basic matrix, the essential matrix, the polar line geometric relationship, the rotation matrix and the translation vector between two local cameras through image processing, center recognition and mark point matching and according to the geometric constraint between the mark points;

step 4, taking each camera as a node, taking every two cameras with overlapped fields of view as edges, constructing a visual graph of the multi-camera system, wherein the direction of each edge is determined by the initially calibrated local rotation matrix and the translation vector, and the former camera points to the other camera after transformation; in the visual map, firstly determining a reference camera, starting from the reference camera, establishing a connection relation among all cameras, and calculating a rotation matrix and a translation vector of each camera relative to the reference camera by adopting a shortest path method to realize global initial calibration of a multi-camera system;

step 5, according to the initial calibration result, the mark points acquired, identified and obtained in the step 2 and the step 3 are back projected to a reference camera coordinate system, so that a three-dimensional calibration point set in a world coordinate system is established; performing optimization estimation on all calibration parameters and the three-dimensional calibration point set by adopting a sparse bundling adjustment algorithm to obtain a robust and high-precision calibration result;

step 6, if the calibration result meets the measurement precision requirement, the calibration process is finished; if the three-dimensional calibration point set does not meet the requirement, the number of the positioning points of the optical imaging measuring head is increased, the image is collected again, the identified and reconstructed new three-dimensional calibration points are added to the original marker point set, all calibration parameters and the three-dimensional calibration point set are optimized again by adopting a sparse binding adjustment algorithm, and all parameters are calibrated again.

Each marking point 5 consists of six LEDs, and the center of gravity of the marking point is the center of the marking point.

The step 2 comprises the following steps:

step 21, establishing a camera pinhole projection model:

first, a world coordinate system X is established_WY_WZ_WCoordinate system X of measuring head_tY_tZ_tAn image coordinate scale coordinate system xy and a pixel coordinate system uv, wherein N is 7 mark points on the optical imaging measuring head,

indicating the coordinates of the mark point in the measuring head coordinate system,

representing the coordinates of the mark point in the world coordinate system, (R, T) are the rotation and translation parameters of the camera relative to the world coordinate system, K is the internal parameter matrix of the camera, and comprises 5 parameters, namely the horizontal and vertical focal lengths (f)_x，f_y) Image tilt factor s and optical center coordinates (u0, v0)(k1, k2, d1 and d2) are distortion parameters in the radial direction and the tangential direction respectively, and the 7 mark points (5) on the measuring head target surface (4) satisfy the following projection relation:

<math> <mrow> <msup> <mi>&lambda;</mi> <mi>j</mi> </msup> <msup> <mover> <mi>x</mi> <mo>~</mo> </mover> <mi>j</mi> </msup> <mo>=</mo> <mi>KR</mi> <msubsup> <mi>X</mi> <mi>t</mi> <mi>j</mi> </msubsup> <mo>+</mo> <mi>KT</mi> <mo>,</mo> <mi>j</mi> <mo>&Element;</mo> <mn>7</mn> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>1</mn> <mo>)</mo> </mrow> </mrow> </math>

wherein λ is^jIs the projected depth of landmark point j;indicating the normalized coordinates of the landmark point j on the image.

Step 22, calibrating image acquisition and marking point identification: selecting a space positioning point according to a principle of space positioning point selection, fixing a reference positioning block (1), rotating an optical imaging measuring head by taking a measuring tip (2) as a circle center, acquiring I1 images with different postures by using a camera, distributing the acquired images in a field space of the camera as much as possible, extracting a sub-pixel center of each LED light spot by adopting a threshold segmentation and ellipse fitting algorithm, and estimating the center of each mark point by adopting a gravity center method;

step 23, according to the invariance of the radius of the concentric circle, establishing an error function of the distance between each mark point and the measuring tip when the optical imaging measuring head rotates around the measuring tip (2), wherein 7 mark points respectively have the radius r_jRotating concentrically around the measuring tip (2), and because the positions of all the mark points are accurately calibrated, the rotating radiuses r of 7 concentric circles_jThe distance from each mark point to the measuring tip is constant according to the constant radius of the concentric circles during the rotation process of the measuring headFor the I1 images acquired by rotation, each label can obtain the radius error equation:

<math> <mrow> <msubsup> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mrow> <mi>I</mi> <mn>1</mn> </mrow> </msubsup> <mrow> <mo>(</mo> <msubsup> <mi>r</mi> <mi>j</mi> <mi>i</mi> </msubsup> <mo>-</mo> <msub> <mi>r</mi> <mi>j</mi> </msub> <mo>)</mo> </mrow> <mo>=</mo> <mn>0</mn> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>2</mn> <mo>)</mo> </mrow> </mrow> </math>

wherein,denotes an estimated rotation radius r of the jth (j-1, …, 7) index point at the ith rotation_jA calibration value representing the turning radius of the jth (j ═ 1, …, 7) index point;

step 24, estimating initial values of internal and external parameters of the camera: solving initial values of a parameter matrix K, a rotation matrix R and a translational vector T in the camera by a radius error function of a minimum formula (3) due to the existence of errors, and estimating an initial value of a distortion parameter by a back projection error minimum

<math> <mrow> <msubsup> <mi>&Sigma;</mi> <mrow> <mi>j</mi> <mo>=</mo> <mn>1</mn> </mrow> <mn>7</mn> </msubsup> <msubsup> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mrow> <mi>I</mi> <mn>1</mn> </mrow> </msubsup> <mrow> <mo>(</mo> <msubsup> <mi>r</mi> <mi>j</mi> <mi>i</mi> </msubsup> <mo>-</mo> <msub> <mi>r</mi> <mi>j</mi> </msub> <mo>)</mo> </mrow> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mtext>3</mtext> <mo>)</mo> </mrow> </mrow> </math>

And 25, further improving the calibration precision by adopting a sparse binding adjustment algorithm.

The step 3 comprises the following steps:

step 31, acquiring the measuring head mark point images in the overlapped view field: reselecting and positioning a space positioning point (L1, L2.. once.), adjusting the posture of the optical imaging measuring head to ensure that the optical imaging measuring head is positioned in the overlapped visual field of every two or more cameras, rotating the optical imaging measuring head, and simultaneously acquiring a three-dimensional image of the measuring head marking point by the cameras with overlapped visual fields in the area;

step 32, identifying and matching the mark points of the stereo image: extracting the center of a sub-pixel of each LED light spot by adopting a threshold segmentation and ellipse fitting algorithm, and estimating the center of each mark point by adopting a gravity center method; fast matching the mark points of the stereo image pair according to the mark point position constraint and the geometric moment invariance rule;

step 33, estimating the fundamental matrix and epipolar geometry:

assuming that the two cameras C1 and C2 have overlapped view field spaces, and no loss of generality, the origin of world coordinates is positioned at the center of the camera C1, and the pose of the camera C1 is (R)₁，T₁) The pose of the camera C2 is (R, 0)₂，T₂) The relative relationship between the two cameras is (R, T). The image coordinates of the same-name mark point j acquired by the two cameras

The following relations exist between the following components:

<math> <mrow> <msubsup> <mi>&lambda;</mi> <mn>2</mn> <mi>j</mi> </msubsup> <msubsup> <mover> <mi>x</mi> <mo>~</mo> </mover> <mn>2</mn> <mi>j</mi> </msubsup> <mo>=</mo> <msubsup> <mi>&lambda;</mi> <mn>1</mn> <mi>j</mi> </msubsup> <msub> <mi>K</mi> <mn>2</mn> </msub> <msub> <mi>R</mi> <mn>2</mn> </msub> <msubsup> <mi>K</mi> <mn>1</mn> <mrow> <mo>-</mo> <mn>1</mn> </mrow> </msubsup> <mrow> <msubsup> <mover> <mi>x</mi> <mo>~</mo> </mover> <mn>1</mn> <mi>j</mi> </msubsup> <mo>+</mo> <msub> <mi>K</mi> <mn>2</mn> </msub> <msub> <mi>T</mi> <mn>2</mn> </msub> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>4</mn> <mo>)</mo> </mrow> </mrow> </mrow> </math>

estimating unknown scale parameters from the equation

Andand obtain

Wherein

I.e. the required fundamental matrix, and if there are enough pairs of landmark points, the 9 parameters of the fundamental matrix are solved linearly. Meanwhile, in step 2, the intrinsic parameter matrix K of the two cameras C1 and C2₁，K₂Has been calibrated, using coordinate values

(i ═ 1, 2) estimating the epipolar constraint relationship of the stereo images:

$x_2^{\mathrm{jT}}{\mathrm{Ex}}_1^j=0---\left(5\right)$

wherein,

the method is an essential matrix, and an essential matrix E can be estimated by using a 7-point algorithm;

step 34, estimating the relative pose and scale factor between every two cameras:

because the positions of 7 mark points on the measuring head are known, a rotation matrix R and a translation vector T between every two cameras can be uniquely determined, and when the scale factor is estimated, the normalized radius of the mark point relative to the rotation of the measuring tip is used

And actual nominal radius

The following results were obtained:

<math> <mrow> <msup> <mi>&lambda;</mi> <mi>j</mi> </msup> <mo>=</mo> <msup> <mi>r</mi> <mi>j</mi> </msup> <mo>/</mo> <msup> <mover> <mi>r</mi> <mo>~</mo> </mover> <mi>j</mi> </msup> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>6</mn> <mo>)</mo> </mrow> </mrow> </math>

taking the average value of 7 point estimates or the average value of N times of measurement as a final scale factor lambda due to the existence of errors;

<math> <mrow> <mi>&lambda;</mi> <mo>=</mo> <msubsup> <mi>&Sigma;</mi> <mrow> <mi>j</mi> <mo>=</mo> <mn>1</mn> </mrow> <mn>7</mn> </msubsup> <msup> <mi>&lambda;</mi> <mi>j</mi> </msup> <mo>/</mo> <mn>7</mn> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>7</mn> <mo>)</mo> </mrow> </mrow> </math>

<math> <mrow> <mi>&lambda;</mi> <mo>=</mo> <msubsup> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>N</mi> </msubsup> <mrow> <mo>(</mo> <msubsup> <mi>&Sigma;</mi> <mrow> <mi>j</mi> <mo>=</mo> <mn>1</mn> </mrow> <mn>7</mn> </msubsup> <msup> <mi>&lambda;</mi> <mi>j</mi> </msup> <mo>/</mo> <mn>7</mn> <mo>)</mo> </mrow> <mo>/</mo> <mi>N</mi> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>8</mn> <mo>)</mo> </mrow> <mo>.</mo> </mrow> </math>

the step 4 comprises the following steps:

step 41, constructing a visual diagram of the multi-camera system according to a diagram theory:

constructing a visual graph of the multi-camera system by taking each camera as a node and taking every two cameras with overlapped view fields as edges, wherein in the visual graph, the direction of each edge is temporarily determined by an initially calibrated local rotation matrix and a translation vector, and the other camera after transformation is pointed by the previous camera;

step 42, determining the reference cameras, and reestablishing the connection relationship between the cameras:

in the visual map, determining a reference camera, considering the relation with the adjacent camera when selecting the reference camera, and readjusting and establishing the connection relation between the cameras from the reference camera;

step 43, determining the rotation and translation amount of each camera in the reference camera coordinate system by adopting a shortest path method;

solving for the absolute position from the reference camera to each camera using the shortest path, let C_i，C_j，C_kThe cameras which are mutually connected on a certain path of the visual map respectively obtain the secondary camera C for every two calibrated cameras_iTo C_jAnd C_jTo C_kOf (A) a transformation matrix (R)_ij，T_ij) And (R)_jk，T_jk) From C_iTo C_kIs calculated by the following equation:

R_ik＝R_ijR_jk，T_ik＝T_ij+R_ijT_jk (9)

and if the number of nodes on the path of the reference camera is more than two, solving by using the above formula for multiple times to finish absolute calibration of all cameras relative to the reference camera.

The step 5 comprises the following steps:

step 51, obtaining an initial three-dimensional calibration point set

In the step 22 and the step 32, image coordinates of different calibration points of the marker point in all cameras are collected and identified, a three-dimensional sparse calibration point set is obtained by back projection according to all calibration parameters obtained in the previous 4 steps, and errors exist in the reconstructed three-dimensional sparse calibration point set and the calibration parameters due to the fact that a simple connection relation of multiple cameras is used in calibration;

step 52, constructing a back projection error function

Supposing that n three-dimensional coordinate points and m cameras are arranged, the projection coordinate of the ith point on the jth camera image is x_ijThe parameter of each camera is represented by a vector a_jRepresenting, each three-dimensional point by a vector b_iShowing that the function Q (,) defines the projection of a three-dimensional point on the image surface of the camera, the function d (x, y) represents the Euclidean distance between the image points x and y, and the following back projection error function is established according to the projective geometrical relationship:

<math> <mrow> <msub> <mi>min</mi> <mrow> <msub> <mi>a</mi> <mi>j</mi> </msub> <mo>,</mo> <msub> <mi>b</mi> <mi>i</mi> </msub> </mrow> </msub> <msubsup> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>n</mi> </msubsup> <msubsup> <mi>&Sigma;</mi> <mrow> <mi>j</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>m</mi> </msubsup> <mi>d</mi> <msup> <mrow> <mo>(</mo> <mi>Q</mi> <mrow> <mo>(</mo> <msub> <mi>a</mi> <mi>j</mi> </msub> <mo>,</mo> <msub> <mi>b</mi> <mi>i</mi> </msub> <mo>)</mo> </mrow> <mo>,</mo> <msub> <mi>x</mi> <mi>ij</mi> </msub> <mo>)</mo> </mrow> <mn>2</mn> </msup> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>10</mn> <mo>)</mo> </mrow> </mrow> </math>

and step 53, using the binding adjustment minimized back projection error function to obtain an accurate calibration result.

The back projection error function is a function containing P e □^MProblem of non-linear minimization of parameter vectors, P ∈ R^MThe parameter vector is formed by the pose parameters of all cameras and a three-dimensional measuring point set X-shaped □^NAnd (4) forming.

In order to realize the calibration of the internal and external parameters of a single camera and the relative pose of every two cameras, the invention adopts an optical imaging measuring head as a calibration object, 7 marked points with calibrated positions are arranged on the optical imaging measuring head (each marked point consists of 6 LEDs, the gravity center of the marked point is the center of the marked point), and the internal and external parameters of the cameras are recovered according to the concentric rotation motion relation of the marked points on the target surface of the measuring head around a measuring tip, thereby realizing the rapid calibration. Because the measuring space is large and the visual field of the camera is limited, the invention regards the multi-camera system as a visual graph structure relationship, each camera is a node of the visual graph, the edge of the visual graph represents the overlapped visual field range between the nodes (cameras), the edge of the visual graph has directionality, and the representing pose is changed from the previous camera to the other camera; and optimizing all pose initial parameters of the multi-camera system on a visual map by a shortest path method. Meanwhile, because the noise and the error in the coordinate transformation can influence the overall precision of the system, the invention further adopts a sparse binding adjustment algorithm to carry out optimization adjustment on the acquired three-dimensional mark points and all initial calibration parameters so as to improve the precision of the calibration parameters. By using the method, the overall error of system calibration is greatly reduced, and the robustness of each parameter is obviously improved.

Drawings

Fig. 1 is a schematic diagram of the projection relationship between the optical imaging probe and the camera.

Wherein the reference numerals denote: 1. reference positioning block, 2, measuring tip, 3, extension bar, 4, measuring head target body, 5, 7 mark points: (Each marker point consists of 6 LED light spots). X_WY_WZ_WRepresenting the world coordinate system, X_tY_tZ_tRepresenting a measuring head coordinate system, an xy image coordinate scale coordinate system and uv representing an image pixel coordinate system; P1-P7 represent 7 mark points on the optical imaging measuring head, and r 1-r 7 are distances from the 7 mark points to the measuring tip respectively.

FIG. 2 is a schematic diagram of relative position calibration of two cameras with overlapping fields of view.

Wherein the reference numbers: C1-C8 represent 8 cameras, L1 and L2 represent the positions of positioning points, and R and T represent relative pose transformation relations.

Fig. 3 is a visual diagram construction diagram of a multi-camera system.

Wherein the reference numbers: C1-C8 represent 8 cameras.

Fig. 4 is a schematic diagram illustrating a change in connection relation for determining the front and rear visual images of the reference camera.

Wherein the reference numbers: C1-C8 represent 8 cameras.

Fig. 5 is a schematic diagram of global calibration under a reference camera coordinate system.

Wherein the reference numbers: C1-C8 represent 8 cameras, L1 and L2 represent the positions of positioning points, and R and T represent relative pose transformation relations.

Detailed Description

The invention will be further explained with reference to the drawings.

The following description will specifically discuss a vision system including 8 cameras as an example. The present invention uses an optical imaging probe as shown in figure 1. The optical imaging measuring head mainly comprises a measuring tip (2), an extension bar (3), a target body (4), 7 mark points (5) and LED light spots. The distribution of 8 cameras is shown in fig. 3, where all cameras do not require overlapping fields of view, but at least every second camera has an overlapping field of view. According to the method, a space positioning point is selected and determined according to the space distribution of 8 cameras, and independent calibration of a single camera and relative pose calibration of every two cameras are completed by utilizing the rotation invariant relation of an optical imaging measuring head around a measuring tip. And then, constructing a connection relation of the multiple cameras according to the graph theory and the visual graph, and completing the calibration of the multiple cameras under a reference camera coordinate system. And finally, globally optimizing the three-dimensional geometric structure and all calibration parameters by adopting a sparse bundling adjustment algorithm in order to eliminate the initial calibration error.

The following describes the steps in a specific order of calibration.

The first stage is as follows: and (4) independently calibrating intrinsic parameters and distortion parameters of the single camera.

The numbers of the cameras are C1-C8, as shown in FIG. 2.

Step 11, establishing a world coordinate system X according to the projection relation between the optical imaging measuring head and the camera shown in figure 1_WY_WZ_WCoordinate system X of measuring head_tY_tZ_tAn image coordinate scale coordinate system xy and a pixel coordinate system uv.

And step 12, determining the spatial position of each camera, determining one or more spatial positioning points for each camera, and positioning the optical imaging measuring head within the field of view of the calibrated camera according to the principle of spatial positioning point selection, namely in the calibration process of a single camera or multiple cameras. In the calibration of a single camera, a reference positioning block (1) is fixed on a corresponding positioning point, an optical imaging measuring head rotates by taking a measuring tip (2) as a circle center, the camera is used for collecting I1 images of different postures, and the collected images are distributed in the view field space of the camera as much as possible. Extracting the center of a sub-pixel of each LED light spot by adopting a threshold segmentation and ellipse fitting algorithm, and estimating the center coordinate of each mark point (consisting of 6 LED light spots) by adopting a gravity center method(I-1, …, I1; j-1, …, 7). Meanwhile, an optical imaging probe is arranged onThe coordinate of N-7 mark points in the measuring head coordinate system is

The coordinates in the world coordinate system areAnd (3) establishing a projection relation of the camera according to the formula (1).

And step 13, constructing a radius error equation (2) of each mark point according to the concentric circle radius invariant geometric constraint.

And step 14, a minimized radius error function formula (3) restores the initial values of the intrinsic parameter matrix K, the rotation matrix R and the translational vector T of each camera, and estimates distortion parameters of each camera through the minimum back projection error (K1, K2, d1 and d 2).

And step 15, optimizing internal and external parameters of each camera by adopting a sparse binding adjustment algorithm, and improving the calibration precision.

And a second stage: and calibrating the relative pose of every two cameras with overlapped view fields.

A pair of 8 cameras is arbitrarily selected and grouped in the order of the small number preceding and the large number following.

Step 21, as shown in fig. 2, selecting and positioning an appropriate spatial positioning point (L1, L2,. cndot.), adjusting the posture of the optical imaging probe to ensure that the optical imaging probe is positioned in the overlapped fields of view of every two or more cameras, rotating the optical imaging probe, and simultaneously acquiring a three-dimensional image of a probe mark point by the cameras with overlapped fields of view in the area.

And step 22, extracting the center of the sub-pixel of each LED light spot by adopting a threshold segmentation and ellipse fitting algorithm, and estimating the center of each mark point by adopting a gravity center method. And quickly matching the mark points of the stereo image pair according to the mark point position constraint and the geometric moment invariance rule.

Step 23, estimating the unknown scale parameters of each pair of cameras according to the formula (4)

And

according to the relationship

Recovering the basic matrix of each two pairs of cameras; and restoring the epipolar constraint relation of each pair of cameras according to the formula (5).

And 24, recovering the rotation matrix R and the translational vector T of each pair of cameras, and calculating a scale factor lambda according to a formula (7) and a formula (8).

And a third stage: and global calibration of the multi-camera system based on the visual map.

Step 31, as shown in fig. 3, a visual diagram of the multi-camera system is constructed by using 8 cameras as nodes and every two cameras with overlapped fields of view as edges. In the visual map, the direction of each edge is temporarily determined by the initially calibrated local rotation matrix and the translation vector, and a directional visual map as shown in fig. 4(a) is constructed.

And step 32, in the visual map, using C8 as a reference camera, readjusting and establishing a connection relationship between the cameras, and determining a new directional visual map by using a shortest path method, as shown in fig. 4 (b).

Step 33, determining the rotation and translation amount of each camera in the reference camera coordinate system according to formula (9), as shown in fig. 5. Thus, the absolute calibration of all cameras relative to the reference camera is initially completed.

A fourth stage: and carrying out global optimization calibration based on sparse binding adjustment.

Step 41, reconstructing a three-dimensional sparse calibration point set X epsilon □ through back projection according to the initial calibration parameters^N。

And 42, establishing a back projection error minimum function (10) by all the calibration data and the three-dimensional point set.

And 43, using the binding adjustment minimized back projection error function to obtain an accurate calibration result.

The fifth stage: and further optimizing and estimating the calibration parameters until the three-dimensional measurement calibration precision requirement is met.

The invention is characterized in that:

the calibration method uses the imaging measuring head with fixed optical mark points to calibrate the internal and external parameters of the camera and the relative pose between the cameras, does not need any other equipment, and has simple method and low cost.

The calibration method does not need the optical imaging measuring head to be visible in all the camera view fields, only needs to be visible in the overlapped view fields of every two cameras, is suitable for multi-camera systems with different distribution structures, and can be used for small-scale space calibration and large-scale space calibration.

The visual map and the binding adjustment optimization algorithm adopted by the calibration method can obviously improve the calibration efficiency, robustness and precision.

Claims (4)

1. A multi-camera system calibration method based on optical imaging measuring head and visual graph structure, it is characterized in that comprising the following steps: Step 1, the establishment of hardware: Using distributed multiple cameras and a calibrated optical imaging probe, all cameras do not require an overlapping field of view space, but at least every two cameras are required to have an overlapping field of view; Step 2. Determine the spatial position of each camera, and determine one or more spatial positioning points for each camera. According to the principle of spatial positioning point selection, the optical imaging probe can be located in the calibration process of a single camera or a multi-camera. Within the calibrated camera field of view, in the calibration of a single camera, first install the reference positioning block (1) on its corresponding positioning point, put the optical imaging probe into the field of view of each camera, and align with the reference positioning block ( 1) Align, rotate the optical imaging probe with the tip (2) of the optical imaging probe as the center of the circle, and the seven mark points (5) on the target body (4) of the optical imaging probe take the tip (2) as the center of the circle Concentric rotation, the camera collects multiple images of different attitudes of the optical imaging probe, in which each marker point (5) is composed of six LEDs, and its center of gravity is the center of the marker point; secondly, threshold segmentation and ellipse fitting algorithms are used to extract each image For the center coordinates of each LED sub-pixel in the image, the center coordinates of each marker point (5) are calculated using the center of gravity method; finally, according to the concentric circle rotation relationship of the marker point (5) and the calibrated spatial distance invariant relationship, Recover the intrinsic parameter matrix and distortion parameters of each camera; Step 3. Reselect and locate the spatial positioning point, adjust the attitude of the optical imaging probe to ensure that the optical imaging probe is located in the overlapping field of view of every two or more cameras; The cameras with overlapping fields simultaneously acquire the LED images of the marker points of the optical imaging probe, through image processing, center recognition and marker point matching, and according to the geometric constraints between the marker points, the local basic matrix, essential matrix, and epipolar line between the two cameras are restored Geometric relationship and rotation matrix, translation vector; Step 4. Taking each camera as a node and every two cameras with overlapping fields of view as sides, construct a visual map of the multi-camera system. The direction of each side is determined by the initial calibration local rotation matrix and translation vector, and Point from the previous camera to the transformed camera; in the visual map, first determine the reference camera, start from the reference camera, establish the connection relationship between all cameras, and use the shortest path method to calculate the rotation matrix of each camera relative to the reference camera and translation vectors to realize the global initial calibration of the multi-camera system; Step 5. According to the global initial calibration result of the multi-camera system, back-project the marker points collected, identified and obtained in Step 2 and Step 3 to the reference camera coordinate system, so as to establish a three-dimensional calibration point set in the world coordinate system; use sparse The bundled adjustment algorithm optimizes and estimates all calibration parameters and 3D calibration point sets to obtain robust and high-precision calibration results; Step 6. If the calibration result meets the measurement accuracy requirements, the calibration process ends; if not, increase the number of positioning points of the optical imaging probe, re-acquire images, and add the new 3D calibration points identified and reconstructed to the original mark points Concentration, using the sparse bundle adjustment algorithm to re-optimize all calibration parameters and three-dimensional calibration point sets, and re-calibrate each parameter. 2. the multi-camera system calibration method based on optical imaging measuring head and visual graph structure as claimed in claim 1, is characterized in that: described step 2 comprises the following steps: Step 21. Establish a camera pinhole projection model: First, the world coordinate system X _W Y _W Z _W , the probe coordinate system X _t Y _t Z _t , the image coordinate scale coordinate system xy and the pixel coordinate system uv are established, and there are N=7 mark points on the optical imaging probe,

Indicates the coordinates of the marker point in the probe coordinate system,

Indicates the coordinates of the marker point in the world coordinate system, (R, T) is the rotation and translation parameters of the camera relative to the world coordinate system, K is the internal parameter matrix of the camera, including 5 parameters, which are the horizontal and vertical focal lengths (f _x , f _y ), image tilt factor s and optical center coordinates (u0, v0), (k1, k2, d1, d2) are the radial and tangential distortion parameters respectively, and the seven on the probe target surface (4) The marker point (5) satisfies the following projection relationship: ${\mathrm{<span\; class="notranslate">\; \&lambda;</span>}}^{\mathrm{<span\; class="notranslate">\; j</span>}}{\stackrel{<span\; class="notranslate">\; ~</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}}^{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; KRX</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}^{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; KT</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1,2</span>}<span\; class="notranslate">\; ,</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 7</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$ Among them, λ ^j is the projection depth of the marker point j;

Indicates the normalized coordinates of the marker point j on the image; Step 22. Calibrate image acquisition and mark point recognition: According to the principle of spatial positioning point selection, select the spatial positioning point, and fix the reference positioning block (1), rotate the optical imaging probe with the tip (2) as the center of the circle, and use the camera Collect images I1 of different postures, and the collected images spread throughout the camera's field of view space as much as possible, use threshold segmentation and ellipse fitting algorithm to extract the sub-pixel center of each LED light point, and use the center of gravity method to estimate the center of each marker point; Step 23. According to the constant radius of the concentric circles, establish the error function of the distance between each marker point and the measuring tip when it rotates. When the optical imaging probe rotates around the measuring tip (2), the 7 marker points are respectively measured around the radius r _j The tip (2) rotates concentrically. Since the positions of all the marking points have been precisely calibrated, the turning radius r _j of the seven concentric circles remains unchanged. According to the constant radius of the concentric circles, the distance from each marking point to the measuring tip during the rotation of the probe Keeping the distance constant, for the I1 images collected by rotation, the radius error equation can be obtained for each mark: ${\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; I</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$ in,

Represents the jth (j=1,...,7) marker point estimated rotation radius when it rotates for the first time, r _j represents the calibration value of the jth (j=1,...,7) marker point's radius of rotation; Step 24. Estimating the initial value of the internal and external parameters of the camera: due to the existence of errors, the initial values of the internal parameter matrix K of the camera, the rotation matrix R and the translation vector T are solved by minimizing the radius error function of formula (3), and the error is minimized by back-projection Estimating the initial value of the distortion parameter ${\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; 7</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; I</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$ Step 25, using the sparse bundle adjustment algorithm to further improve the calibration accuracy. 3. the multi-camera system calibration method based on optical imaging measuring head and visual graph structure as claimed in claim 1, is characterized in that: described step 4 comprises the following steps: Step 41. According to the graph theory, construct the visual graph of the multi-camera system: With each camera as a node and every two cameras with overlapping fields of view as edges, a visual graph of a multi-camera system is constructed. In the visual graph, the direction of each edge is temporarily determined by the initially calibrated local rotation matrix and translation vector Confirm, and point the previous camera to the transformed camera; Step 42, determine the reference camera, and re-establish the connection relationship between the cameras: In the visual map, determine a reference camera, consider the relationship with adjacent cameras when selecting the reference camera, start from the reference camera, readjust and establish the connection relationship between the cameras; Step 43, using the shortest path method to determine the rotation and translation of each camera in the reference camera coordinate system; Use the shortest path to solve the absolute position from the reference camera to each camera. Let C _i , C _j , and C _k be the interconnected cameras on a certain path of the visual map _. Transformation matrices (R _ij , T _ij ) and (R _jk , T _jk ) to C _j and C _j to C _k , the relationship from C _i to C _k is calculated by the following formula: R _ik =R _ij R _jk , T _ik =T _ij +R _ij T _jk (9) If there are more than two nodes on the path from the reference camera, the above formula is used to solve for multiple times to complete the absolute calibration of all cameras relative to the reference camera. 4. the multi-camera system calibration method based on optical imaging measuring head and visual graph structure as claimed in claim 2, is characterized in that: described step 5 comprises the following steps: Step 51. Obtain an initial 3D calibration point set From step 22, collect and identify the image coordinates of different calibration points of the marker points in all cameras. According to all the calibration parameters obtained in the first 4 steps, a three-dimensional sparse calibration point set is obtained by back projection. Since the calibration uses a simple connection of multiple cameras relationship, there are errors in the reconstructed 3D sparse calibration point set and calibration parameters; Step 52. Construct the backprojection error function Suppose there are n three-dimensional coordinate points and m cameras, the projection coordinates of the i-th point on the j-th camera image are x _ij , the parameters of each camera are represented by vector a _j , and each three-dimensional point can be represented by vector b _i , the function Q(,) defines the projection of a three-dimensional point on the camera image plane, and the function d(x, y) represents the Euclidean distance between the image point x and y. According to the projective geometric relationship, the following back-projection error function is established: ${\mathrm{<span\; class="notranslate">\; min</span>}}_{{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; m</span>}}\mathrm{<span\; class="notranslate">\; d</span>}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; Q</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; )</span>$ Step 53: Minimize the back-projection error function using bundle adjustment to obtain accurate calibration results.

Images