CN103985121B - Method for calibrating structural light of underwater projector - Google Patents

Abstract

The invention belongs to the technical field of measurement and provides a method for calibrating a structural light of an underwater projector. The method comprises the following steps: shooting the pair of images of the waterproof target and the projection target which are positioned on the same plane at least three positions; under a camera coordinate system, calculating a plane normal vector and a projection target three-dimensional coordinate, preliminarily calibrating a projector and structural parameters by adopting a single viewpoint model, and converting the projection target three-dimensional coordinate into a projector coordinate system as a reference value; establishing an incident light and refracted light model of the projection target under a projector coordinate system, and calculating the three-dimensional coordinate of the projection target; and establishing an objective function according to the calculated distance between the three-dimensional coordinates of the projection target and the reference value, and performing recursive optimization on all parameters. The invention considers the refraction phenomenon when the light of the projector is transmitted from the air medium to the water medium and the condition that the optical axis of the projector is not parallel to the normal vector of the window of the waterproof device, and can meet the calibration requirement of the underwater projector.

Description

A calibration method for structure cursor of underwater projector

technical field

The invention relates to a structural cursor calibration method for an underwater projector, belonging to the technical field of measurement.

Background technique

The structured light mode combined with projector and camera is an important three-dimensional measurement method and technology in visual measurement. The principle is to use a projector structured light sensor. The projector actively projects features according to the measurement needs, such as dot matrix, grating stripes, coding grid, etc. The image captured by the camera contains projection features that are deformed due to changes in the surface topography of the object. According to the principle of structured light triangulation, the three-dimensional information of the object surface is recovered. Since an image captured by a camera can contain a large amount of feature information, for short-distance underwater 3D measurement applications, projector structured light technology is an effective means to restore the 3D information of the measured object.

The underwater projector structured light measurement environment is different from the air medium environment on land. The projector for the underwater environment needs to be fixed in a sealed waterproof device, and project features to the measured object in the water through the transparent window in front of the projector lens. Due to the use of the waterproof device, the measured object and the image plane of the projector are located in different media, that is, the measured object is located in the water medium, while the projector image plane is located in the air medium in the waterproof device, and the light from the optical center of the projector When entering the water medium through the window plane, refraction occurs. At this time, the single-view model based on the propagation of light along a straight line is no longer applicable.

The projection process of the projector is opposite to the imaging process of the camera, which can be regarded as a reverse camera. Due to the bending of light due to refraction, the position of the optical center of the projector described by the single-viewpoint model is inconsistent with the actual position of the optical center. Treibitz et al pointed out in the article "Flat Refractive Geometry.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012,34(1):51-65" that the distortion caused by refraction depends on the distance between the measured object and the camera. If the viewpoint model or radial distortion model is used to represent this distortion, a large error will occur, so the single-viewpoint model cannot accurately describe the projection process of the underwater projector. However, this paper only considers the case when the optical axis is parallel to the normal vector of the window plane. When the optical axis is not parallel to the normal vector of the window plane, the model based on the analysis in this paper will still have deviations.

From the above analysis, it can be seen that due to the refraction effect of the waterproof device window on the light and the non-parallel effect of the optical axis of the projector and the normal vector of the window plane, the traditional single-viewpoint model of the projector is no longer applicable. Therefore, it is of great significance to study a model and calibration method suitable for the underwater projector structured light vision measurement sensor.

Contents of the invention

The technical problem to be solved by the present invention is to provide an optimal calibration method for projector parameters and structural parameters of an underwater projector structured light sensor.

The technical solution of the present invention is: a kind of underwater projector calibration method, it is characterized in that this method comprises the following steps:

1.1. Adjust the structured light sensor of the projector located underwater to ensure that the camera can capture clear images within the measurement range and the projector can project clear images within the measurement range. Within the common field of view of the projector and the calibrated camera, use A flat panel with a waterproof target, and the projector projects another target on the flat panel, which is called a projected target. Adjust the position of the flat panel until the two target feature points are within the common field of view; move the target freely at least three positions, Every time a position is moved, an image pair containing two targets is taken;

1.2. Extract the image coordinates of the waterproof target feature points in the image, and obtain the normal vector of the plane where the plate is located in the camera coordinate system according to the parameters of the calibrated camera;

1.3. Extract the image coordinates of the projected target feature points in the image, and use the normal vector obtained in step 1.2 to calculate the corresponding space point of the projected target projected on the flat panel by the camera parameters and the image coordinates of the feature points under the camera coordinate system three-dimensional coordinates;

1.4. According to the image coordinates of the projection target and the three-dimensional coordinates of its corresponding space point obtained in step 1.3, use the single-viewpoint model to initially calibrate the projector parameters and the structural parameters between the projector and the camera;

1.5. Using the structural parameters obtained in step 1.4, the normal vector obtained in step 1.2 is expressed in the projector coordinate system;

1.6. In the coordinate system of the projector, given the initial value of the distance from the optical center of the projector to the window plane of the waterproof device along the optical axis, establish the time when the light emitted by each feature point on the projector image propagates in the air medium in the waterproof device The ray model, called the incident ray;

1.7. Establish the unit vector of the refracted ray, solve the scale factor according to the geometric constraints, determine the length of the refracted ray, and solve the three-dimensional coordinates of the projected target feature points in the coordinate system of the projector;

1.8. Convert the three-dimensional coordinates calculated in step 1.3 to the projector coordinate system using the structural parameters in step 1.4 as reference values;

1.9. Given the initial value of the deflection angle and rotation angle between the optical axis of the projector and the normal vector of the window plane, the objective function is established with the distance between the three-dimensional coordinates described in step 1.7 and the reference value described in 1.8. The projector parameters and structural parameters are optimized and recursively optimized, that is, the obtained structural parameters are used in steps 1.5 to 1.8 until the objective function value meets the set threshold, which is set according to the measurement accuracy requirements, generally 10 ^-5 to 0.1mm.

2, a kind of underwater projector calibration method described, it is characterized in that:

2.1. The waterproof target described in step 1.1 is a planar target made of waterproof material, which is fixed on a flat plate. The thickness of the waterproof target is negligible compared with the measurement distance. The target feature points are grid points on the plane. The design of the distance is determined by the actual measurement distance and the field of view of the camera, etc., generally 10-100mm; the image coordinate distance between the grid points of the projection target is determined according to the resolution of the projector image, generally 10-100 pixels ;

2.2. For the waterproof target and projection target described in step 1.1, on the premise that the camera uses a reasonable filter device such as a filter, the areas occupied by the two targets on the plate can overlap; when the plate is at the same position, the camera does not use a filter When the optical device is used and the projector does not project the target, the camera will only get the image of the waterproof target. When the camera uses a filter device and the projector projects the target, the target and the waterproof target have an overlapping area, and the camera will only get the image of the projected target;

2.3. The photographing of the image pair containing two targets described in step 1.1 is characterized in that when using a filter device, two images are photographed at each position, which are image pairs containing the waterproof target and the projection target respectively.

The advantages of the present invention are:

First, considering the refraction phenomenon when the projector works underwater from the air medium in the waterproof device to the water medium outside the waterproof device, a light refraction model of the projector is established;

Second, consider the situation when the optical axis of the projector is not parallel to the normal vector of the window plane of the waterproof device, describe this phenomenon with the deflection angle and rotation angle of the optical axis relative to the normal vector, and obtain the values of the two angles through an optimization method ;

Third, reasonable use of optical filters, the use of overlapping waterproof targets and projection targets, so that the two targets can occupy the field of view as much as possible at the same time, improving the calibration accuracy.

Description of drawings

Fig. 1 is a schematic diagram of an underwater projector light refraction model established by the present invention.

Fig. 2 is a schematic diagram showing that the optical axis of the underwater projector is not parallel to the normal vector of the window plane.

Fig. 3 is a flow chart of the underwater projector calibration method.

Fig. 4 is a schematic diagram of the target position used for calibrating the underwater projector.

detailed description

The present invention will be described in further detail below. The invention establishes the light refraction model of the underwater projector, optimizes and calibrates the projector parameters and structural parameters when there is a rotation angle between the optical axis and the normal vector of the window plane of the waterproof device, and realizes high-precision underwater projector calibration.

Figure 1 shows a schematic diagram of the light refraction model of an underwater projector. The projector coordinate system O _p -X _p Y _p Z _p is established with the optical center of the projector as the origin, the original image coordinate system is o _u -uv, and (u ₀ , v ₀ ) is the coordinate of the main point. The image coordinate system o-xy is established with the principal point position on the projector image as the origin, and x=uu ₀ , y=vv ₀ . Define ox||O _p X _p , oy||O _p Y _p . x=[x,y] ^T represents the ideal image coordinates of a point on the image plane of the projector, and x _d =[x _d ,y _d ] ^T represents the image coordinates actually affected by lens distortion. The light projected from the point x _d = [x _d , y _d ] ^T on the image plane of the projector is refracted when it passes through the window plane of the waterproof device. Record the incident light as R _in and the refracted light as R _out . Use n=[w ₁ -w ₂ 0] ^T to represent the unit normal vector of the window plane in the projector coordinate system, where w ₁ and w ₂ respectively represent the deflection angles between the optical axis of the projector and the normal vector of the window plane and corners, as shown in Figure 2, and satisfy Indicates the angle between the optical axis and the normal vector. The normal vector of the window plane can also be expressed as:

In the formula, d represents the distance from the optical center of the projector to the window plane along the optical axis.

Assuming that the distortion center coincides with the principal point, considering the second-order radial and tangential distortion, the following distortion model is available:

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}\end{array}\right]<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; 4</span>}}<span\; class="notranslate">\; )</span>\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; x</span>}\\ \mathrm{<span\; class="notranslate">\; the\; y</span>}\end{array}\right]<span\; class="notranslate">\; +</span>\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}\end{array}\right]<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; \&lsqb;</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; \&rsqb;</span>$

In the formula, k ₁ and k ₂ are radial distortion coefficients, p ₁ and p ₂ are tangential distortion coefficients, and there are:

$\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}}{{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}}{{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; \&lsqb;</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; \&rsqb;</span>$

where f _x and f _y are the x and y components of the focal length of the projector lens.

As shown in Figure 1, the incident light R _in is emitted from the optical center of the projector to the point (x _d , y _d ) on the image plane, and after refraction, the light continues to propagate along the R _out direction until it intersects the surface of the measured object at point P (X _p , Y _p , Z _p ). Use γ to represent the angle of incidence, γ' to represent the angle of refraction, according to the law of refraction:

$\left\{\begin{array}{c}\mathrm{<span\; class="notranslate">\; c</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; the\; s</span>}\mathrm{<span\; class="notranslate">\; \&gamma;</span>}<span\; class="notranslate">\; =</span>{\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; R</span>}}}_{\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; \&CenterDot;</span>\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; no</span>}}\\ {\mathrm{<span\; class="notranslate">\; sin\&gamma;</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; sin</span>}\mathrm{<span\; class="notranslate">\; \&gamma;</span>}}{{\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; e</span>}\mathrm{<span\; class="notranslate">\; r</span>}}}\end{array}\right.<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; \&lsqb;</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; \&rsqb;</span>$

In the formula a unit vector representing the direction of the incident ray, A unit vector representing the window plane normal vector. The incident ray R _in can be expressed as:

${\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; k</span>}{\left[\begin{array}{ccc}\frac{{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}}{{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}}& \frac{{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}}{{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]}^{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; \&lsqb;</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; \&rsqb;</span>$

where k is a scaling factor. The incident ray R _in and the window plane normal vector n satisfy:

R _in n = 1 [6]

Substituting equations [1] and [5] into [6] can solve the scaling factor k. Since the thickness of the flashing window is negligible compared to d, the unit vector of the refracted ray is It can be expressed as:

${\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; R</span>}}}_{\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; u</span>}\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}{\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; R</span>}}}_{\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; \&lsqb;</span>\mathrm{<span\; class="notranslate">\; 7</span>}<span\; class="notranslate">\; \&rsqb;</span>$

where m1 and _m2 are _:

$\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; cos\&gamma;</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; c</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; the\; s</span>}\mathrm{<span\; class="notranslate">\; \&gamma;</span>}\\ {\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; r</span>}\mathrm{<span\; class="notranslate">\; f</span>}\mathrm{<span\; class="notranslate">\; no</span>}}\end{array}\right.<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; \&lsqb;</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; \&rsqb;</span>$

where rfn is the refractive index of the aqueous medium.

The refracted light continues to propagate and intersects the surface of the measured object at point P. In the coordinate system of the projector, the coordinates of point P can be expressed as:

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}\\ {\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}\\ {\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}\end{array}\right]<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&beta;</span>}{\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; R</span>}}}_{\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; u</span>}\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; \&lsqb;</span>\mathrm{<span\; class="notranslate">\; 9</span>}<span\; class="notranslate">\; \&rsqb;</span>$

The above formula expresses the process of light propagation, where β is the scale factor, which determines the length of the refracted ray, and the value of β is determined by the refracted ray of the feature point and the camera ray determined by the image point on the camera image.

According to the above model, the calibration of the underwater projector includes the parameters of the projector (focal length, principal point coordinates, distortion coefficient, the distance from the optical center of the projector to the window plane along the optical axis, and the distance between the optical axis of the projector and the normal vector of the window plane. The declination and rotation angle) and the calibration of the structural parameters (rotation and translation relationship) between the projector and the camera, the specific steps are as follows:

1. Use the camera that has been calibrated to form a structured light sensor with a projector, that is, the focal length of the camera, the principal point coordinates and the distortion coefficient are known. For the camera calibration method, see Zhang’s article "A Flexible New Technique for Camera Calibration.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11): 1330-1334".

2. As shown in Figure 4, make a red and white waterproof checkerboard target (waterproof target) and paste it on a flat plate. The feature points on the target, namely the corner spacing, are known, and the setting of the corner spacing is determined by the field of view of the camera. It is determined together with the working distance, generally 10-100mm; the projector projects a red and black checkerboard target (projection target), and the coordinates of the corner points on the target in the projector image coordinate system are known, denoted as x _p = (x _pi , y _pi ), i=1 . . . , m, the setting of the corner point spacing is determined by the resolution of the projector image, generally 10-100 pixels. The specific image acquisition steps are:

1) Place the waterproof target within the common field of view of the camera and projector, and try to fill the field of view of the camera. Turn on the projector to project a red and black checkerboard target, and adjust the position of the tablet until both the projection target and the waterproof target are within the public field of view;

2) Put the red filter paper in front of the camera. Since the red filter paper can absorb the red in the waterproof target, the waterproof target will appear as a uniform background color in the camera with the red filter paper. Use a camera to collect an image, the image only contains the imaging of the projected target, denoted as p _i , (i=1,2,3,...g);

3) Turn off the projector, remove the red filter paper, and collect the image of the red and white checkerboard waterproof target, which only contains the image of the waterproof target, denoted as c _i , (i=1,2,3,...g );

4) Repeat steps 1) to 3) until enough images are collected, and at least 3 pairs of images need to be collected.

The use of red and white checkerboard waterproof targets and red and black checkerboard projection targets is one of the examples of rational use of optical filter devices in the present invention, and any calibration method that uses optical filter devices in a similar manner is within the scope of protection.

3. Extract the image coordinates of the corner points in the image c _i , and calculate the normal vector of the plane where the plate in the image c _i is located in the camera coordinate system according to the calibrated camera For the calculation method, see "Multiple view geometry in computer vision, Cambridge University Press, 2004" by Hartley RI, Zisserman.

4. Extract the image coordinates of the corner points in the image p _i , determine a ray from the camera parameters and the image coordinates of the corner points, and use the ray and the plane normal vector obtained in step 3 to obtain the projection projected on the plate The three-dimensional coordinates of the target in the camera coordinate system

5. According to the projector image coordinate x _p of the corner point of the projected target, and the three-dimensional coordinates of the corresponding space point obtained in step 4 Use the single-view point model to preliminarily calibrate the parameters of the projector and the structural parameters between the projector and the camera. For specific methods, please refer to Zhou Fuqiang's "Research on Key Technologies of Binocular Stereo Vision Detection", Postdoctoral Research Work Report of Beihang University, 2002 ".

6. Use the structural parameters obtained in step 5 to represent the normal vector of the plane where the plate is located in the coordinate system of the projector

${\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; R</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; \&CenterDot;</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; c</span>}}}{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; R</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; T</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; \&CenterDot;</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; c</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; \&lsqb;</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; \&rsqb;</span>$

In the formula, the structural parameters R and T are the rotation matrix and translation vector from the camera coordinate system to the projector coordinate system, respectively.

7. Given the initial value of the distance d from the optical center of the projector to the window plane along the optical axis direction, for example, when the projector lens is very close to the window plane, d=0 can be set; the method of setting the projector optical axis relative to the window plane The initial values of the deflection angle w ₁ and the rotation angle w ₂ between the direction vectors are both 0. Using the image coordinates of the corner points in the image _pi , the light model of the light emitted by each corner point when propagating through the air medium in the waterproof device is established by formula [1] ~ formula [6], which is called the incident light R _in .

8. According to each incident ray R _in established in step 7, the unit vector of the corresponding refracted ray is established by formula [7] and formula [8] After determining the ray trajectory corresponding to each corner point, it is necessary to determine the scale factor β in formula [9] to determine where the refracted ray intersects the plate.

9. According to geometric constraints:

${\left[\begin{array}{ccc}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}& {\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}& {\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}\end{array}\right]}^{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; \&lsqb;</span>\mathrm{<span\; class="notranslate">\; 11</span>}<span\; class="notranslate">\; \&rsqb;</span>$

Substituting Equation [9] into Equation [11], the scaling factor β can be obtained from the following equation.

$\mathrm{<span\; class="notranslate">\; \&beta;</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span>{\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; R</span>}}}_{\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; no</span>}}}{{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span>{\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; R</span>}}}_{\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; u</span>}\mathrm{<span\; class="notranslate">\; t</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; \&lsqb;</span>\mathrm{<span\; class="notranslate">\; 12</span>}<span\; class="notranslate">\; \&rsqb;</span>$

10. Substituting the scale factor β obtained in step 9 into the formula [9], the three-dimensional coordinate X _p of the corresponding spatial point projected on the flat plate by the projected target in the coordinate system of the projector can be obtained.

11. The three-dimensional coordinates calculated in step 4 Using the structural parameters in step 6 to convert to the coordinate system of the projector is

12. will As a reference value, use the distance between it and the three-dimensional coordinate _Xp obtained in step 10 to establish the objective function:

$\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; g</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; ||</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; -</span>{\stackrel{<span\; class="notranslate">\; ~</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}}_{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ||</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; \&lsqb;</span>\mathrm{<span\; class="notranslate">\; 13</span>}<span\; class="notranslate">\; \&rsqb;</span>$

The projector parameters and structural parameters calibrated in step 5 are used as the initial value of the optimization search, the initial value of the distortion coefficient is set to 0, and the initial value of other parameters is referred to in step 7. Use the Levenberg-Marquardt algorithm to nonlinearly optimize the formula [13], apply the optimized structural parameters R and T to the coordinate transformation in step 6 and step 11, and recursively search for the optimal value of the parameters until the objective function value is less than the set value A certain distance error threshold, the threshold is set according to the measurement accuracy requirements, generally 1E-5 to 0.1mm.

Example

Using a 3500 lumens projector, the resolution is 1024 pixels x 768 pixels. Install the projector into a waterproof device and fix it, and form a structured light sensor with a fiber optic camera with a resolution of 1024 pixels × 768 pixels. The relative positions of the projector and the camera are fixed.

The waterproof target produced is a red and white checkerboard target, and the distance between the corner points of the target is 34.75mm. The designed projection target is a red and black checkerboard target, and the image distance between its corner points is 64 pixels. Using the image collection method of the present invention, the structured light sensor was put into the experimental pool to collect 10 groups of 20 images in total. Utilize the method described in the present invention to calibrate the projector and structural parameters, and the results are shown in Table 1. Table 2 shows the root mean square error between the three-dimensional coordinates of the reference value and the three-dimensional coordinates calculated by the model.

As can be seen from the data in Table 1 and Table 2, using the underwater projector calibration method proposed by the present invention, the projector parameters and structural parameters obtained by calibration are reliable, and the calibration accuracy of the method can meet the requirements of underwater qualitative monitoring and quantitative measurement. Task.

Table 1 Calibration results of the projector structured light sensor

Table 2 Root mean square error of three-dimensional coordinates

The above descriptions are examples of the present invention, and are not intended to limit the protection scope of the present invention.

Claims (2)

1. An underwater projector calibration method is characterized in that the method comprises the following steps: 1.1. Adjust the structured light sensor of the projector located underwater to ensure that the camera can capture clear images within the measurement range and the projector can project clear images within the measurement range. Within the common field of view of the projector and the calibrated camera, use A flat panel with a waterproof target, and the projector projects another target on the flat panel, which is called a projected target. Adjust the position of the flat panel until the two target feature points are within the common field of view; move the target freely at least three positions, Every time a position is moved, an image pair containing two targets is taken; 1.2. Extract the image coordinates of the waterproof target feature points in the image, and obtain the normal vector of the plane where the plate is located in the camera coordinate system according to the parameters of the calibrated camera; 1.3. Extract the image coordinates of the projected target feature points in the image, and use the normal vector obtained in step 1.2 to calculate the corresponding space point of the projected target projected on the flat panel by the camera parameters and the image coordinates of the feature points under the camera coordinate system three-dimensional coordinates; 1.4. According to the image coordinates of the projected target and the three-dimensional coordinates of its corresponding space points obtained in step 1.3, use the single-viewpoint model to perform preliminary calibration on the projector structured light sensor. The calibration parameters are the projector parameters and the representation of the projector. The structural parameters of the rigid body transformation between the camera and the camera; 1.5. Using the structural parameters obtained in step 1.4, the normal vector obtained in step 1.2 is expressed in the projector coordinate system; 1.6. In the coordinate system of the projector, given the initial value of the distance from the optical center of the projector to the window plane of the waterproof device along the optical axis, establish the time when the light emitted by each feature point on the projector image propagates in the air medium in the waterproof device The ray model, called the incident ray; 1.7. Establish the unit vector of the refracted ray, solve the scale factor according to the geometric constraints, determine the length of the refracted ray, and solve the three-dimensional coordinates of the projected target feature points in the coordinate system of the projector; 1.8. Convert the three-dimensional coordinates calculated in step 1.3 to the projector coordinate system using the structural parameters in step 1.4 as reference values; 1.9. Given the initial value of the deflection angle and rotation angle between the optical axis of the projector and the normal vector of the window plane, the objective function is established with the distance between the three-dimensional coordinates described in step 1.7 and the reference value described in 1.8. The projector parameters and structural parameters are optimized and recursively optimized, that is, the obtained structural parameters are used in steps 1.5 to 1.8 until the objective function value meets the set threshold. The threshold is set according to the measurement accuracy requirements, and its range is 10 ^-5 ~0.1mm. 2. A kind of underwater projector calibration method according to claim 1, is characterized in that: 2.1. The waterproof target described in step 1.1 is a planar target made of waterproof material, which is fixed on a flat plate. The thickness of the waterproof target is negligible compared with the measurement distance. The target feature points are grid points on the plane. The design of the distance is determined by the actual measurement distance and the field of view of the camera, and its range is 10-100 mm; the image coordinate distance between the grid points of the projection target is determined according to the resolution of the projector image, and its range is 10-100 mm. pixel; 2.2. For the waterproof target and projection target described in step 1.1, on the premise that the camera uses the filter device reasonably, the areas occupied by the two targets on the plate can overlap; when the plate is at the same position, the camera does not use the filter device and projects When the camera does not project the target, the camera only gets the image of the waterproof target. When the camera uses a filter device and the projector projects the target, the target overlaps with the waterproof target, and the camera only gets the image of the projected target; 2.3. The photographing of the image pair containing two targets described in step 1.1 is characterized in that when using a filter device, two images are photographed at each position, which are image pairs containing the waterproof target and the projection target respectively.