CN101476882B - Three-dimensional detection method of structured light based on homography matrix - Google Patents

Abstract

A homography matrix-based structured light three-dimensional detection method in the field of computer vision technology. In the present invention, the feature points in the checkerboard pattern are used to obtain the homography matrices corresponding to the camera imaging planes and planes _P1 and _P2 respectively, At the same time, the homography matrices corresponding to the projection plane of the projector and the imaging plane of the camera are measured on the planes P ₁ and P ₂ respectively, given the straight line equation of a certain grating on the projection plane, the imaging plane and the world coordinates are obtained according to the homography matrix The plane under the world coordinate system is established, and the relationship between the imaging surface and the plane under the world coordinate system is established, and the position of each grating in the world coordinate system is obtained through the transformation relationship, so as to obtain three-dimensional information. The invention can easily measure the world coordinates and avoid the problem of equipment correlation. The invention simplifies the operation of three-dimensional detection under the condition of ensuring the accuracy.

Description

Three-dimensional detection method of structured light based on homography matrix

technical field

The invention relates to a detection method in the technical field of metrology, in particular to a three-dimensional detection method of structured light based on a homography matrix.

Background technique

The three-dimensional detection of objects is an important direction of computer vision research, and its goal is to obtain an accurate description of the surface geometry of the target object. Structured light three-dimensional inspection has been widely used for its advantages of simple use, accurate measurement, and non-destructive. The three-dimensional detection method of structured light is mainly divided into two parts. The first part is to project the structured light and take pictures to obtain the structured light image, and then decode the structured light to obtain the two-dimensional information of the structured light image; the second part is based on the projector, camera and target. The relative position of the object converts the two-dimensional information of the structured light image into the three-dimensional information of the target object.

After searching the prior art documents, it was found that Tsai R Y wrote an article "A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the- -shelf TV cameras and lenses" (a common camera calibration technology using spot cameras and applying general-purpose camera calibration technology in high-precision 3D machine vision measurement) proposes a method for the second part of structured light 3D measurement, which is mainly to first test the projector Calibrate with the camera, determine the focus position of the projector and the camera, and then calculate the three-dimensional information of the target object according to the relationship of triangulation. This method needs to measure the world coordinates of irregular feature points when the projector is calibrated; the manufacturer needs to provide the actual distance between two adjacent pixels (horizontal and vertical) between the projector and the camera; when the projector and the camera are calibrated The focal length of the projector and the camera must be considered, and there are many calculation steps; after the preparations are completed, the pixel coordinates of a point on the photo (a point on the grating) can be used to obtain the world coordinates of the corresponding object point. The calculation is relatively complicated. First, the world coordinates of this point must be obtained through a matrix multiplication, and then the linear equation determined by this point and the camera is solved. Finally, the prepared projector and the plane equation determined by the grating are combined to form a ternary once Equations, solve this equations to get the world coordinates of the object point.

Contents of the invention

The present invention aims at the deficiencies of the above-mentioned prior art, and proposes a three-dimensional detection method of structured light based on a homography matrix, so that it uses the homography matrix to determine the relative positions of projectors, cameras, and target objects , the world coordinates of the object point can be obtained from the pixel coordinates of a certain point on the photo (the point on the grating) only through a matrix multiplication, and the present invention simplifies the operation of three-dimensional detection under the condition of ensuring accuracy.

The present invention is achieved through the following technical solutions, and the present invention comprises the following steps:

Step 1, two planes P ₁ and P ₂ parallel to each other are set between the projector and the camera, the two planes P ₁ and P ₂ are used to place the checkerboard pattern for obtaining the homography matrix in the following steps, the projector A projection surface with a grating is provided between the plane _P2 , and a grating on the projector and the projection surface determines a light plane;

Step 2, use the feature points in the checkerboard pattern to obtain the homography matrices H _CQ1 and H _CQ2 corresponding to the camera imaging plane and the planes P ₁ and P ₂ respectively, as follows:

First, place the checkerboard pattern on the planes P ₁ and P ₂ respectively, and then take pictures with a camera to obtain two images respectively;

Then, the relative position of the feature points in the checkerboard image is measured, and one of the feature points is taken as the origin of the world coordinate system, and the coordinates of other feature points are determined, and the distance between the planes P ₁ and P ₂ is measured at the same time , that is, the distance of the feature point in the Z direction;

Finally, according to the pixel coordinates of the feature points in the checkerboard image, the homography matrices H _CQ1 and H _CQ2 corresponding to the camera imaging surface and the planes P ₁ and P ₂ are obtained respectively. The homography matrix makes the plane P ₁ or P ₂ The coordinates (X, Y) of any point (x, y) imaged on the camera satisfy the following relationship:

$\left(\begin{array}{c}\mathrm{<span\; class="notranslate">\; x</span>}\\ \mathrm{<span\; class="notranslate">\; the\; y</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right)<span\; class="notranslate">\; =</span>\left(\begin{array}{ccc}{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 11</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 13</span>}}\\ {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; one</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; two</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; three</span>}}\\ {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 31</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 32</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 33</span>}}\end{array}\right)\left(\begin{array}{c}\mathrm{<span\; class="notranslate">\; x</span>}\\ \mathrm{<span\; class="notranslate">\; Y</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right)<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

Among them, (X, Y) is the pixel coordinate of the feature point in the imaging plane, and (x, y) is the world coordinate of the feature point in the plane _P1 or _P2 .

The checkerboard pattern is placed on the planes _P1 and _P2 respectively, specifically: the coordinates of the feature points of the checkerboard pattern on the two planes in the X and Y directions are the same, and one of the feature points is taken as the world At the same time, the coordinates of other feature points can be determined to facilitate measurement.

Step 3, use the feature points in the checkerboard pattern to measure the homography matrices H _PC1 and H _PC2 corresponding to the projection plane of the projector and the imaging plane of the camera on the planes P ₁ and P ₂ respectively, as follows:

First, project the checkerboard images onto the planes P ₁ and P ₂ respectively, take pictures with the camera, and obtain two images respectively;

Then, measure the pixel coordinates of the feature points of the checkerboard on the projected checkerboard image and the captured two images;

Finally, using the homography matrix in the same form as in step 2, the homography matrices HP _PC1 and HP _PC2 corresponding to the projection plane of the projector and the imaging plane of the camera on the planes P ₁ and P ₂ are obtained.

Step 4, given the straight line equation of a certain grating on the projection surface, according to the homography matrix H _PC1 and matrix H _PC2 in step 3, the straight line C ₁₁ C ₁₂ and the straight line C ₂₁ C ₂₂ are obtained, and the straight line C ₁₁ C ₁₂ and C ₂₁ C ₂₂ obtains the imaging plane C ₁₁ C ₁₂ C ₂₂ C ₂₁ ;

Step 5, _obtain straight line Q ₁₁ Q ₁₂ and straight line Q ₂₁ Q ₂₂ according to the straight line C ₁₁ C 12 , straight line C ₂₁ C ₂₂ obtained in step 4 and the homography matrix H _CQ1 and matrix H _CQ2 in step 2. Q ₁₁ Q ₁₂ and Q ₂₁ Q ₂₂ determine the plane Q ₁₁ Q ₁₂ Q ₂₂ Q ₂₁ in the world coordinate system, and the plane Q ₁₁ Q ₁₂ Q ₂₂ Q ₂₁ is a plane in three-dimensional space;

Step six, establish the relationship between the imaging plane C ₁₁ C ₁₂ C ₂₂ C ₂₁ and the plane Q ₁₁ Q ₁₂ Q ₂₂ Q ₂₁ in the world coordinate system: (X, Y, 1) ^T = R(X _w , Y _w , Z _w ) ^T +T, where (X, Y) is the coordinates in the imaging plane C ₁₁ C ₁₂ C ₂₂ C ₂₁ , (X _w , Y _w , Z _w ) is the plane Q ₁₁ Q ₁₂ Q in the world coordinate system ₂₂ The coordinates in Q ₂₁ , R is the rotation matrix, and T is the translation vector, then the formula (1) in step 2 is transformed into:

$\left(\begin{array}{c}\mathrm{<span\; class="notranslate">\; x</span>}\\ \mathrm{<span\; class="notranslate">\; the\; y</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right)<span\; class="notranslate">\; =</span>\left(\begin{array}{ccc}{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 11</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 13</span>}}\\ {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; one</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; two</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; three</span>}}\\ {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 31</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 32</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 33</span>}}\end{array}\right)\left(\begin{array}{cccc}{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 11</span>}}& {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}& {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 13</span>}}& {\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}\\ {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; one</span>}}& {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; two</span>}}& {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; three</span>}}& {\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}\\ {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 31</span>}}& {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 32</span>}}& {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 33</span>}}& {\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; z</span>}}\end{array}\right)\left(\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right)<span\; class="notranslate">\; =</span>\left(\begin{array}{cccc}{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 11</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 13</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 14</span>}}\\ {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; one</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; two</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; three</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; four</span>}}\\ {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 31</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 32</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 33</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 34</span>}}\end{array}\right)\left(\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right)<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

It can be seen that the 3×4 matrix in formula (2) represents the transformation relationship between the two surfaces. The 3×4 matrix can be obtained by using enough corresponding feature points on the two surfaces.

Step 7: Put the target object on the plane _P2 , project the structured light, take a set of images containing structured light, and then decode the structured light to distinguish the number of gratings, and obtain the grating under the influence of the target object. The position on the imaging surface of the camera;

Step eight, on the basis of obtaining the position of the grating on the imaging plane of the camera under the influence of the object in step seven, obtain the planes C ₁₁ C ₁₂ C ₂₂ C ₂₁ and Q ₁₁ Q ₁₂ Q ₂₂ Q ₂₁ respectively determined by each grating , and the 3×4 matrix mapped from the plane C ₁₁ C ₁₂ C ₂₂ C ₂₁ (camera imaging surface coordinate system) to the plane Q ₁₁ Q ₁₂ Q ₂₂ Q ₂₁ (world coordinate system), and each The position of the raster in the world coordinate system to obtain three-dimensional information.

In the present invention, the optical plane determined by the projector and a certain grating completely coincides with the plane Q ₁₁ Q ₁₂ Q ₂₂ Q ₂₁ obtained in the above step 5. Using this characteristic, the plane Q ₁₁ Q ₁₂ Q ₂₂ Q ₂₁ is calculated and obtained The light plane determined by the projector and a certain grating, where the plane P ₂ is the target plane where the items will be placed, the plane P ₁ and P ₂ are parallel, the straight line Q ₁₁ Q ₁₂ is the intersection line between the plane P ₁ and the light plane, and the straight line Q ₂₁ Q ₂₂ is the intersection of the plane P ₂ and the light plane. The imaging of the straight line Q ₁₁ Q ₁₂ on the camera is the straight line C ₁₁ C ₁₂ , and the imaging of the straight line Q ₂₁ Q ₂₂ on the camera is the straight line C ₂₁ C ₂₂ . Suppose the arc on the plane Q ₁₁ Q ₁₂ Q ₂₂ Q ₂₁ is formed by grating projection on an object, then the image of this arc on the camera must be an arc on the plane C ₁₁ C ₁₂ C ₂₂ C ₂₁ Wire. It can be seen from the above description that there is a 3×4 matrix that makes the points on the plane C ₁₁ C ₁₂ C ₂₂ C ₂₁ and Q ₁₁ Q ₁₂ Q ₂₂ Q ₂₁ one-to-one correspondence, then this matrix must be able to make two arcs on the two planes The points above correspond one by one. That is to say, when the 3×4 matrix is known, specifying a point (pixel coordinate) on the imaginary arc can get the corresponding point (world coordinate) on the real arc, which achieves the purpose of three-dimensional measurement.

Compared with the prior art, the present invention has the following beneficial effects:

1. The present invention can easily measure the world coordinates. The prior art method needs to measure the world coordinates of irregular feature points when the projector is calibrated, but the method of the present invention only needs to measure the world coordinates of regular feature points. For example, the feature points used are 7 * 5=35, the prior art method needs to measure 35 feature points, and after the method of the present invention only needs to measure 2, calculate and obtain other feature points again;

2. The present invention does not require the manufacturer to provide the actual distance between the two adjacent pixels (horizontal and vertical) of the projector and the camera, that is, the method in the prior art is related to equipment, and the method of the present invention avoids in this respect Fixed device dependency issues;

3. The present invention omits the complicated calculation process of the focal length of the projector and the camera, thereby also avoiding the problem of device correlation in this aspect;

4. In the present invention, it is more convenient to obtain the world coordinates of the corresponding object point from the pixel coordinates of a certain point on the photo (the point on the grating), and the world coordinates of the object point can be obtained only by a matrix multiplication, while the existing The technical method first obtains the world coordinates of this point through a matrix multiplication, then solves the line equation determined by this point and the camera, and finally combines the prepared projector and the plane equation determined by the grating to form a ternary linear equation group. The world coordinates of the object point can be obtained by solving this system of equations.

Description of drawings

Fig. 1 is a schematic diagram of the working principle of the present invention.

Fig. 2 is the real chessboard that the present invention uses;

In the figure, (a) is the real checkerboard on the P ₁ plane captured by the camera, and (b) is the real checkerboard on the P ₂ plane captured by the camera.

Fig. 3 is the checkerboard of projection among the present invention;

In the figure, (a) is the projection checkerboard on the P ₁ plane captured by the camera, and (b) is the projection checkerboard on the P ₂ plane captured by the camera.

Fig. 4 is the image of target object under structured light in the present invention;

In the figure, (a) is the object captured by the camera under the influence of the first frame of structured light on the _P2 plane, and (b) is the object captured by the camera on the _P2 plane under the influence of the second frame of structured light (c) is the object captured by the camera on the P ₂ plane under the influence of the third frame of structured light, (d) is the object captured by the camera on the P ₂ plane under the influence of the fourth frame of structured light (e) is the object captured by the camera on the P ₂ plane under the influence of the fifth frame of structured light, (f) is the object captured by the camera on the P ₂ plane under the influence of the sixth frame of structured light (g) is the object captured by the camera on the P ₂ plane under the influence of the structured light in the seventh frame.

Fig. 5 is the three-dimensional outline of target object among the present invention;

In the figure, (a) is the three-dimensional outline of the target object at a certain angle, and (b) is the three-dimensional outline of the target object at another angle.

Detailed ways

The embodiments of the present invention are described in detail below in conjunction with the accompanying drawings: this embodiment is implemented on the premise of the technical solution of the present invention, and detailed implementation methods and specific operating procedures are provided, but the protection scope of the present invention is not limited to the following the described embodiment.

As shown in Figure 1, this embodiment includes the following steps:

Step 1, two planes P ₁ and P ₂ parallel to each other are set between the projector and the camera, the two planes P ₁ and P ₂ are used to place the checkerboard pattern for obtaining the homography matrix in the following steps, the projector A projection surface with a grating is provided between the plane _P2 , and a grating on the projector and the projection surface determines a light plane;

Step 2, use the feature points in the checkerboard pattern to obtain the homography matrices H _CQ1 and H _CQ2 corresponding to the camera imaging plane and the planes P ₁ and P ₂ respectively, as follows:

First, place the checkerboard pattern on the planes P ₁ and P ₂ respectively, and then take pictures with a camera to obtain two images respectively, as shown in Figure 2;

Then, the relative position of the feature points in the checkerboard image is measured, and one of the feature points is taken as the origin of the world coordinate system, and the coordinates of other feature points are determined, and the distance between the planes P ₁ and P ₂ is measured at the same time , that is, the distance of the feature point in the Z direction;

Finally, according to the pixel coordinates of the feature points in the checkerboard image, the homography matrices H _CQ1 and H _CQ2 corresponding to the camera imaging surface and the planes P ₁ and P ₂ are obtained respectively. The homography matrix makes the plane P ₁ or P ₂ The coordinates (X, Y) of any point (x, y) imaged on the camera satisfy the following relationship:

$\left(\begin{array}{c}\mathrm{<span\; class="notranslate">\; x</span>}\\ \mathrm{<span\; class="notranslate">\; the\; y</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right)<span\; class="notranslate">\; =</span>\left(\begin{array}{ccc}{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 11</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 13</span>}}\\ {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; one</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; two</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; three</span>}}\\ {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 31</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 32</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 33</span>}}\end{array}\right)\left(\begin{array}{c}\mathrm{<span\; class="notranslate">\; x</span>}\\ \mathrm{<span\; class="notranslate">\; Y</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right)<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

Among them, (X, Y) is the pixel coordinate of the feature point in the imaging plane, and (x, y) is the world coordinate of the feature point in the plane _P1 or _P2 .

The checkerboard pattern is placed on the planes _P1 and _P2 respectively, specifically: the coordinates of the feature points of the checkerboard pattern on the two planes in the X and Y directions are the same, and one of the feature points is taken as the world At the same time, the coordinates of other feature points can be determined to facilitate measurement.

Step 3, use the feature points in the checkerboard pattern to measure the homography matrices H _PC1 and H _PC2 corresponding to the projection plane of the projector and the imaging plane of the camera on the planes P ₁ and P ₂ respectively, as follows:

First, the checkerboard images are projected onto the planes P ₁ and P ₂ respectively, and the camera takes pictures to obtain two images respectively, as shown in Figure 3;

Then, measure the pixel coordinates of the feature points of the checkerboard on the projected checkerboard image and the two images captured, and the pixel coordinates of the feature points of the checkerboard on the projected checkerboard image and the two images captured can be relatively easy Measured (it can be automatically measured by computer, or can be manually found out by using image software such as Photoshop);

Finally, using the homography matrix in the same form as in step 2, the homography matrices HP _PC1 and HP _PC2 corresponding to the projection plane of the projector and the imaging plane of the camera on the planes P ₁ and P ₂ are obtained.

Step 4, given the straight line equation of a certain grating on the projection surface, according to the homography matrix H _PC1 and matrix H _PC2 in step 3, the straight line C ₁₁ C ₁₂ and the straight line C ₂₁ C ₂₂ are obtained, and the straight line C ₁₁ C ₁₂ and C ₂₁ C ₂₂ obtains the imaging plane C ₁₁ C ₁₂ C ₂₂ C ₂₁ ;

Step 5, _obtain straight line Q ₁₁ Q ₁₂ and straight line Q ₂₁ Q ₂₂ according to the straight line _C 11 C 12 , straight line C ₂₁ C ₂₂ obtained in step 4 and the homography matrix H _CQ1 and matrix H _CQ2 in step 2. Q ₁₁ Q ₁₂ and Q ₂₁ Q ₂₂ determine the plane Q ₁₁ Q ₁₂ Q ₂₂ Q ₂₁ in the world coordinate system, and the plane Q ₁₁ Q ₁₂ Q ₂₂ Q ₂₁ is a plane in three-dimensional space;

Step six, establish the relationship between the imaging plane C ₁₁ C ₁₂ C ₂₂ C ₂₁ and the plane Q ₁₁ Q ₁₂ Q ₂₂ Q ₂₁ in the world coordinate system: (X, Y, 1) ^T = R(X _w , Y _w , Z _w ) ^T +T, where (X, Y) is the coordinates in the imaging plane C ₁₁ C ₁₂ C ₂₂ C ₂₁ , (X _w , Y _w , Z _w ) is the plane Q ₁₁ Q ₁₂ Q in the world coordinate system ₂₂ The coordinates in Q ₂₁ , R is the rotation matrix, and T is the translation vector, then the formula (1) in step 2 is transformed into:

$\left(\begin{array}{c}\mathrm{<span\; class="notranslate">\; x</span>}\\ \mathrm{<span\; class="notranslate">\; the\; y</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right)<span\; class="notranslate">\; =</span>\left(\begin{array}{ccc}{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 11</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 13</span>}}\\ {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; one</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; two</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; three</span>}}\\ {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 31</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 32</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 33</span>}}\end{array}\right)\left(\begin{array}{cccc}{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 11</span>}}& {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}& {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 13</span>}}& {\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}\\ {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; one</span>}}& {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; two</span>}}& {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; three</span>}}& {\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}\\ {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 31</span>}}& {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 32</span>}}& {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 33</span>}}& {\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; z</span>}}\end{array}\right)\left(\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right)<span\; class="notranslate">\; =</span>\left(\begin{array}{cccc}{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 11</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 13</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 14</span>}}\\ {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; one</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; two</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; three</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; four</span>}}\\ {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 31</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 32</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 33</span>}}& {\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 34</span>}}\end{array}\right)\left(\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right)<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

It can be seen that the 3×4 matrix in formula (2) represents the transformation relationship between the two surfaces. The 3×4 matrix can be obtained by using enough corresponding feature points on the two surfaces.

Step 7, put the target object on the plane _P2 , project the structured light, take a group of images containing structured light (as shown in Figure 4), and then decode the structured light to distinguish the number of gratings. This implementation In the example, there are ²⁷ gratings in total, and the position of the grating on the imaging surface of the camera under the influence of the target object is obtained;

The decoding of the structured light is specifically: according to the number of images, the binary number of the corresponding number of digits is set, and the corresponding binary number is set in any point area in the image, and the number indicates which grating it belongs to, such as: Figure 4 (a )(b) Two pictures, the first picture has only two areas black and white, the second picture has four black and white areas, which can be represented by two-digit binary numbers and distinguish these 4 areas——(00, 01 , 10, 11), 0 means black, 1 means white, the first bit means the first picture, and the second bit means the second picture (the first bit of area 01 is 0 means that this area is black in the first picture , the second bit is 1, indicating that this area is white in the second picture). There are 7 pictures in Fig. 4, therefore, a 7-bit binary number should be selected.

Step eight, on the basis of obtaining the position of the grating on the imaging plane of the camera under the influence of the object in step seven, obtain the planes C ₁₁ C ₁₂ C ₂₂ C ₂₁ and Q ₁₁ Q ₁₂ Q ₂₂ Q ₂₁ respectively determined by each grating , and the 3×4 matrix mapped from the plane C ₁₁ C ₁₂ C ₂₂ C ₂₁ (camera imaging surface coordinate system) to the plane Q ₁₁ Q ₁₂ Q ₂₂ Q ₂₁ (world coordinate system), and each The position of the raster in the world coordinate system to obtain three-dimensional information.

As shown in Figure 5, it is the effect of looking at the past from two angles. A white line is a grating, and the white line formed by all the gratings can construct the three-dimensional outline of the target object.

The method of this embodiment can easily measure the world coordinates, avoid the problem of device correlation, and simplify the operation of three-dimensional detection under the condition of ensuring accuracy.

Claims (4)

1. the structured light three-dimensional detection method based on homography matrix is characterized in that, comprises the steps:

Step 1 is provided with two plane P parallel to each other between projector and video camera ₁And P ₂, two plane P ₁And P ₂Be used to place gridiron pattern pattern, projector and the plane P of obtaining homography matrix ₁Between be provided with the projecting plane that has grating, a grating on projector and the projecting plane is determined an optical plane;

Step 2, utilize unique point in the gridiron pattern pattern obtain the camera imaging face respectively with plane P ₁And P ₂Corresponding homography matrix H _CQ1And H _CQ2

Step 3 utilizes unique point in the gridiron pattern pattern in plane P ₁And P ₂On measure the homography matrix H of projector projecting plane and camera imaging face correspondence respectively _PC1And H _PC2

Step 4, the straight-line equation of certain bar grating on the given projecting plane is according to the homography matrix H in the step 3 _PC1, matrix H _PC2Obtain straight line C ₁₁C ₁₂With straight line C ₂₁C ₂₂, by straight line C ₁₁C ₁₂And C ₂₁C ₂₂Be specified to image planes C ₁₁C ₁₂C ₂₂C ₂₁

Step 5, the straight line C that tries to achieve according to step 4 ₁₁C ₁₂, straight line C ₂₁C ₂₂With the homography matrix H in the step 2 _CQ1, matrix H _CQ2Obtain straight line Q ₁₁Q ₁₂With straight line Q ₂₁Q ₂₂, by straight line Q ₁₁Q ₁₂And Q ₂₁Q ₂₂Determine the plane Q under the world coordinate system ₁₁Q ₁₂Q ₂₂Q ₂₁, plane Q ₁₁Q ₁₂Q ₂₂Q ₂₁Be on three-dimensional plane;

Step 6 is created as image planes C ₁₁C ₁₂C ₂₂C ₂₁With world coordinate system lower plane Q ₁₁Q ₁₂Q ₂₂Q ₂₁Between relation: (X, Y, 1) ^T=R (X _w, Y _w, Z _w) ^T+ T, wherein, (X Y) is imaging surface C ₁₁C ₁₂C ₂₂C ₂₁In coordinate, (X _w, Y _w, Z _w) be world coordinate system lower plane Q ₁₁Q ₁₂Q ₂₂Q ₂₁In coordinate, R is a rotation matrix, T is translation vector, then plane P ₁Or P ₂Go up any one point (x, y) with the world coordinate system lower plane on corresponding point (X _w, Y _w, Z _w) relation as follows:

$\left(\begin{array}{c}x\\ y\\ 1\end{array}\right)=\left(\begin{array}{ccc}{h}_{11}& h_{12}& h_{13}\\ h_{21}& h_{22}& h_{23}\\ h_{31}& h_{32}& h_{33}\end{array}\right)\left(\begin{array}{cccc}{r}_{11}& r_{12}& r_{13}& T_x\\ r_{21}& r_{22}& r_{23}& T_y\\ r_{31}& r_{32}& r_{33}& T_z\end{array}\right)\left(\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right)=\left(\begin{array}{cccc}{H}_{11}& H_{12}& H_{13}& H_{14}\\ H_{21}& H_{22}& H_{23}& H_{24}\\ H_{31}& H_{32}& H_{33}& H_{34}\end{array}\right)\left(\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right)$

In the formula, 3 * 4 matrix tables illustrate the transformation relation of two faces, utilize on imaging surface and the world coordinate system lower plane characteristic of correspondence point can determine this 3 * 4 matrix;

Step 7 is placed on plane P to target object ₂On, projecting structural optical is taken the set of diagrams picture that obtains containing structured light, then structured light is decoded, and distinguishes the number of grating, and obtaining grating is having the position on the camera imaging face under the target object influence;

Step 8, obtaining grating in step 7 is having under the object influence, on the basis of the position on the camera imaging face, acquires the plane C that every grating is determined separately ₁₁C ₁₂C ₂₂C ₂₁And Q ₁₁Q ₁₂Q ₂₂Q ₂₁, and by plane C ₁₁C ₁₂C ₂₂C ₂₁To plane Q ₁₁Q ₁₂Q ₂₂Q ₂₁3 * 4 matrixes of mapping obtain every grating position under world coordinate system by the transformation relation in the step 6, thereby obtain three-dimensional information.

2. the structured light three-dimensional detection method based on homography matrix according to claim 1 is characterized in that, described utilize unique point in the gridiron pattern pattern obtain the camera imaging face respectively with plane P ₁And P ₂Corresponding homography matrix H _CQ1And H _CQ2, specific as follows:

At first, the gridiron pattern pattern is placed in plane P respectively ₁And P ₂On, take pictures with camera then, obtain two width of cloth images respectively;

Then, measure the relative position of the unique point in the cross-hatch pattern picture, get the initial point of one of them unique point as world coordinate system, other characteristic point coordinates have also just been determined, have been measured plane P simultaneously ₁And P ₂Between distance, i.e. the distance of unique point on the Z direction;

At last, according to the pixel coordinate of the unique point in the cross-hatch pattern picture, obtain the camera imaging face respectively with plane P ₁And P ₂Corresponding homography matrix H _CQ1And H _CQ2, homography matrix makes plane P ₁Or P ₂Go up any one point (x, y) on camera the coordinate of imaging (X, Y) satisfy following relation:

$\left(\begin{array}{c}x\\ y\\ 1\end{array}\right)=\left(\begin{array}{ccc}{h}_{11}& h_{12}& h_{13}\\ h_{21}& h_{22}& h_{23}\\ h_{31}& h_{32}& h_{33}\end{array}\right)\left(\begin{array}{c}X\\ Y\\ 1\end{array}\right)$

Wherein, (X Y) is the pixel coordinate of unique point in the imaging surface, and (x y) is plane P ₁Or P ₂The world coordinates of middle unique point.

3. the structured light three-dimensional detection method based on homography matrix according to claim 1 is characterized in that, describedly utilizes unique point in the gridiron pattern pattern in plane P ₁And P ₂On measure the homography matrix H of projector projecting plane and camera imaging face correspondence respectively _PC1And H _PC2, specific as follows:

At first, the cross-hatch pattern picture is projected to plane P respectively ₁And P ₂On, take pictures by camera, obtain two width of cloth images respectively;

Then, measure projection cross-hatch pattern picture and two width of cloth images that photograph on the pixel coordinate of tessellated unique point;

At last, the homography matrix of same form in employing and the step 2 obtains in plane P ₁And P ₂The homography matrix H of last projector projecting plane and camera imaging face correspondence _PC1And H _PC2

4. according to claim 2 or 3 described structured light three-dimensional detection methods, it is characterized in that, described the gridiron pattern pattern is projected to plane P respectively based on homography matrix ₁And P ₂On, be specially: the gridiron pattern pattern is identical with coordinate on the Y direction at X in the unique point on two planes, to realize getting the initial point of one of them unique point as world coordinate system, just can determine other characteristic point coordinates simultaneously.

Images