CN109556535B - A one-step reconstruction method of 3D surface shape based on color fringe projection - Google Patents

Abstract

The invention relates to a three-dimensional surface type one-step reconstruction method based on color stripe projection, which comprises the steps of generating a color sine stripe image through a computer, transmitting the color sine stripe image to a projector to be projected on the surface of a measured object, and collecting the image of the measured object with a stripe image by a color industrial camera which forms a certain angle with the projector; then, solving the acquired measured object image by using a phase shift method to obtain a phase image; performing foreground and background segmentation to obtain partial background information, and performing function fitting on the partial background information to recover a complete background phase diagram; and finally, mapping depth information through the phase difference between the foreground and the background. Compared with the traditional phase shift method, the method does not need to separately acquire a plurality of background images, but completes reconstruction in one step by utilizing function fitting, reduces the number of acquired measured object images, directly skips the step of separately acquiring the background images, and realizes flexible and rapid three-dimensional surface type reconstruction.

Description

A one-step reconstruction method of 3D surface shape based on color fringe projection

technical field

The invention belongs to optical detection technology and relates to the technical field of non-contact three-dimensional surface reconstruction and measurement, in particular to a three-dimensional surface reconstruction method based on color fringe projection.

Background technique

The development of stereo vision is becoming more and more popular, and the reconstruction of the three-dimensional appearance of objects is widely used, such as three-dimensional face recognition, three-dimensional size measurement of workpieces, and three-dimensional face observation in medical images.

At present, the 3D reconstruction technology is mainly divided into two types: passive and active.

Among them, the passive 3D reconstruction is a binocular vision technology, which uses the simulated human eyes to perceive the depth of the object. This scheme is not ideal for the reconstruction of objects whose surface lacks texture. The active reconstruction technology mainly includes the scheme of structured light projection. Among them, the sinusoidal stripe structured light method can achieve relatively high reconstruction accuracy by using the phase and height mapping method, but its disadvantage is that using the phase shift method requires at least 6 images to be captured, and the background and foreground must be captured in two steps. This operation Greatly affects the reconstruction efficiency.

SUMMARY OF THE INVENTION

The purpose of the present invention is to overcome the deficiencies of the prior art and provide a one-step reconstruction method of three-dimensional surface shape based on color fringe projection. By fitting the background phase data, only two structures can be projected in the three-dimensional reconstruction of structured light projection. The reconstruction process can be completed quickly with the light image, and this process does not require the cumbersome steps of separately collecting background images.

For achieving the above object, the technical scheme provided by the present invention is:

A one-step reconstruction method of three-dimensional surface shape based on color fringe projection, comprising the following steps:

S1: Build a structured light projection 3D reconstruction system;

S2: Measure the distance L between the optical center of the RGB industrial camera and the receiving plane, and the distance D between the optical center of the RGB industrial camera and the projector;

S3: Projecting structured light to collect the measured object images I_obj1 and I_obj2;

S4: Process the collected images to complete the three-dimensional surface reconstruction.

Further, the structured light projection three-dimensional reconstruction system built in the step S1 includes a projector, an RGB industrial camera, a computer, an optical flat panel and a receiving plane; when the system is built, the projector and the camera are respectively fixed on the optical flat panel, and the two The axis of the optical center is at a certain angle, and the line connecting the two optical centers is kept parallel to the receiving plane.

Further, the specific process of projecting structured light to collect the measured object images I_obj1 and I_obj2 in the step S3 is as follows:

Four sinusoidal fringe patterns with the same phase difference are generated by the computer, and the four sinusoidal fringe patterns are divided into two groups and input to the R and B channels of the two color images by taking advantage of the advantage of the color image with three RGB channels; two color fringes are obtained. After the structured light pattern is formed, a projector is used to project the two colored striped structured light patterns on the surface of the object to be measured in turn, and an RGB industrial camera is used to sequentially collect images I_obj1 and I_obj2 with striped structured light.

Further, the specific steps of processing the collected images in the step S4 are as follows:

S4-1: Use the four-step phase shift method to obtain the wrapped phase map of the measured object, and unwrap it to obtain the continuous phase map unwrap_obj of the measured object;

S4-2: In the original measured object image I_obj1 or I_obj2, extract the measured object ROI area and segment its minimum circumscribed rectangular area, and remove the data in this area, so that the measured object in the continuous phase map of the measured object is unwrap_obj. The data in the minimum circumscribed rectangular area is all empty, leaving the remaining background phase information, and the background incomplete phase map unwrap_subori is obtained at this time;

S4-3: Divide the background incomplete phase map unwrap_subori into two parts from the middle, and then divide the left and right parts into several sub-regions per N lines respectively;

S4-4: Perform the least squares binary quadratic term fitting on all sub-regions respectively;

S4-5: splicing the fitting results of all sub-regions into a complete background phase map;

S4-6: Use the phase difference height mapping formula to calculate the depth of each point of the measured object, obtain its three-dimensional point cloud data, and complete the three-dimensional surface reconstruction.

Further, the phase difference height mapping in the step S4-6 is the phase of the measured object minus the background phase, combined with the pre-calibrated system parameters L, D and f, through the following formula:

Obtain the depth information of the measured object; in the formula, L is the distance between the optical center of the known RGB industrial camera and the receiving plane; D is the distance between the optical center of the RGB industrial camera and the projector; f is the frequency of generating fringes.

Compared with the prior art, the principle and advantages of this scheme are as follows:

1. In the current three-dimensional reconstruction scheme using phase shift technology, it is necessary to collect three or more images of the object to be measured and three background images, and at least six images in total. The solution of the invention realizes that three-dimensional surface reconstruction can be performed only by collecting two or more images of the object to be measured, and without collecting background images separately, thus simplifying the operation process and reducing the collecting time, and greatly improving the working efficiency of the system.

2. Since the solution of the present invention does not need to collect the background image separately, there is no problem that the position of the background receiving surface moves before and after shooting, resulting in incorrect reconstruction results. There is no need to set a fixed receiving plane, and the flexibility and applicability of the system are greatly improved.

3. Due to the influence of uncontrollable factors such as the reference surface is not strictly flat, the quality of the stripes projected by the projector and the illumination is not strictly uniform, the actual continuous phase map must not be an ideal spatial plane. By dividing the background phase image into several sub-regions by row, the fitting accuracy can be improved, the fitted background phase image is closer to the real distribution, and the reconstruction error is reduced.

4. Taking advantage of the RGB three-channel color image, one color stripe image can store up to three grayscale stripe images. Considering the effect of crosstalk between the three-color channels, only the sinusoidal fringe information is stored in the R and B channels, which greatly reduces the effect of color crosstalk. At the same time, under the premise of using the four-step phase-shifting technique, the gray-scale fringe image needs to project four images, while using the RB color image only needs to project two images to solve the continuous phase map.

Description of drawings

1 is a schematic diagram of a structured light projection system;

Fig. 2 is a schematic diagram of block fitting of background phase map;

FIG. 3 is a working flow chart of a one-step reconstruction method of three-dimensional surface shape based on color fringe projection according to the present invention.

Detailed ways

Below in conjunction with specific embodiment, the present invention will be further described:

As shown in FIG. 3 , a one-step reconstruction method for three-dimensional surface shape based on color fringe projection described in this embodiment includes the following steps:

S1: Build a structured light projection 3D reconstruction system:

The structured light projection 3D reconstruction system is shown in Figure 1, including a projector, an RGB industrial camera, a computer, an optical flat panel and a receiving plane; when the system is built, the projector and the camera are respectively fixed on the optical flat panel, and the optical center axes of the two are At a certain angle, the line connecting the optical centers of the two remains parallel to the receiving plane.

S2: After the system is built, measure the distance L between the optical center of the RGB industrial camera and the receiving plane, and the distance D between the optical center of the RGB industrial camera and the projector.

S3: Projecting structured light to collect the measured object images I_obj1 and I_obj2, the specific steps are:

Four sinusoidal fringe patterns with the same phase difference are generated by the computer, and the fringe patterns are input into the RGB channels by using the advantage of the color image with three RGB channels. Considering that crosstalk will occur between the three RGB channels and affect the quality of the fringe images collected later, the present invention chooses to divide the four sinusoidal fringe images into two groups and input them to the R and B channels of the two color images to reduce the crosstalk phenomenon. For example: the computer generates four sine fringe patterns I1, I2, I3, I4, the phases are 0, pi/2, pi, 3pi/2 in turn, and then I1 and I2 are input to the R and B channels of the color image I_color1 respectively, and the I3 and I4 are input to the R and B channels of the color image I_color2, respectively.

After obtaining I_color1 and I_color2, use a projector to project these two colored striped structured lights onto the surface of the object to be measured in turn, and use a color industrial camera to sequentially collect images I_obj1 and I_obj2 with striped structured light.

S4: Process the collected image, and the specific process is as follows:

S4-1: Use the four-step phase shift method to obtain the wrapped phase map of the measured object, and unwrap it to obtain the continuous phase map unwrap_obj of the measured object;

S4-2: In the original measured object image I_obj1 or I_obj2, extract the measured object ROI area and segment its minimum circumscribed rectangular area, and remove the data in this area, so that the measured object in the continuous phase map of the measured object is unwrap_obj. The data in the minimum circumscribed rectangular area is all empty, leaving the remaining background phase information, and the background incomplete phase map unwrap_subori is obtained at this time;

S4-3: As shown in Figure 2(a), the background incomplete phase map unwrap_subori is divided into two parts from the middle, and then the left and right parts are divided into several sub-regions by N lines respectively; the left half of the image data is used to fit the The data of the left half of the measured object, and the right half of the image data fit the data of the right half of the measured object. Assuming that the background incomplete phase map unwrap_subori has N lines in total, then the left and right half of the image are divided into sub-regions by every n lines, as shown in Figure 2(b), the entire image is divided into 2N/n sub-regions data.

S4-4: Perform the least-squares binary quadratic term fitting for all sub-regions:

Z(x,y)=p ₀₀ +p ₁₀ x+p ₀₁ y+p ₂₀ x ² +p ₁₁ xy+p ₀₂ y ²

In the formula, p ₀₀ , p ₁₀ , p ₀₁ , p ₂₀ , p ₁₁ , p ₀₂ represent the parameter values of the fitting space surface equation, and the corresponding fitting space surface shape can be determined by obtaining the determined parameter values.

As shown in Figure 2(b), the blank data in the white part is fitted with the known data in the black part.

S4-5: Splicing the fitting results of all sub-regions into a complete background phase map unwrap_ori;

S4-6: Use the phase difference height mapping formula to calculate the depth of each point of the measured object, obtain its 3D point cloud data, and complete the 3D surface reconstruction. The specific process is as follows:

It is known that the distance L between the optical center of the RGB industrial camera and the receiving plane, the distance D between the optical center of the RGB industrial camera and the projector, the frequency f of the generated fringes, and the phase difference are expressed as:

According to the principle of triangulation, the mapping relationship between the phase difference and the depth of the measured object can be expressed by the following formula:

In this way, the spatial point cloud data of the object can be obtained, and the three-dimensional surface reconstruction can be completed.

The embodiment of the present invention only needs to collect more than or equal to two images of the object to be measured, and can perform three-dimensional surface reconstruction without separately collecting background images, thus simplifying the operation process and reducing the collection time, and greatly improving the system work efficiency. In addition, since the embodiment of the present invention does not need to collect the background image separately, there is no problem that the position of the background receiving surface moves before and after shooting, which leads to an error in the reconstruction result. There is no need to set a fixed receiving plane, and the flexibility and applicability of the system are greatly improved. . Furthermore, since the reference surface is not strictly flat, the quality of the stripes projected by the projector and the illumination are not strictly uniform and other uncontrollable factors, the actual continuous phase map must not be an ideal spatial plane. The line is divided into several sub-regions and then fitted to improve the fitting accuracy, make the fitted background phase map closer to the real distribution, and reduce the reconstruction error. Finally, considering the effect of crosstalk between the three-color channels, only the sinusoidal fringe information is stored in the R and B channels, which greatly reduces the effect of color crosstalk. At the same time, under the premise of using the four-step phase-shifting technique, the gray-scale fringe image needs to project four images, while using the RB color image only needs to project two images to solve the continuous phase map.

The above-mentioned embodiments are only preferred embodiments of the present invention, and are not intended to limit the scope of implementation of the present invention. Therefore, any changes made according to the shape and principle of the present invention should be included within the protection scope of the present invention.

Claims (4)

1. a three-dimensional surface reconstruction method based on color fringe projection, is characterized in that, comprises the following steps: S1: Build a structured light projection 3D reconstruction system; S2: Measure the distance L between the optical center of the RGB industrial camera and the receiving plane, and the distance D between the optical center of the RGB industrial camera and the projector; S3: Projecting structured light to collect the measured object images I_obj1 and I_obj2; S4: Process the collected images to complete the three-dimensional surface reconstruction; The specific steps of processing the collected image in the step S4 are as follows: S4-1: Use the four-step phase shift method to obtain the wrapped phase map of the measured object, and unwrap it to obtain the continuous phase map unwrap_obj of the measured object; S4-2: In the original measured object image I_obj1 or I_obj2, extract the measured object ROI area and segment its minimum circumscribed rectangular area, and remove the data in this area, so that the measured object in the continuous phase map of the measured object is unwrap_obj. The data in the minimum circumscribed rectangular area is all empty, leaving the remaining background phase information, and the background incomplete phase map unwrap_subori is obtained at this time; S4-3: Divide the background incomplete phase map unwrap_subori into two parts from the middle, and then divide the left and right parts into several sub-regions per N lines respectively; S4-4: Perform the least squares binary quadratic term fitting on all sub-regions respectively; S4-5: splicing the fitting results of all sub-regions into a complete background phase map; S4-6: Use the phase difference height mapping formula to calculate the depth of each point of the measured object, obtain its three-dimensional point cloud data, and complete the three-dimensional surface reconstruction. 2. a kind of three-dimensional surface reconstruction method based on color fringe projection according to claim 1, is characterized in that, the structured light projection three-dimensional reconstruction system that described step S1 builds comprises projector, RGB industrial camera, computer, Optical flat plate and receiving plane; when the system is built, the projector and the camera are respectively fixed on the optical flat plate, and the optical center axes of the two are at a certain angle, and the line connecting the two optical centers is kept parallel to the receiving plane. 3. a kind of three-dimensional surface reconstruction method based on color fringe projection according to claim 1, is characterized in that, the concrete process of described step S3 projecting structured light collection measured object image I_obj1 and I_obj2 is as follows: Four sinusoidal fringe patterns with the same phase difference are generated by the computer, and the four sinusoidal fringe patterns are divided into two groups and input to the R and B channels of the two color images by taking advantage of the advantage of the color image with three RGB channels; two color fringes are obtained. After the structured light pattern is formed, a projector is used to project the two colored striped structured light patterns on the surface of the object to be measured in turn, and an RGB industrial camera is used to sequentially collect images I_obj1 and I_obj2 with striped structured light. 4. a kind of three-dimensional surface reconstruction method based on color fringe projection according to claim 1, is characterized in that, in the described step S4-6, the phase difference height mapping is the measured object phase minus the background phase , combined with the pre-calibrated system parameters L, D and f, through the following formula:

Obtain the depth information of the measured object; in the formula, L is the distance between the optical center of the known RGB industrial camera and the receiving plane; D is the distance between the optical center of the RGB industrial camera and the projector; f is the frequency of generating fringes,

is the difference between the measured object phase and the fitted background phase.

Images