CN111811432A - Three-dimensional imaging system and method - Google Patents

Abstract

The invention discloses a three-dimensional imaging system and method, and relates to the technical field of spectral imaging, in which a PC, a camera, a projector I and a projector II are involved. Trigger control channel and data transmission channel. Projector I and Projector II have different projection light source bands; Projector I can project single-band structured light. Projector II can project structured light in a variety of different wavelength bands. The camera is located between said projector I and projector II. The camera, projector I and projector II can all be adjusted up and down, and their lenses can be adjusted horizontally and vertically. The projection focal points of the camera, projector I and projector II converge at one point. The present invention can project structured light of multiple light source bands, can be applied to three-dimensional imaging of objects of different materials, can avoid the lack and incompleteness of the basic information of object imaging, a single system can realize the information collection of multiple spectral bands, and the fusion in the later stage is simple and efficient. Error is small.

Description

A three-dimensional imaging system and method

technical field

The present invention relates to the technical field of spectral imaging, in particular to a multi-spectral structured light three-dimensional imaging system and method.

Background technique

Multispectral imaging technology is a remote sensing technology that appeared in the early 1960s. The selection of the band range and the number of bands is directly related to the application target. The simultaneous acquisition of radiation information and spectral information can improve the comprehensive detection perception and identification of target characteristics, and greatly expand the target discrimination and monitoring capabilities of remote sensing detection technology.

Three-dimensional imaging is a technology that extracts three-dimensional information of an object by optical means, and can completely restore the three-dimensional characteristics of the object during the reconstruction process. How to obtain the 3D information of a scene quickly and well is the key of 3D imaging technology. Optical 3D imaging technology is divided into two categories: passive and active. The difference between the two is whether the light source is used for illumination. Passive 3D imaging uses unstructured illumination to obtain 2D images in different visual directions from one or more camera systems. The three-dimensional surface shape of the object is reconstructed through computer matching and operation. This method requires a lot of calculation for images without obvious features, and the matching accuracy cannot be guaranteed. To completely and accurately restore the object structure, multiple two-dimensional images are required. The image is reversed. Based on active 3D imaging, it is necessary to actively project structured light onto the measured object, and determine the 3D information of the measured object through the deformation of the structured light (or time of flight, etc.). Due to its high efficiency and high precision, it is currently the main 3D imaging method. technology.

The existing active structured light 3D imaging system can usually only project a single light source band structured light, which has the following problems:

1) Due to the difference in the material of the object, the absorption of different light source bands is different, and a single light source will greatly limit the material of the object to be measured.

2) A single light source causes the missing and incomplete imaging information of the object.

3) It requires the combination of multiple systems to complete the information collection of multiple spectral bands, and the later fusion is complex and the error is large.

SUMMARY OF THE INVENTION

The present invention is to provide a three-dimensional imaging system and method, which can alleviate the above problems.

In order to alleviate the above-mentioned problems, the technical scheme adopted by the present invention is as follows:

In a first aspect, the present invention provides a three-dimensional imaging system, including a PC, a camera, a projector I and a projector II, and a trigger control channel is set between the PC and the camera, the projector I, and the projector II and data transmission channels;

The projection light source bands of the projector I and the projector II are different;

The projector I can project single-band structured light;

The projector II can project structured light in various wavelength bands;

the camera is located between the projector I and the projector II;

The camera, projector I and projector II can all be adjusted up and down, and their lenses can be adjusted horizontally and vertically;

The projection focal points of the camera, projector I and projector II converge at one point.

The technical effects of this technical solution are: it can project multiple light source band structured lights, can be applied to the three-dimensional imaging of a variety of objects under test with different materials, can avoid the lack and incompleteness of the basic information of object imaging, and a single system can achieve multiple The information collection of each spectral band, the later fusion is simple and the error is small.

Further, the camera, the projector I and the projector II are respectively assembled on three gimbals, and they can be adjusted up and down through the gimbals, and the horizontal and up and down orientations of the lenses can be adjusted. Electromechanical connection.

The technical effect of the technical solution is that the gimbal, as an existing mature device, is easy to obtain, and can meet the posture and position adjustment requirements of the camera, projector I and projector II.

Further, the camera uses a 6-Pin circular connector as an IO interface, which includes a trigger control port and a data input port.

The technical effect of the technical solution is that the structure is flexible, and the range and area of the imaging field of view can be dynamically adjusted according to the characteristics of the equipment or the needs of the measured object, without the need to redesign the structure.

In a second aspect, the present invention provides a three-dimensional imaging method using the above-mentioned three-dimensional imaging system, the method comprising:

S1. Use a PC to generate structured light stripes and upload them to the projector I and projector II;

S2. Carry out system calibration in the public field of view of the camera, projector I and projector II where the measured object is placed, and obtain the relative calibration parameters between the camera, projector I and projector II through the PC;

S3. Adjust the exposure brightness of the camera according to the ambient light and the projected light intensity of the projector on the measured object;

S4. Projector II is in standby, projector I projects and scans the single-band structured light fringes to the measured object, the camera captures the measured object, obtains a set of object deformation fringe images, and uploads them to the PC;

S5. The projector I is in standby, and the projector II projects and scans N structured light stripes of different wavelength bands to the measured object in turn. During this process, the camera captures the measured object and obtains structured light stripes corresponding to N different wavelength bands respectively. N groups of object deformation fringe images, and upload to PC;

S6, 3D scanning, the PC calculates the N+1 groups of object deformation fringe images obtained by snapping according to the calibration parameters, and obtains N+1 groups of object 3D scanning data;

S7. Data reconstruction. According to the calibration parameters, the three-dimensional scanning data of N+1 groups of objects are fused to obtain three-dimensional data of unified coordinates, and the three-dimensional data is filtered to obtain the final three-dimensional point cloud data of the measured object, and the three-dimensional data of the measured object is completed. imaging.

The technical effect of this technical solution is: by projecting multiple light source band structured lights onto the measured object, and then capturing the image of the object, it can be applied to the three-dimensional imaging of the measured object of various materials, and can avoid the lack of basic information of the object imaging. and incomplete, the whole operation method is simple and very convenient to use; all calibrations between the camera, the projector and the projector and the camera can be completed at one time, and the N+1 sets of measurement data can be directly fused by using the calibration parameters, without the need for feature or marker fusion. , the calculation is simple and accurate.

Further, the projector II can project structured light with four different wavelength bands.

The technical effect of the technical solution is that it can cover the common visible light projection band.

Further, the projector I is an infrared projector, and the projector II is an RGB projector.

The technical effect of the technical solution is that it can cover the reflection band range of common materials of the object to be measured.

Further, in the step S1, according to the three-dimensional imaging task conditions, the structured light fringes are generated based on a time phase and a phase shift phase structured light algorithm.

Further, in the step S2, a nine-point calibration method is used to obtain the calibration parameters.

The technical effects of this technical solution are: the nine-point calibration method utilizes a two-dimensional translation stage, which can realize horizontal translation and 360-degree rotation; the calibration plate is fixed on the two-dimensional translation stage, moves horizontally in three positions, and rotates three positions at each position. Make the plane of the calibration plate and the center line of the camera's field of view at 90°, 90°-α and 90°+α, where α is less than 30°, so as to ensure the imaging quality of the calibration plate and the accuracy of calibration; collect nine positions After calibrating the plate image, the calibration parameters of the camera and projector can be calculated by using the known calibration plate data; the calibration plate can use calibration points or checkerboards.

Further, in the step S3, when adjusting the exposure brightness of the camera, the camera can clearly obtain the image of the object to be measured.

Further, in the step S6, for each group of object deformation fringe patterns, first obtain its truncated phase based on time and phase shift phase, then obtain the unwrapped continuous phase according to the truncated phase, and finally use the calibration parameters and the unwrapped continuous phase to calculate and obtain the object. 3D scan data.

The technical effect of the technical solution is that the commonly used active structured light algorithm can be applied, and the complete surface of the measured object with high precision can be quickly and effectively obtained.

In order to make the above-mentioned objects, features and advantages of the present invention more clearly understood, the following specific embodiments of the present invention are given and described in detail in conjunction with the accompanying drawings.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings used in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore do not It should be regarded as a limitation of the scope, and for those of ordinary skill in the art, other related drawings can also be obtained according to these drawings without any creative effort.

1 is a schematic block diagram of the structure of a three-dimensional imaging system according to an embodiment of the present invention;

2 is a schematic diagram of the position point of the calibration plate when the nine-point calibration method is used for system calibration according to an embodiment of the present invention;

3 is a schematic diagram of the layout of a camera and a projector according to an embodiment of the present invention;

4 is a schematic diagram of trigger control according to an embodiment of the present invention;

5 is a flowchart of a three-dimensional imaging method according to an embodiment of the present invention;

FIG. 6 is a flow chart of phase calculation and unwrapping by the difference frequency method according to an embodiment of the present invention.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, but not all embodiments. The components of the embodiments of the invention generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations.

Thus, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but is merely representative of selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Referring to FIG. 1, an embodiment of the present invention provides a three-dimensional imaging system, including a PC, a camera, a projector I and a projector II, and a trigger control channel and a data transmission channel.

Among them, the trigger controller is used to connect between the PC and the camera, projector I and projector II, and is used to form a trigger control channel.

Among them, projector I and projector II have different projection light source bands; projector I can project single-band structured light; projector II can project structured light of multiple different bands; the camera is located between projector I and projector II ; The camera, projector I and projector II can all be adjusted up and down, and their lenses can be adjusted horizontally and vertically; the projection focus of the camera, projector I and projector II converges at one point.

In this embodiment, the camera, projector I and projector II are respectively assembled on three gimbals, and they are adjusted up and down through the gimbals, as well as the horizontal and up and down orientations of the lenses. The gimbals are electrically connected to the PC. The attitude and position adjustment signals can be sent to the PTZ through the PC to adjust the attitude and up and down positions of the camera, projector I and projector II.

In this embodiment, in order to synchronize the projector with the displayed pattern, both Projector II and Projector I support a set of trigger inputs and outputs, which are accessed through the "Trigger Mode" section and the "Trigger Control" sub-tab to configure. The camera uses a 6-Pin circular connector as an IO interface, including a trigger control port for forming a trigger control channel and a data input port for forming a data transmission channel.

When the system is connected, connect the trigger input pin of the camera to the trigger output interface of the projector II, and set the trigger output of the projector II to The required logic voltage level is connected to the trigger output interface of the trigger output port of the projector I.

LightCrafter4500，用于向被测物体投影红色、蓝色、绿色和白色图像；相机所选的型号为M3S501M-H，用于抓拍采集被测物体图像。In this embodiment, projector II is selected to use equipment capable of projecting structured light of four different wavelength bands; projector I is an infrared projector, and projector II is an RGB projector. The model selected for projector I is PDC03, which is used to project infrared images to the object to be measured; the model selected for projector II is

LightCrafter4500 is used to project red, blue, green and white images to the measured object; the selected model of the camera is M3S501M-H, used to capture the measured object image.

3 is a schematic diagram of the layout of the camera and the projector, wherein the camera is located in the middle of the two projectors, the projector II is located on the left side of the camera, and the projector I is located on the right side of the camera. The lens of projector II, the camera lens, and the lens of projector I are at the same height in the z direction. The distance between the center of the camera lens and the center of the lens of projector II is d1, and the distance between the center of the camera lens and the center of the lens of projector I is also d2, usually d1=d2. The angle between the optical axis of the camera lens and the optical axis of the projector II lens is α, and the angle between the optical axis of the camera lens and the optical axis of the projector I lens is β. Usually, for system symmetry, α=β.

Due to the difference in the projection direction of the projector, there are front projection (the projection direction is consistent with the optical axis of its lens), down projection (the projection direction is at an angle with the optical axis of its lens, that is, the projector projects upward at a certain angle) and down projection (projection The direction is at an angle to the optical axis of the lens, that is, the projector projects downward at a certain angle). Therefore, in order to ensure the convenience of the system, a gimbal is designed, and their posture and position are adjusted through the gimbal. The pan-tilt projector can be adjusted to any angle as needed. At the same time, in order to ensure the imaging range of the camera, the pan-tilt of the camera can be raised and lowered, and the level of the camera can be adjusted as needed to ensure the proper imaging range.

In this embodiment, for the projector I and the camera, the optical axes of both are consistent with the projection or imaging direction. When the projector I and the camera are placed horizontally, their optical axis is also in the horizontal direction. The projection direction of the projector II is at an angle with the optical axis of the lens, that is, the projector II projects upward θ. In order to ensure that the center of the projected image of the projector II and the center of the camera image are in the same plane and intersect at the measurement distance D, the projector II must be projected downward by θ, that is, the angle with the xoz plane is θ.

Referring to FIGS. 1 to 4 , an embodiment of the present invention further provides a three-dimensional imaging method, using the above-mentioned three-dimensional imaging system, and the method includes:

1) Use a PC to generate structured light stripes and upload them to Projector I and Projector II.

In this embodiment, the PC implements the calculation of the structured light based on the temporal phase and the phase-shift phase structured light algorithm according to the three-dimensional imaging task conditions, and generates structured light fringes.

The projector I and the projector II are powered on and connected to the PC through a USB data cable to form a data transmission channel, and the PC uploads the generated structured light stripes to the projector through this channel and stores them.

2), system calibration

The system is calibrated in the public field of view of the camera, projector I and projector II where the measured object is placed, and the relative calibration parameters between the camera, projector I and projector II are obtained through the PC.

As shown in FIG. 2 , in the embodiment of the present invention, a nine-point calibration method is used for system calibration to obtain calibration parameters.

When placing the calibration board, it is necessary to ensure that the entire calibration board is within the common field of view of the camera and the projector, and the calibration board is placed in 9 different positions within the field of view and scanned. The PC sends a trigger control command to the camera and the projector through the trigger control channel to complete the scanning of the calibration board.

After scanning the 9 positions, calculate the calibration parameters and complete the calibration. If the calibration fails, re-execute the calibration process. In Figure 2, c1 is the calibrated minimum imaging distance, and c2 is the calibrated maximum imaging distance. In three-dimensional imaging, the measured object must be between c1 and c2 to obtain a clear image.

3), projection capture

Firstly, the exposure brightness of the camera needs to be adjusted according to the ambient light and the light intensity projected by the projector on the measured object.

When adjusting the exposure brightness of the camera, the camera can clearly obtain the image of the object to be measured. When the projector is in video mode, the camera screen can clearly see the object to be measured, and the adjustment is completed when there is not too much overexposure. , otherwise you can adjust the exposure time to change the screen brightness.

In this embodiment, adaptive exposure brightness adjustment is adopted, the camera captures the test image, and the computer brightness automatically adjusts the exposure brightness according to the brightness distribution interval to make it within a suitable brightness range.

Projection scanning is performed after the camera exposure brightness adjustment has been completed.

First, projector II is on standby, projector I projects and scans infrared structured light fringes on the object to be measured, and the camera captures the object to be measured, obtains a set of object deformation fringe images, and uploads them to the PC;

Then, the projector I is in standby, and the projector II projects and scans N=4 structured light stripes with different wavelength bands to the object to be measured in turn, namely red structured light, blue structured light, green structured light and white structured light. During this process In , the camera captures the measured object, obtains 4 groups of object deformation fringe images corresponding to the structured light fringes of 4 different wavelength bands, and uploads them to the PC.

A total of N+1=5 groups of object deformation fringe images were obtained by the PC.

In this embodiment, the triggering principle of projection capture is shown in FIG. 4 . When the system is working, the projector I and the camera constitute the measurement system T1, and the projector II and the camera constitute the measurement system T2. The camera in the measurement system T1 is in the state of receiving the trigger signal, the trigger signal generated by the projector I is transmitted to the camera, and the camera is triggered to take pictures after receiving the trigger signal. The working process of the measurement system T2 is the same as that of T1. When the system is working, first measure the scanning performed by the system T1 on the object, at this time the projector II is in a waiting state and does not project any image. After the measurement system T1 completes the scanning, the projector I is placed in a waiting state, the image projection is turned off, and the measurement system T2 is activated to generate red, blue, green and white structured light respectively to scan the object.

4), 3D scanning

For each group of object deformation fringe patterns, the PC first obtains its truncated phase based on time and phase shift phase, then obtains the unwrapped continuous phase according to the truncated phase, and finally uses the calibration parameters and the unwrapped continuous phase to calculate the three-dimensional scanning data of the object.

Finally, a total of 5 sets of 3D scanning data of objects are calculated.

In this embodiment, the three-dimensional scanning algorithm of multi-frequency heterodyne is used to calculate:

The principle of heterodyne is to use optical modulation technology and electronic phase meter to measure the phase of the modulated optical signal, and to superpose the phase functions φ ₁ and φ ₂ with wavelengths of λ ₁ and λ ₂ (λ ₂ <λ ₁ ) to obtain a new The phase function φ, whose wavelength is λ. Then there are:

Formula (1) shows that the low-frequency phase function can be obtained by heterodyne of the high-frequency phase function, and then the time phase unwrapping can be realized by using the relationship between the low-frequency phase display and the high-frequency function. Knowing φ ₁ and φ ₂ , the heterodyne low-frequency phase function can be calculated by formula (2).

其中λ₃<λ₂<λ₁。构造光栅，使得f₄＝f₂-f₁＝1，

则利用时间相位展开可以实现对高频相位的展开，流程如图6所示。Taking the three-frequency time-phase unwrapping as an example, set the frequency division of the projected fringes as

where λ ₃ <λ ₂ <λ ₁ . Construct the grating such that f ₄ =f ₂ -f ₁ =1,

Then, the high-frequency phase can be unwrapped by using the time phase unwrapping, and the process is shown in Figure 6.

Finally, using the geometric model parameters obtained by calibration, the height information is obtained from the continuous phase inverse calculation.

S7, data reconstruction

According to the calibration parameters, 5 groups of 3D scanning data of objects are fused to obtain 3D data with unified coordinates, and the 3D data is filtered to obtain the final 3D point cloud data of the measured object, and the 3D imaging of the measured object is completed.

In this embodiment, the data reconstruction method based on the Poisson equation can be used to achieve local conformality and global optimization. The process is: calculate the boundary and normal vectors of each set of scan data, and solve the Poisson equation, so that the model indicates that the gradient of the function is equal to the surface normal field integral.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims (10)

1. A three-dimensional imaging system is characterized by comprising a PC (personal computer), a camera, a projector I and a projector II, wherein a trigger control channel and a data transmission channel are arranged between the PC and the camera as well as between the projector I and the projector II;

the projection light source wave bands of the projector I and the projector II are different;

the projector I can project structured light of a single waveband;

the projector II can project a plurality of structured lights with different wave bands;

the camera is positioned between the projector I and the projector II;

the camera, the projector I and the projector II can be adjusted in a lifting mode, and the horizontal direction and the vertical direction of lenses of the camera, the projector I and the projector II can be adjusted;

the projection focuses of the camera, the projector I and the projector II converge at one point.

2. The system of claim 1, wherein said camera, projector i and projector ii are mounted on three respective holders that allow for elevation adjustment by the holders, and adjustment of the horizontal and up and down orientation of the lens, said holders being electrically and mechanically connected to said PC.

3. The system of claim 1, wherein the camera uses a 6-Pin circular connector as an IO interface, including a trigger control port and a data input port.

4. A three-dimensional imaging method, characterized in that the three-dimensional imaging system of any one of claims 1 to 3 is used, the method comprising:

s1, generating structural light stripes by using a PC (personal computer) and uploading the structural light stripes to the projector I and the projector II;

s2, carrying out system calibration in the common vision field of the camera, the projector I and the projector II, where the object to be measured is placed, and acquiring relative calibration parameters among the camera, the projector I and the projector II through a PC;

s3, adjusting the exposure brightness of the camera according to the ambient light and the projection light intensity of the projector on the measured object;

s4, standby the projector II, projecting and scanning the single-waveband structured light stripe to the object to be measured by the projector I, capturing the object to be measured by the camera to obtain a group of object deformation stripe images, and uploading the stripe images to the PC;

s5, the projector I is standby, the projector II sequentially projects and scans N kinds of structural light stripes with different wave bands to the measured object, in the process, the measured object is captured by the camera, N groups of object deformation stripe graphs corresponding to the N kinds of structural light stripes with different wave bands are obtained and uploaded to the PC;

s6, three-dimensional scanning, wherein the PC respectively calculates the N +1 groups of object deformation stripe patterns obtained by snapshot according to the calibration parameters to obtain N +1 groups of object three-dimensional scanning data;

and S7, reconstructing data, fusing the N +1 groups of object three-dimensional scanning data according to the calibration parameters to obtain three-dimensional data with unified coordinates, filtering the three-dimensional data to obtain final three-dimensional point cloud data of the measured object, and finishing three-dimensional imaging of the measured object.

5. The method of claim 4, wherein projector II is capable of projecting four different wavelength bands of structured light.

6. The method of claim 5, wherein projector I is an infrared projector and projector II is an RGB projector.

7. The method according to claim 4, wherein in step S1, the structured light stripe is generated based on a time-phase and phase-shift phase structured light algorithm according to three-dimensional imaging task conditions.

8. The method as claimed in claim 4, wherein in step S2, the calibration parameters are obtained by a nine-point calibration method.

9. The method according to claim 4, wherein in step S3, the camera is allowed to clearly acquire the image of the object to be measured when adjusting the exposure brightness of the camera.

10. The method according to claim 4, wherein in step S6, for each set of deformed fringe patterns of the object, the truncated phase is first obtained based on time and phase shift phase, then the unwrapped continuous phase is obtained based on the truncated phase, and finally the three-dimensional scan data of the object is obtained by using the calibration parameters and the unwrapped continuous phase.

Images