CN102831642B - System and method for object surface reconstruction - Google Patents
System and method for object surface reconstruction Download PDFInfo
- Publication number
- CN102831642B CN102831642B CN201210332221.XA CN201210332221A CN102831642B CN 102831642 B CN102831642 B CN 102831642B CN 201210332221 A CN201210332221 A CN 201210332221A CN 102831642 B CN102831642 B CN 102831642B
- Authority
- CN
- China
- Prior art keywords
- image
- structured light
- imageing sensor
- target area
- specular
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000012545 processing Methods 0.000 claims abstract description 33
- 238000006073 displacement reaction Methods 0.000 claims description 7
- 230000001360 synchronised effect Effects 0.000 abstract description 3
- 238000005286 illumination Methods 0.000 abstract 2
- 238000013459 approach Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 239000011159 matrix material Substances 0.000 description 6
- 230000008520 organization Effects 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 3
- 241000143437 Aciculosporium take Species 0.000 description 2
- XEEYBQQBJWHFJM-UHFFFAOYSA-N Iron Chemical compound [Fe] XEEYBQQBJWHFJM-UHFFFAOYSA-N 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 2
- 239000013256 coordination polymer Substances 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000005484 gravity Effects 0.000 description 2
- 238000010191 image analysis Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000004806 packaging method and process Methods 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- ZOKXTWBITQBERF-UHFFFAOYSA-N Molybdenum Chemical compound [Mo] ZOKXTWBITQBERF-UHFFFAOYSA-N 0.000 description 1
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 239000004411 aluminium Substances 0.000 description 1
- XAGFODPZIPBFFR-UHFFFAOYSA-N aluminium Chemical compound [Al] XAGFODPZIPBFFR-UHFFFAOYSA-N 0.000 description 1
- 229910052782 aluminium Inorganic materials 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 229910052742 iron Inorganic materials 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 229910052750 molybdenum Inorganic materials 0.000 description 1
- 239000011733 molybdenum Substances 0.000 description 1
- 238000000059 patterning Methods 0.000 description 1
- 238000004064 recycling Methods 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
- 239000002210 silicon-based material Substances 0.000 description 1
- WFKWXMTUELFFGS-UHFFFAOYSA-N tungsten Chemical compound [W] WFKWXMTUELFFGS-UHFFFAOYSA-N 0.000 description 1
- 229910052721 tungsten Inorganic materials 0.000 description 1
- 239000010937 tungsten Substances 0.000 description 1
Images
Landscapes
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention relates to a system and a method for object surface reconstruction. The system comprises an illumination component, an image sensor, a controller and a data processing unit, wherein the illumination component comprises a light source and a mirror surface reflecting device, and is used for forming a linear array structural light pattern in a target area; the image sensor is used for gradual exposure to shoot a projective image of the structural light pattern of at least a part of area in the target area; the controller is used for controlling an exposure region of the image sensor to be synchronized with a structural light projective region of the mirror surface reflecting device; and the data processing unit is used for receiving and obtaining the projective image and a shift value of the structural light pattern in a pre-shot standard image in real time, and is used for calculating three-dimensional cloud data of the object in the target area. By the adoption of the system and the method, a preset structural light pattern is projected in the target area according to the requirement, the synchronized exposure of the image sensor is controlled to shoot the projective image of at least a part of area in the target area, the projective image is compared with the standard image, the three-dimensional depth data of the object in the target area is calculated, and thus real-time moving data of a moving object is obtained.
Description
The application be that January 27, application number in 2011 are 201110029945.2 the applying date, name is called the dividing an application of application for a patent for invention of " a kind of system and method for reconstructing surface of object ".
Technical field
The present invention relates to three-dimensional image reconstruction technique, more particularly, relate to a kind of system and method for reconstructing surface of object.
Background technology
Cloud data is to utilize 3-D scanning equipment to carry out Direct Sampling to body surface and the point that obtains, and these points are that three-dimensional model surface is initial, are also the most directly to represent.By analyzing cloud data, can rebuild the geometric jacquard patterning unit surface of model.
The method that object point cloud obtains is a lot, and common method utilizes structure light image information to obtain cloud data, also has the binocular of employing visible light image information to obtain data.The pattern light that the existing method based on speckle projects carries out the method for three-dimensional reconstruction, and its speckle pattern is fixing in its pattern maker not to be changed, due to speckle, higher to the equipment requirement of imageing sensor, and the dirigibility of equipment is poor.
Summary of the invention
The technical problem to be solved in the present invention is, thereby the poor defect of the constant equipment requirement to imageing sensor of the pattern photofixation projecting for the existing method based on speckle dirigibility higher and equipment, provide a kind of and can control as required specular-reflection unit, thereby generate the structured light patterns be suitable for measuring object, and the system and method for synchro control imageing sensor reconstructing surface of object that required region is scanned.
The technical solution adopted for the present invention to solve the technical problems is: construct a kind of system of reconstructing surface of object, it is characterized in that, comprising:
Light fixture, comprises light source and specular-reflection unit, and described specular-reflection unit is adjusted minute surface attitude for passing through, and the light reflection that described light source is sent forms the structured light patterns of linear array shape to target area;
Imageing sensor, for progressively exposing to take the projected image of the linear array shape pattern at least part of region in described target area;
Controller, synchronizes with the view field of the structured light of described specular-reflection unit reflection for controlling the exposure area of described imageing sensor;
Data processing unit, be connected with described imageing sensor, receive in real time the projected image of described structured light patterns, and obtain the structured light patterns shift value in the benchmark image of described projected image and shooting in advance, calculate the three dimensional point cloud of object in described target area.
Preferably, when described light source adopts pointolite, described specular-reflection unit adopts 2-D vibration mirror; When described light source adopts line source, described specular-reflection unit adopts one dimension galvanometer.
Preferably, described one dimension galvanometer or 2-D vibration mirror adopt MEMS galvanometer.
Preferably, described data processing unit calculates first respectively the coordinate array of every lines in described projected image and described benchmark image linear array; Then, utilize lines number of coordinates batch total to calculate the displacement of lines; Finally, by following formula computation structure light pattern shift value, be Δ X
nthe distance z of impact point;
Wherein, B is the distance at the center of imageing sensor and galvanometer device, the equipment lens focus that f is described imageing sensor, and L is the distance of the standard flat at the benchmark image place chosen.
Preferably, described data processing unit is in structured light line thickness direction, and the barycentric coordinates of calculating maximum gradation value pixel are lines coordinate; Or described data processing unit, in structured light line thickness direction, directly calculates the centre coordinate of live width as the coordinate of lines.
A kind of method that the invention also discloses reconstructing surface of object, is characterized in that, comprises the following steps:
S1, by controlling specular-reflection unit, the light reflection that light source is sent is to target area and form the structured light patterns of linear array shape;
S2, utilize imageing sensor progressively to expose to take in described target area the projected image of the structured light patterns in region at least partly, and the view field that controls the structured light exposure area of described imageing sensor reflecting with described specular-reflection unit synchronize;
S3, receive in real time the projected image of described structured light patterns, and obtain described projected image and the benchmark image taken in advance in shift value, calculate the three dimensional point cloud of object in described target area.
Preferably, when light source described in described step S1 adopts pointolite, described specular-reflection unit adopts 2-D vibration mirror; When described light source adopts line source, described specular-reflection unit adopts one dimension galvanometer.
Preferably, the one dimension galvanometer adopting in described step S1 or 2-D vibration mirror are MEMS galvanometer.
Preferably, wherein, described step S3 further comprises:
S31, calculate respectively the coordinate array of every lines in described projected image and described benchmark image linear array;
S32, utilize lines number of coordinates batch total to calculate the displacement of lines;
S33, by following formula computation structure light pattern shift value, be Δ X
nthe distance z of impact point;
Wherein, B is the distance at the center of imageing sensor and galvanometer device, the equipment lens focus that f is described imageing sensor, and L is the distance of the standard flat at the benchmark image place chosen.
Preferably, described step S31 comprises:
In structured light line thickness direction, the barycentric coordinates of calculating maximum gradation value pixel are lines coordinate; Or
In structured light line thickness direction, directly calculate the centre coordinate of live width as lines coordinate.
Implement the system and method for reconstructing surface of object of the present invention, there is following beneficial effect: the present invention can project default structured light patterns by controlling specular-reflection unit as required, and the projected image at least part of region in target area is obtained in the synchronous exposure of control chart image-position sensor, compare with benchmark image the range data that utilizes principle of triangulation to calculate object in target area, this range data can real-time update, thereby obtains the real time kinematics data of moving object.
Accompanying drawing explanation
Below in conjunction with drawings and Examples, the invention will be further described, in accompanying drawing:
Fig. 1 is the index path of the system of reconstructing surface of object according to the preferred embodiment of the invention;
Fig. 2 is the schematic diagram of the benchmark image taken of the system of reconstructing surface of object according to the preferred embodiment of the invention;
Fig. 3 is the schematic diagram of the projected image taken of the system of reconstructing surface of object according to the preferred embodiment of the invention;
Fig. 4 is the method flow diagram of reconstructing surface of object according to the preferred embodiment of the invention.
Embodiment
Below in conjunction with accompanying drawing and by embodiment, further illustrate technical scheme of the present invention.Be understandable that, specific embodiment described herein is only for explaining the present invention, but not limitation of the invention.It also should be noted that, for convenience of description, in accompanying drawing, only show step related to the present invention but not all processes.
Referring to Fig. 1, is the index path of the system of reconstructing surface of object according to the preferred embodiment of the invention.The system of the reconstructing surface of object that as shown in Figure 1, this embodiment provides mainly comprises light fixture 10, imageing sensor 20, data processing unit 30 and controller (not shown).
Specular-reflection unit 12 can be controlled its minute surface attitude by controller, and the light reflection that described light source 11 is sent forms default structured light patterns to target area.Preferably, specular-reflection unit can adopt one dimension galvanometer or 2-D vibration mirror, and the corresponding structured light patterns generating is respectively linear array shape pattern or lattice-like pattern.When light source adopts pointolite, specular-reflection unit can adopt 2-D vibration mirror, and the structured light patterns of generation is linear array shape pattern or lattice-like pattern.When light source adopts line source, specular-reflection unit can adopt one dimension galvanometer, and the pattern of generation is linear array shape pattern.In addition, when light source adopts pointolite, can also first by optical element, pointolite be converted to linear light, then generate linear array shape pattern by one dimension galvanometer.One dimension galvanometer or 2-D vibration mirror also can adopt MEMS(Micro-Electro-Mechanical Systems, MEMS (micro electro mechanical system)) galvanometer, rotary shaft galvanometer etc. can be realized the device of controlling minute surface motion.MEMS is a kind of brand-new research and development field that must simultaneously consider multiple physical field immixture, and with respect to traditional machinery, their size is less, and maximum is no more than one centimetre, is even only several microns, and its thickness is just more small.Employing be take silicon as main material, excellent electrical properties, and the intensity of silicon materials, hardness and Young modulus and iron are suitable, and density and aluminium are similar, and pyroconductivity approaches molybdenum and tungsten.Adopt and the similar generation technique of integrated circuit, can utilize in a large number mature technology, technique in IC production, carry out in enormous quantities, low-cost production, cost performance is increased substantially with respect to tradition " machinery " manufacturing technology.The attitude of the minute surface 121 in the specular-reflection unit that one dimension galvanometer or 2-D vibration mirror form can be controlled by controller, by adjusting the attitude of minute surface 121, light source 11 is irradiated to light reflection on minute surface 121 to target area, and form the pattern of project organization light in target area projection. for example 2-D vibration mirror projects lattice-like pattern, and one dimension galvanometer is linear array shape pattern.
Imageing sensor 20 is for progressively exposing with the projected image of the structured light patterns at least part of region in photographic subjects region.Because system of the present invention needs to gather the benchmark image of primary structure light before using first.In native system the relative position and attitude of imageing sensor 20, specular-reflection unit 12 and light source 11 need to when taking benchmark image, remain unchanged, comprise skew and rotation here.If variation has appearred in relative position, just need Resurvey benchmark image.Unique distinction of the present invention is to be synchronizeed with the view field of the structured light of specular-reflection unit 12 reflections by the exposure area of controller control chart image-position sensor 20.That is to say, when linear structured light is progressively projected on target area, also simultaneously the expose region of this linear smooth projection of imageing sensor 12, therefore imageing sensor 12 can be only in target area region at least partly, the specific region acquired projections image at the target object place of for example pre-estimating, and do not need to gather the image in all target areas.And the pattern that can the relatively low imageing sensor of option and installment gathers linear light.
Below the positional structure of all parts in the system of reconstructing surface of object of the present invention is described.Set up three-dimensional system of coordinate as shown in Figure 1, in this coordinate system, each device is installed.In Fig. 1, coordinate system meets the right-hand rule.
A. the X-direction of coordinate system is the line of the galvanometer minute surface center P of original state as shown in the figure and the camera lens photocentre C of imageing sensor, and positive dirction is CP, as made progress in Fig. 1 as X-axis positive dirction, and the distance that wherein B is PC.
B.Z direction of principal axis is through minute surface center P, and perpendicular to minute surface, and positive dirction is to point to target area by minute surface center P, as being to the right Z axis positive dirction in Fig. 1.
C.Y direction of principal axis is to determine according to the right-hand rule, in Fig. 1, be vertical paper inwards.
E. the photocentre axis of imageing sensor 20 is set to parallelly with Z axis, and guarantees that the visual field of imageing sensor and projection target region exist common factor, and the lap of the angular field of view of emergent ray slewing area and imageing sensor camera lens is measured zone.And the equipment lens focus of imageing sensor is f.
If f. the specular-reflection unit of this system adopts one dimension galvanometer, minute surface 121 can be around XP axle or around the rotation of YP axle or vibration; If employing 2-D vibration mirror, minute surface can rotate or vibration simultaneously around diaxon simultaneously.In order to obtain above-mentioned vibration characteristics, can adopt existing MEMS one dimension or 2-D vibration mirror to realize, its emergent ray slewing area covers whole testee.
G. the data of imageing sensor 20, by existing common data transfer mode, are transferred to data processing unit 30.
Below the realization flow of the system of reconstructing surface of object of the present invention and principle are described.
1) according to above-mentioned positional structure, each parts are assembled.Can carry out integration packaging to a certain degree, for example, light source 11 and specular-reflection unit 12 are integrated to formation light fixture 10 simultaneously.
2) design of projection pattern: the galvanometer system adopting due to the present invention can utilize galvanometer posture changing characteristic at a high speed, projects lattice-like, linear array shape etc. image in target area.Here mainly adopt one dimension galvanometer to irradiate linear array shape or adopt 2-D vibration mirror to project dot matrix.
3) gather image information step:
A. take benchmark image.For example, at system of distance L=2000mm place, the 3*3m standard flat T vertical with Z axis is set, the image of choosing this plane T photographs is benchmark image.Although provided the distance of concrete benchmark image in this embodiment, the present invention is not limited to this, this area basic technology personnel can choose suitable plane according to actual conditions and take benchmark image.Utilize imageing sensor 20 gather linear array shape that light fixtures 10 project or dot pattern in the imaging of standard flat as benchmark image, and be kept in the storer 31 of data processing unit 30.
B. the projected image of photographic subjects.Behind object target approach region, projection pattern is irradiated on body surface, now utilizes imageing sensor 20 to gather the projection pattern of target area, and for example linear array shape pattern, sends to data processing unit 30;
4) surface point cloud data reconstruction step:
Refer to Fig. 2 and Fig. 3, be respectively the benchmark image of the system shooting of reconstructing surface of object according to the preferred embodiment of the invention and the schematic diagram of projected image.As shown in Figure 2, this structured light patterns is linear array shape, and the height of line is H, and width is W.It is corresponding that can to calculate the line-spacing between n bar line in linear array be W
n.In Fig. 3, can see that the lines in linear array in target object region will be offset when target object enters picture, side-play amount is S.All projective patterns are along with the increase of distance L, and pattern there will be towards fixed-direction and is offset, and side-play amount S diminishes gradually.And the maximal value of skew S must be less than the minimum period W of periodic pattern
n.Side-play amount S can calculate acquisition for projected image and benchmark image analysis by data processing unit.
A. establish the image that the projected image that collects and benchmark image are m*n pixel.Take below projection structured light patterns as linear array light be example, the projected image obtaining is analyzed, adopt the analytical approach of structured light patterns of lattice-like identical with it.
First, utilize image algorithm to calculate respectively in projected image and benchmark image the coordinate array P of every line in linear array
n(i, j) and Q
n(i, j), wherein n represents the numbering of linear array center line, (i, j) is the coordinate of every bit on line.At this, because the grey scale pixel value of structured light lines in image is greater than surrounding pixel gray-scale value, so can tell structured light lines in background.The present invention can adopt the way of Threshold segmentation that structured light lines are split.Due to Threshold, structured light line thickness may be comprised of one or more than one pixel, so can adopt bright pixel coordinate in the center of gravity of live width or live width to carry out the coordinate of representative structure light line.Wherein, in first method, in structured light line thickness direction, calculate the barycentric coordinates of maximum gradation value pixel, as lines coordinate.In second method, in structured light line thickness direction, directly calculate the centre coordinate of live width, as the coordinate of lines.The coordinate that calculates structured light lines by column respectively after the alignment of projected image and benchmark image.Lattice-like pattern if, surpasses the situation of a pixel for the profile of every in dot matrix, calculate the barycentric coordinates of doing weight by gray threshold, as the coordinate of point.
B. utilize lines array of pixels to calculate the displacement Δ X of lines
n(i, j)=P
n(i, j)-Q
n(i, j).Can be by the shift value of two structured light lines alignment coordinates computed by column.The side-play amount S of lines and Δ X in target object region in Fig. 3
n(i, j) is corresponding.
C. utilize the ultimate principle of structural light measurement, data processing unit is Δ X by following formula computation structure light pattern shift value
nthe distance z of impact point;
Wherein, B is the distance at the center of imageing sensor and galvanometer device, the equipment lens focus that f is described imageing sensor, and L is the distance of the standard flat at the benchmark image place chosen.In projection pattern, the distance z of difference is the object dimensional cloud data calculating.
Because B, f in above-mentioned formula and L can pre-set for Bu Tong target object and the distance of size, the therefore particular value of the X ' in above-mentioned formula for setting.When needs are during for different target object project organization light pattern, only need to adjust the mirror status of one dimension or 2-D vibration mirror, just can obtain different structured lights, for example linear array light or dot matrix light, can also adjust the line-spacing of linear array light, thereby be convenient to identify the region at target object place in the projected image gathering.
The present invention also correspondingly provides a kind of method of reconstructing surface of object.Refer to Fig. 4, for according to the method flow diagram of reconstructing surface of object in the preferred embodiment of the present invention.As shown in Figure 4, the method S400 starts from step S402:
Subsequently, in next step S404, by controlling specular-reflection unit, the light reflection that light source is sent is to target area and form default structured light patterns.This step mainly adopts light fixture to realize, and as shown in Figure 1, light fixture 10 comprises light source 11 and specular-reflection unit 12.The light source of this programme preferably adopts semiconductor laser diode.
Specular-reflection unit 12 can be controlled its minute surface attitude by controller, and the light reflection that described light source 11 is sent forms default structured light patterns to target area.Preferably, specular-reflection unit can adopt one dimension galvanometer or 2-D vibration mirror, and the corresponding structured light patterns generating is respectively linear array shape pattern or lattice-like pattern.When light source adopts pointolite, specular-reflection unit can adopt 2-D vibration mirror, and the structured light patterns of generation is linear array shape pattern or lattice-like pattern.When light source adopts line source, specular-reflection unit can adopt one dimension galvanometer, and the pattern of generation is linear array shape pattern.In addition, when light source adopts pointolite, can also first by optical element, pointolite be converted to linear light, then generate linear array shape pattern by one dimension galvanometer.One dimension galvanometer or 2-D vibration mirror also can adopt MEMS(Micro-Electro-Mechanical Systems, MEMS (micro electro mechanical system)) galvanometer, rotary shaft galvanometer etc. can be realized the device of controlling minute surface motion.The attitude of the minute surface 121 in the specular-reflection unit that one dimension galvanometer or 2-D vibration mirror form can be controlled by controller, by adjusting the attitude of minute surface 121, light source 11 is irradiated to light reflection on minute surface 121 to target area, and forms the pattern of project organization light in target area projection.For example 2-D vibration mirror projects lattice-like pattern, and one dimension galvanometer is linear array shape pattern.
Set up three-dimensional system of coordinate as shown in Figure 1, in this coordinate system, each device is installed.In Fig. 1, coordinate system meets the right-hand rule.The X-direction of coordinate system is the line of the galvanometer minute surface center P of original state as shown in the figure and the camera lens photocentre C of imageing sensor, and positive dirction is CP, as made progress in Fig. 1 as X-axis positive dirction, and the distance that wherein B is PC.Z-direction is through minute surface center P, and perpendicular to minute surface, and positive dirction is to point to target area by minute surface center P, as being to the right Z axis positive dirction in Fig. 1.Y direction is to determine according to the right-hand rule, in Fig. 1, be vertical paper inwards.Light source 11 is launched collimation laser, and laser axis is on plane X PZ, and through the center of reflecting surface.The angle of laser axis and Z axis is relevant with target area, is set to 45 degree in this method.If the specular-reflection unit of the method adopts one dimension galvanometer, minute surface 121 can be around XP axle or around the rotation of YP axle or vibration; If employing 2-D vibration mirror, minute surface can rotate or vibration simultaneously around diaxon simultaneously.In order to obtain above-mentioned vibration characteristics, can adopt existing MEMS one dimension or 2-D vibration mirror to realize, its emergent ray slewing area covers whole testee.
Subsequently, in next step S406, utilize imageing sensor progressively to expose to take in described target area the projected image of the structured light patterns in region at least partly, and the view field that controls the structured light exposure area of described imageing sensor reflecting with described specular-reflection unit synchronize.Unique distinction of the present invention is to be synchronizeed with the view field of the structured light of specular-reflection unit 12 reflections by the exposure area of controller control chart image-position sensor 20.That is to say, when linear structured light is progressively projected on target area, also simultaneously the expose region of this linear light projection of imageing sensor 12, therefore imageing sensor 12 can be only in target area region at least partly, the specific region acquired projections image at the target object place of for example pre-estimating, and do not need to gather the image in all target areas.And the pattern that can the relatively low imageing sensor of option and installment gathers linear light.
As shown in Figure 1, because method of the present invention needs to gather the benchmark image of primary structure light before using first, therefore in this step the relative position and attitude of imageing sensor 20, specular-reflection unit 12 and light source 11 need to when taking benchmark image, remain unchanged, comprise skew and rotation here.If variation has appearred in relative position, just need Resurvey benchmark image.The photocentre axis of imageing sensor 20 is set to parallel with Z axis, and guarantees that the visual field of imageing sensor and projection target region exist common factor, and the lap of the angular field of view of emergent ray slewing area and imageing sensor camera lens is measured zone.And the equipment lens focus of imageing sensor is f.The data of imageing sensor 20, by existing common data transfer mode, are transferred to data processing unit 30.
Subsequently, in next step S408, receive in real time the projected image of the structured light patterns that imageing sensor 20 takes, and utilize image algorithm to obtain the shift value in the benchmark image of described projected image and shooting in advance, calculate the three dimensional point cloud of object in described target area.Can adopt data processing unit as shown in Figure 1 to realize this step.Data processing unit 30 is connected with imageing sensor 20, receive the projected image of the structured light patterns of imageing sensor 20 shootings, and in data processing unit 30, store the benchmark image of taking in advance, utilize image algorithm to obtain the shift value of the structured light patterns in described projected image and benchmark image, calculate the three dimensional point cloud of object in described target area.Therefore, data processing unit 30 may further include storer 31 and processor 32, and data processing unit 30 also has input/output device 33.Wherein storer 31 can be used for being stored in the benchmark image information of taking while there is no target object.32 of processors can receive the projected image that imageing sensor is taken by input/output device 33, and from storer 31, obtain said reference image information, carry out above-mentioned calculating and processing, obtain three dimensional point cloud, realize body surface cloud data and rebuild.
Finally, method S400 ends at step S410.
Below the specific operation process of the method for reconstructing surface of object of the present invention is described.
1) according to above-mentioned positional structure, each parts are assembled.Can carry out integration packaging to a certain degree, for example, light source 11 and specular-reflection unit 12 are integrated to formation light fixture 10 simultaneously.
2) design of projection pattern: the galvanometer system adopting due to the inventive method can utilize galvanometer posture changing characteristic at a high speed, projects lattice-like, linear array shape etc. image in target area.Here mainly adopt one dimension galvanometer to irradiate linear array or adopt 2-D vibration mirror to project dot matrix.
3) gather image information step:
A. take benchmark image.For example, at system of distance L=2000mm place, the 3*3m standard flat T vertical with Z axis is set, the image of choosing this plane T photographs is benchmark image.Although provided the distance of concrete benchmark image in this embodiment, the present invention is not limited to this, this area basic technology personnel can choose suitable plane according to actual conditions and take benchmark image.Utilize imageing sensor 20 gather linear array shape that light fixtures 10 project or lattice-like pattern in the imaging of standard flat as benchmark image, and be kept in the storer 31 of data processing unit 30.
B. the projected image of photographic subjects.Behind object target approach region, projection pattern is irradiated on body surface, now utilizes imageing sensor 20 to gather the projection pattern of target area, and for example linear array shape pattern, sends to data processing unit 30;
4) surface point cloud data reconstruction step:
Refer to Fig. 2 and Fig. 3, be respectively the benchmark image of the system shooting of reconstructing surface of object according to the preferred embodiment of the invention and the schematic diagram of projected image.As shown in Figure 2, this structured light patterns is linear array, and the height of line is H, and width is W.It is corresponding that can to calculate the line-spacing between n bar line in linear array be W
n.In Fig. 3, can see that the lines in linear array in target object region will be offset when target object enters picture, side-play amount is S.All projective patterns are along with the increase of distance L, and pattern there will be towards fixed-direction and is offset, and side-play amount S diminishes gradually.And the maximal value of skew S must be less than the minimum period W of periodic pattern
n.Side-play amount S can calculate acquisition for projected image and benchmark image analysis by data processing unit.
A. establish the image that the projected image that collects and benchmark image are m*n pixel.Take below projection structured light patterns as linear array light be example, the projected image obtaining is analyzed, adopt the analytical approach of structured light patterns of lattice-like identical with it.
First, utilize image algorithm to calculate respectively in projected image and benchmark image the coordinate array P of every line in linear array
n(i, j) and Q
n(i, j), wherein n represents the numbering of linear array center line, (i, j) is the coordinate of every bit on line.At this, because the grey scale pixel value of structured light lines in image is greater than surrounding pixel gray-scale value, so can tell structured light lines in background.The present invention can adopt the way of Threshold segmentation that structured light lines are split.Due to Threshold, structured light line thickness may be comprised of one or more than one pixel, so can adopt bright pixel coordinate in the center of gravity of live width or live width to carry out the coordinate of representative structure light line.Wherein, in first method, in structured light line thickness direction, calculate the barycentric coordinates of maximum gradation value pixel, as lines coordinate.In second method, in structured light line thickness direction, directly calculate the centre coordinate of live width, as the coordinate of lines.The coordinate that calculates structured light lines by column respectively after the alignment of projected image and benchmark image.Lattice-like pattern if, surpasses the situation of a pixel for the profile of every in dot matrix, calculate the barycentric coordinates of doing weight by gray threshold, as the coordinate of point.
B. utilize lines array of pixels to calculate the displacement Δ X of lines
n(i, j)=P
n(i, j)-Q
n(i, j).Can be by the shift value of two structured light lines alignment coordinates computed by column.The side-play amount S of lines and Δ X in target object region in Fig. 3
n(i, j) is corresponding.
C. utilize the ultimate principle of structural light measurement, data processing unit is Δ X by following formula computation structure light pattern shift value
nthe distance z of impact point;
Wherein, B is the distance at the center of imageing sensor and galvanometer device, the equipment lens focus that f is described imageing sensor, and L is the distance of the standard flat at the benchmark image place chosen.In projection pattern, the distance z of difference is the object dimensional cloud data calculating.
Because B, f in above-mentioned formula and L can pre-set for Bu Tong target object and the distance of size, the therefore particular value of the X ' in above-mentioned formula for setting.When needs are during for different target object project organization light pattern, only need to adjust the mirror status of one dimension or 2-D vibration mirror, just can obtain different structured lights, for example linear array light or dot matrix light, can also adjust the line-spacing of linear array light, thereby be convenient to identify the region at target object place in the projected image gathering.
In sum, the system and method for reconstructing surface of object of the present invention, the some light or the linear light that first light source are irradiated, be irradiated on specular-reflection unit.By controlling specular-reflection unit rotation and being offset light reflection to target area, form structured light patterns.While utilizing imageing sensor to obtain without object in the internal standard plane of target area the pattern of structured light as reference pattern.Behind object target approach region, synchronizeing with the view field of the structured light of described specular-reflection unit reflection in the exposure area of recycling imageing sensor control chart image-position sensor, obtains the structured light projection pattern at least part of region in target area.Store data into data processing unit, calculate the displacement information of two kinds of patterns, then utilize principle of triangulation to calculate the range data of object.This range data can real-time update, thereby obtains the real time kinematics data of moving object.And adopt galvanometer mode of the present invention, just can be to same set of equipment, according to diverse ways, adjustment projects the pattern of structured light.The method and system can be applied in the real-time reconstruction of object, can use in the field of man-machine interaction, as virtual mouse, the man-machine interaction of body-building, the man-machine interface of game, the detection of mobile device objects in front, human motion analysis, distance estimations and warning, intrusion alarm etc. aspect.
The present invention is described according to specific embodiment, but it will be understood by those skilled in the art that when not departing from the scope of the invention, can carry out various variations and be equal to replacement.In addition,, for adapting to specific occasion or the material of the technology of the present invention, can carry out many modifications and not depart from its protection domain the present invention.Therefore, the present invention is not limited to specific embodiment disclosed herein, and comprises all embodiment that drop into claim protection domain.
Claims (6)
1. a system for reconstructing surface of object, is characterized in that, comprising:
Light fixture, comprise light source and specular-reflection unit, described specular-reflection unit is for passing through to adjust minute surface attitude, the light reflection that described light source is sent forms the structured light patterns of linear array shape to target area, wherein, when described light source adopts pointolite, described specular-reflection unit adopts 2-D vibration mirror; When described light source adopts line source, described specular-reflection unit adopts one dimension galvanometer;
Imageing sensor, for progressively exposing to take the projected image of the linear array shape pattern at least part of region in described target area;
Controller, synchronizes with the view field of the structured light of described specular-reflection unit reflection for controlling the exposure area of described imageing sensor;
Data processing unit, be connected with described imageing sensor, receive in real time the projected image of described structured light patterns, and obtain the structured light patterns shift value in the benchmark image of described projected image and shooting in advance, calculate the three dimensional point cloud of object in described target area;
Wherein, described data processing unit calculates first respectively the coordinate array of every lines in described projected image and described benchmark image linear array; Then, utilize lines number of coordinates batch total to calculate the displacement of lines; Finally, by following formula computation structure light pattern shift value, be △ X
nthe distance z of impact point;
Wherein, B is the distance at the center of imageing sensor and galvanometer device, the equipment lens focus that f is described imageing sensor, and L is the distance of the standard flat at the benchmark image place chosen.
2. the system of reconstructing surface of object according to claim 1, is characterized in that, described one dimension galvanometer or 2-D vibration mirror adopt MEMS galvanometer.
3. the system of reconstructing surface of object according to claim 1, is characterized in that, described data processing unit is in structured light line thickness direction, and the barycentric coordinates of calculating maximum gradation value pixel are lines coordinate; Or described data processing unit, in structured light line thickness direction, directly calculates the centre coordinate of live width as the coordinate of lines.
4. a method for reconstructing surface of object, is characterized in that, comprises the following steps:
S1, by controlling specular-reflection unit, the light reflection that light source is sent is to target area and form the structured light patterns of linear array shape, wherein, when light source described in described step S1 adopts pointolite, described specular-reflection unit employing 2-D vibration mirror; When described light source adopts line source, described specular-reflection unit adopts one dimension galvanometer;
S2, utilize imageing sensor progressively to expose to take in described target area the projected image of the structured light patterns in region at least partly, and the view field that controls the structured light exposure area of described imageing sensor reflecting with described specular-reflection unit synchronize;
S3, receive in real time the projected image of described structured light patterns, and obtain described projected image and the benchmark image taken in advance in shift value, calculate the three dimensional point cloud of object in described target area;
Wherein, described step S3 further comprises:
S31, calculate respectively the coordinate array of every lines in described projected image and described benchmark image linear array;
S32, utilize lines number of coordinates batch total to calculate the displacement of lines;
S33, by following formula computation structure light pattern shift value, be △ X
nthe distance z of impact point;
Wherein, B is the distance at the center of imageing sensor and galvanometer device, the equipment lens focus that f is described imageing sensor, and L is the distance of the standard flat at the benchmark image place chosen.
5. the method for reconstructing surface of object according to claim 4, is characterized in that, the one dimension galvanometer adopting in described step S1 or 2-D vibration mirror are MEMS galvanometer.
6. the method for reconstructing surface of object according to claim 4, is characterized in that, described step S31 comprises:
In structured light line thickness direction, the barycentric coordinates of calculating maximum gradation value pixel are lines coordinate; Or
In structured light line thickness direction, directly calculate the centre coordinate of live width as lines coordinate.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210332221.XA CN102831642B (en) | 2011-01-27 | 2011-01-27 | System and method for object surface reconstruction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210332221.XA CN102831642B (en) | 2011-01-27 | 2011-01-27 | System and method for object surface reconstruction |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011100299452A Division CN102074044B (en) | 2011-01-27 | 2011-01-27 | System and method for reconstructing surface of object |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102831642A CN102831642A (en) | 2012-12-19 |
| CN102831642B true CN102831642B (en) | 2014-04-23 |
Family
ID=47334755
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210332221.XA Active CN102831642B (en) | 2011-01-27 | 2011-01-27 | System and method for object surface reconstruction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102831642B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105865350A (en) * | 2016-04-30 | 2016-08-17 | 广东工业大学 | 3D Object Point Cloud Imaging Method |
| US10527711B2 (en) * | 2017-07-10 | 2020-01-07 | Aurora Flight Sciences Corporation | Laser speckle system and method for an aircraft |
| CN111854642B (en) * | 2020-07-23 | 2021-08-10 | 浙江汉振智能技术有限公司 | Multi-line laser three-dimensional imaging method and system based on random dot matrix |
| CN112433382B (en) * | 2020-11-25 | 2024-01-23 | Oppo（重庆）智能科技有限公司 | Speckle projection device and method, electronic equipment and distance measurement system |
| CN112866596B (en) * | 2021-01-08 | 2023-11-21 | 跨维(深圳)智能数字科技有限公司 | Anti-strong light three-dimensional capturing method and system based on CMOS sensor |
| CN113703248B (en) * | 2021-08-11 | 2022-09-09 | 深圳市睿识科技有限公司 | 3D structured light module and depth map point cloud image acquisition method based on same |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US3828126A (en) * | 1973-04-13 | 1974-08-06 | American Express Invest | Real time interferometry |
| US4794550A (en) * | 1986-10-15 | 1988-12-27 | Eastman Kodak Company | Extended-range moire contouring |
| CN1702491A (en) * | 2005-04-28 | 2005-11-30 | 徐能佳 | Reflecting line-structured light |
| WO2010048960A1 (en) * | 2008-10-28 | 2010-05-06 | 3Shape A/S | Scanner with feedback control |
| CN101871773A (en) * | 2009-04-22 | 2010-10-27 | 陈亮嘉 | Synchronous hue phase shift conversion method and three-dimensional morphology measurement system thereof |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7440590B1 (en) * | 2002-05-21 | 2008-10-21 | University Of Kentucky Research Foundation | System and technique for retrieving depth information about a surface by projecting a composite image of modulated light patterns |
| US7616985B2 (en) * | 2002-07-16 | 2009-11-10 | Xenogen Corporation | Method and apparatus for 3-D imaging of internal light sources |
-
2011
- 2011-01-27 CN CN201210332221.XA patent/CN102831642B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US3828126A (en) * | 1973-04-13 | 1974-08-06 | American Express Invest | Real time interferometry |
| US4794550A (en) * | 1986-10-15 | 1988-12-27 | Eastman Kodak Company | Extended-range moire contouring |
| CN1702491A (en) * | 2005-04-28 | 2005-11-30 | 徐能佳 | Reflecting line-structured light |
| WO2010048960A1 (en) * | 2008-10-28 | 2010-05-06 | 3Shape A/S | Scanner with feedback control |
| CN101871773A (en) * | 2009-04-22 | 2010-10-27 | 陈亮嘉 | Synchronous hue phase shift conversion method and three-dimensional morphology measurement system thereof |
Non-Patent Citations (4)
| Title |
|---|
| 《一种线结构光自扫描测量系统的研究》;商大伟;《机电工程》;20101031;第27卷(第10期);全文 * |
| 《全场视觉自扫描测量系统》;解则晓;《机械工程学报》;20071130;第11卷(第43期);全文 * |
| 商大伟.《一种线结构光自扫描测量系统的研究》.《机电工程》.2010,第27卷(第10期), |
| 解则晓.《全场视觉自扫描测量系统》.《机械工程学报》.2007,第11卷(第43期), |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102831642A (en) | 2012-12-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102074044B (en) | System and method for reconstructing surface of object | |
| CN106767527B (en) | A kind of optics mixing detection method of three-D profile | |
| EP3418681B1 (en) | Calibration of a triangulation sensor | |
| CN102831642B (en) | System and method for object surface reconstruction | |
| JP6447055B2 (en) | Calibration method, calibration device, measuring tool and program | |
| CN108759669B (en) | Indoor self-positioning three-dimensional scanning method and system | |
| US20070211258A1 (en) | Three-dimensional shape measurement apparatus and method for eliminating2pi ambiguity of moire principle and omitting phase shifting means | |
| CN103782129B (en) | Utilize 3 d shape measuring apparatus and the method for projection grating amplitude | |
| CN101825431A (en) | Reference image techniques for three-dimensional sensing | |
| CN102867328B (en) | A system for object surface reconstruction | |
| US20140174149A1 (en) | Calibration method for trigonometric-based ranging systems in multiple media | |
| CN107179069B (en) | Device and method for measuring flexible motion parameters of satellite solar panels based on binocular stereo vision | |
| CN110612428B (en) | Three-dimensional measurement method using characteristic quantity and apparatus therefor | |
| CN106123802A (en) | A kind of autonomous flow-type 3 D measuring method | |
| CN102867329B (en) | Object surface reconstruction system and method | |
| CN102466478A (en) | Distance measuring system and method for moving object | |
| CN115053151A (en) | Nano-imprinting microlens array and method of fabricating the same | |
| CN209486405U (en) | The assembling & adjusting system of optical plate in a kind of optics module | |
| CN102095386B (en) | Two-dimensional small-angle laser-vision precise measurement device and implementation method thereof | |
| JP2017125801A (en) | Three-dimensional shape measuring method, displacement measuring method, three-dimensional shape measuring device, displacement measuring device, structure manufacturing method, structure manufacturing system, and three-dimensional shape measuring program | |
| CN110135011A (en) | A Vision-Based Visualization Method for Vibration Shape of Flexible Board | |
| CN114111626A (en) | Light field camera three-dimensional measuring device and system based on coaxial projection | |
| Li et al. | Measurement of three-dimensional profiles with multi structure linear lighting | |
| CN206235304U (en) | A kind of laser measuring apparatus | |
| KR101071861B1 (en) | 3-dimensional measuring apparatus for tire |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C56 | Change in the name or address of the patentee | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 518054 high tech Industrial Park, Guangdong, Shenzhen Province, the south side of the building, building, building 4 Patentee after: SHENZHEN TAISHAN SPORTS TECHNOLOGY CORP., LTD. Address before: 518054 high tech Industrial Park, Guangdong, Shenzhen Province, the south side of the building, building, building 4 Patentee before: Shenzhen Tol Technology Co., Ltd. |
|
| CP01 | Change in the name or title of a patent holder |
Address after: 518054 4th floor, Fangda building, South District, high tech Industrial Park, Shenzhen, Guangdong Patentee after: Shenzhen Taishan Sports Technology Co.,Ltd. Address before: 518054 4th floor, Fangda building, South District, high tech Industrial Park, Shenzhen, Guangdong Patentee before: SHENZHEN TAISHAN SPORTS TECHNOLOGY Corp.,Ltd. |
|
| CP01 | Change in the name or title of a patent holder |