CN106568394A - Hand-held three-dimensional real-time scanning method - Google Patents

Abstract

The invention provides a hand-held three-dimensional real-time scanning method. The method comprises steps that (1), a measurement angle of a measured object is selected, and an adjacent angle superposition rate is made to be greater than 60%; (2), a measurement mode is selected, and inching measurement or continuous measurement is carried out; (3), the exposure time of a camera is adjusted according to the information of different-angle illumination and reflectivity; (4), scanning measurement is carried out by the camera according to the selected measurement mode and the selected measurement angle, namely shooting; (5), the three frequency colorful phase shift profilometry and a colorful stripe decoupling method are employed to acquire three-dimensional point cloud and a texture graph of each angle; and (6), the acquired different-angle three-dimensional point cloud acquired in the step (5) are spliced to be an integral model. The method is advantaged in that the method is applicable to most measurement objects, especially dynamic objects, the method needs no artificial mark points, and a multi-angle measurement splicing problem can be excellently solved.

Description

Hand-held three-dimensional real-time scanning method

Technical Field

The invention relates to the field of optical detection, in particular to a full-field rapid three-dimensional measurement method based on a novel multi-angle splicing method.

Background

The portable scanners in the market have smaller volumes and are convenient to carry, but the three-dimensional point cloud can be reconstructed after a multi-frame continuous exposure acquisition deformation fringe pattern is needed in each measurement, and the operation mode is generally as follows: selecting a position, fixing an instrument, measuring, moving the position, fixing and measuring. The working mode has low measuring efficiency, cannot realize handheld continuous scanning, and is particularly not suitable for scanning dynamic objects. Although some portable scanners can adopt a handheld scanning mode by means of a stereoscopic vision technology, mark points need to be marked on a measured object for positioning before measurement, so that the measurement efficiency is low, the workload is large, and points reconstructed by a machine vision method are related to the number and the positions of the marked mark points, so that dense point clouds cannot be reconstructed. The laser three-dimensional scanner can also be made into a portable scanner, which also requires the scanner to be kept still during the measurement process, so that the handheld scanning cannot be realized, and in addition, the laser scanner is point scanning or line scanning, so that the scanning efficiency is low.

Multi-angle stitching of handheld three-dimensional scanners is also a concern with respect to the several scanning techniques described above. On one hand, splicing can be achieved by calibrating the position of the multiple measurement scanner or the position of the measured object, but calibration requires auxiliary equipment, such as a displacement table, which increases cost and limits application occasions. On the other hand, the splicing is assisted by attaching some mark points on the surface of the measured object, which can cause the measurement period to be lengthened and the measurement to be complicated.

In view of the foregoing, there is a need for a handheld three-dimensional real-time scanner that can meet both handheld measurements and dynamic measurements, and that is better able to match up the angular measurements.

Disclosure of Invention

The invention aims to realize a portable and operational handheld scanning method which is suitable for most measuring objects, especially dynamic objects, and can well realize the splicing problem of multi-angle measurement without manual marking points.

The invention provides a hand-held three-dimensional real-time scanning method, which comprises the following steps:

(1) selecting a measurement angle of a measured object, and enabling the coincidence rate of adjacent angles to be larger than 60%;

(2) selection of measuring mode, jog or continuous measurement

Inching measurement, namely, performing measurement once when triggering is performed once under the condition of meeting the measurement angle in the step (1); continuous measurement, namely moving the scanner at a constant speed around a target object, and simultaneously continuously exposing the camera to realize continuous measurement;

(3) the camera automatically adjusts the exposure time according to the information of the illumination and the reflectivity at different angles;

(4) the camera performs scanning measurement according to the selected measurement mode and the selected measurement angle, namely photographing;

(5) obtaining three-dimensional point clouds and texture maps of all angles by adopting a three-frequency color phase-shift profilometry and a color fringe decoupling method;

(6) and (5) automatically splicing the three-dimensional point clouds with different angles obtained in the step (5) into a complete model.

In one embodiment of the invention, when a person carries out handheld measurement, the trigger speed of continuous measurement is set to be 5-10 frames/s.

In one embodiment of the present invention, the exposure automatic adjustment method of step (3) is as follows:

obtaining a high-frequency component of the three-frequency color stripes collected by the camera by adopting empirical mode decomposition and two-dimensional Hilbert transform;

extracting two-dimensional amplitude of a high-frequency component of a picture acquired by a camera by adopting two-dimensional Hilbert, and calculating the maximum value and the minimum value of the two-dimensional amplitude;

and calculating the difference between the maximum value and the minimum value of the fringe pattern acquired by the camera and the maximum value and the minimum value of the projection fringe pattern respectively, and adjusting the exposure time to ensure that the sum of the difference is minimum.

In one embodiment of the present invention, the tri-band color phase shift profilometry and texture retrieval method of step (5) is as follows:

projecting a three-frequency color sine fringe pattern on the surface of a measured object by adopting a color grating projection method;

setting RGB three-channel frequency ratio, and shooting a deformed fringe pattern projected on the surface of an object by using a color camera;

using Fourier transform to separate three carrier frequency fringe patterns of high, middle and low of the deformed fringe pattern, namely three frequencies;

respectively unfolding the phases of the three-frequency carrier frequency fringe pattern, and unwrapping the phases to recover the high-frequency carrier frequency phase;

obtaining three-dimensional point cloud through parameters calibrated by a system;

and obtaining a three-frequency sine fringe component by adopting a frequency domain minimization method for the deformed fringe pattern, and subtracting the three-frequency sine fringe component from the color deformed fringe pattern so as to recover the texture pattern of the surface of the measured object.

In one embodiment of the present invention, the RGB three channel frequency ratio is 1: 4: 12.

In one embodiment of the invention, the method of obtaining a three-dimensional point cloud is as follows:

(a) generating a three-frequency color fringe pattern by a computer, and projecting three-frequency color fringes onto the surface of a measured object by a projection device;

(b) shooting a measured object by a color camera to generate a monocular color deformation fringe pattern;

(c) separating high, medium and low three carrier frequencies by Fourier change to obtain three deformation fringe patterns of high, medium and low of a single unit;

(d) performing stripe self-adaptive analysis and phase extraction by using Hilbert-Huang transform HHT to obtain high, medium and low frequency wrapping phases;

(e) obtaining a high-precision absolute phase by a time domain variable precision unwrapping method;

(f) obtaining three-dimensional point cloud of the measured object by system calibration parameters according to the absolute phase obtained in the step (e);

(g) obtaining a texture map: respectively subtracting the high, middle and low three-frequency components obtained in the step (c) from the obtained color deformation fringe pattern to obtain background components; separating and recovering the obtained background components through texture illumination to obtain original texture information of the measured object;

(h) obtaining three-dimensional point cloud containing texture information: and (d) assigning the texture map obtained in the step (g) to the point cloud obtained in the step (f), so as to obtain the high-precision texture-containing three-dimensional point cloud of the measured object.

In one embodiment of the present invention, the automatic splicing method of step (6) is as follows:

assuming that a world coordinate system is coincident with a camera coordinate system under the 1 st angle, point clouds obtained by n angles measured each time are converted into the world coordinate system, and multi-angle splicing can be realized; the world coordinate system is expressed as (X)^w，Y^w，Z^w，O^w) The camera coordinate system at each angle is represented as (X)ⁱ，Yⁱ,Zⁱ，Oⁱ) Simultaneously (X)^w，Y^w，Z^w，O^w)＝(X¹，Y¹，Z¹，O¹) I.e. the world coordinates coincide with the camera coordinate system of the first location; the representation of the point cloud isWherein i represents the point cloud obtained by the ith measurement, j represents j points in the point cloud, and l represents coordinates under the l angular coordinate system;

1) splicing results of two adjacent angle scans

For the splicing of exposure measurement results of two adjacent angles, a method for splicing point clouds obtained by two adjacent scans is explained by taking the splicing of three-dimensional point clouds measured at the ith angle and the (i + 1) th angle as an example; wherein the point cloud obtained by the ith measurement is represented ask represents the number of points of the point cloud i; the point cloud obtained from the (i + 1) th measurement is represented ass represents the number of points of the point cloud i + 1; the specific splicing method comprises the following steps:

a) finding sparse corresponding feature points in the texture maps i and i +1 by using a scale invariant feature transformation method, wherein the sparse corresponding feature points are marked as { a1, a 2., am } and { b1, b 2., bm }, and m is the number of corresponding points found in the texture maps;

b) points with larger epipolar line errors are removed by a random sampling consistency method, the reserved points are recorded as { a1, a 2., ar } and { b1, b 2., br }, and r is the reserved corresponding points found in the texture map;

c) finding the spatial point coordinates { A1, A2, A3,. Ar } and { B1, B2, B3,. Br } corresponding to the matching feature points, wherein,representing the feature points in the ith point cloud,representing the corresponding point in the (i + 1) th point cloud, wherein r is less than or equal to s, k;

d) the iterative method solves the formula 1 to obtain p1, wherein p1 represents the (i + 1) th coordinate system (X)ⁱ⁺¹，Yⁱ⁺¹，Zⁱ⁺¹，Oⁱ¹) In the i-th coordinate system (X)ⁱ，Yⁱ，Zⁱ，Oⁱ) Pose potential matrix of (X)₀，Y₀，Z₀) And represents the position of the position, gamma, theta,representing the attitude, i.e. the (i + 1) th coordinate system (X)ⁱ⁺¹，Yⁱ⁺¹，Zⁱ⁺¹，Oⁱ⁺¹) With the i-th coordinate system (X)ⁱ，Yⁱ，Zⁱ，Oⁱ) Angle between coordinate axes

Wherein p1 ═ R T]，R＝RotX·RotY·RotZ，T＝[X_0，Y_0，，Z_0，1]ⁱ，

e) All points in the point cloud i +1The matrix p1 is multiplied to the left, and the transformed matrix is converted into the coordinate system of the point cloud i, namely

f) Solving a space coordinate matrix p2 according to the consistency of the curvature of the corresponding point and the normal direction, wherein the meaning represented by p2 is the same as that of p 1;

g) iterative optimization is carried out on the splicing result by adopting an iterative closest point ICP iterative method, an optimization parameter p3 in a formula (2) is solved, the meaning of p3 is the same as that of p1, and coordinate transformation is carried out on the point cloud i +1 obtained in e) again;

f(p3)＝∑||M_i，j-p3·M_i+1，j||²＝min (2)

to this end, a transformation matrix p (i) ═ p3 · p2 · p1 is obtained by stitching the i +1 th point cloud and the i-th point cloud;

2) stitching of multiple exposure measurements

Obtaining P (1), P (2), P (3), and P (n-1) according to the method described in 1), splicing the ith point cloud with the 1 st point cloud obtained by the first exposure measurement, and performing coordinate transformationWherein,representing transformation of the i +1 th angle measurement to the world coordinate system (X)^w，Y^w，Z^w，O^w) I.e., the coordinates of the 1 st measurement angle coordinate system,represents the coordinates of the (i + 1) th angle measurement result in the original coordinate system, i.e. the (i + 1) th measurement angle coordinate system, and is recorded as H_i＝P(1)·P(2)·P(3).....P(i-1)；

And (5) converting the 2 nd to n th angle measurement results into a world coordinate system according to the steps, completing the splicing of multi-angle measurement, and reconstructing a complete three-dimensional model.

In one embodiment of the present invention, the step of solving for p2 in step f) is:

the method comprises the following steps: initializing p2 to 0;

step two: respectively carrying out triangular patch division on the point cloud i and the point cloud i +1, namely constructing a triangular patch model with a connected topological relation; the point cloud i and the divided patch are marked as a model i and expressed asSimilarly, point cloud i +1 and its divided patch are denoted as model i +1, and are represented as

Step three: for model i +1, i.e.Performing a coordinate transformation of the transformation matrix P2, i.e.And calculating the curvature and normal direction of each point after coordinate transformation, and expressing as r_i，1，r_i，2，r_i，3，...，r_i，k}，{r_i+1，1，r_i+1，2，r_i+1，3，...，r_i+1，sAnd { N }_i，1，N_i，2，N_i，3，...，N_i，k}，{N_i+1，1，N_i+1，2，N_i+1，3，...，N_i+1，s}；

Step four: each point of the surface model i +1 is along its normal direction { N_i+1，1，N_i+1，2，N_i+1，3，...，N_i+1，sFinding the corresponding point on the surface model i;

define Q as a point on the surface model i +1If a point Q' on the surface model is found to satisfy the following two conditions, it is the corresponding point on the surface model i.

i.Q' connecting the Q line direction and the normal direction N of Q_n+1，jAt very small included angles, i.e.

angle[|Q′-Q|，N_n+1，j]≤ω；

The distance between Q 'and Q is less than a certain range, namely distance | Q' -Q | is less than or equal to d;

of the points satisfying the i and ii conditions, the point having the closest curvature is selected as the matching point, i.e., the point having the closest curvature

f(Q′)＝min|r_Q′-r_i+1，j|；

If the point meeting the conditions i and ii is not found, the point Q is considered to not find a matching point; wherein, omega and d are obtained by experimental calibration; corresponding points on the model i found by the model i +1 are respectively represented as a point set { Q }₁，Q₂，...，Q_vAnd { Q }₁′，Q₂′，...，Q_v' }, v denotes the number of found corresponding point pairs, the curvatures of which are expressed asAnd

step five, calculating the curvature variance of the corresponding points

Step six, if g is less than or equal to sigma, outputting p 2; if g > σ, change p2 and repeat steps two through six.

The invention has the advantages that:

1) three-dimensional point cloud measurement can be completed by realizing single-frame exposure by using a method of projecting a three-frequency color sine fringe pattern, and the measurement result point cloud is dense and high in precision;

2) automatically adjusting the exposure time of the camera according to the information such as color, texture, reflectivity and the like of the measured object;

3) the two measurement modes of inching measurement and continuous measurement can be realized, and in the continuous measurement mode, the scanner can continuously expose at a certain frame rate to realize continuous measurement;

4) the single-angle three-dimensional measurement of the motion curved surface can be realized;

5) obtaining a three-frequency sine stripe component by a frequency domain minimization method, and subtracting the three-frequency sine stripe component from the color deformation stripe graph to recover the texture graph of the surface of the measured object;

6) and realizing the splicing of multi-angle and multi-frame exposure measurement according to the texture map recovered from the deformed stripe map, the curvature of the curved surface, the normal direction and other information.

Drawings

FIG. 1 illustrates a general flow diagram of the handheld three-dimensional scanning method of the present invention;

FIG. 2 shows a projected color sine-fringe pattern (R: G: B: 1: 4: 12);

FIG. 3 shows a flow chart of a three-frequency color sinusoidal fringe profiling and color texture restoration method;

FIG. 4 shows a flow chart of a method for stitching two adjacent angle measurements;

FIG. 5 is a diagram illustrating a multi-angle scanning result stitching method;

FIG. 6 illustrates a face 6 angular scan model;

FIG. 7 shows a multi-angle presentation of the stitched model.

Detailed Description

The present invention will be described in detail below with reference to the accompanying drawings.

1. Overall measurement method

The invention relates to a hand-held three-dimensional scanning method which is applied to a mobile, hand-held and portable three-dimensional scanner. The handheld three-dimensional scanning method has two measurement modes, namely inching measurement and continuous measurement. In both measurement modes, the measurement process and the measurement method are the same, and the overall measurement method is shown in fig. 1.

(1) And selecting the measurement angle of the measured object to ensure that the coincidence rate of adjacent angles is more than 60 percent.

For example, the measurement angles are sequentially selected to be about 0, 30, 60, 90,. and 330 degrees.

(2) Selecting a measuring mode, a jog measurement or a continuous measurement.

Inching measurement, namely, performing measurement once when triggering is performed once under the condition of meeting the measurement angle in the step (1); during continuous measurement, the scanner is moved at a constant speed around the target object, and the camera is continuously exposed, so that continuous measurement is realized, for example, for a field size of 0.5m × 0.5m, if the moving speed of an operator is 1m/s, and the trigger speed of the signal generator is 4 frames/s, the coincidence rate between two adjacent measurements can be ensured to be more than 60%. When people hold the measuring tool, in order to avoid overlarge data amount caused by too much overlapped parts, the triggering speed of continuous measurement can be set to be 5-10 frames/s through experimental verification.

(3) The camera automatically adjusts the exposure time according to the information of illumination, reflectivity and the like at different angles. The camera auto-adjustment method is described in detail in the following 2.

(4) And the camera performs scanning measurement, namely photographing according to the selected measurement mode and the selected measurement angle.

(5) And obtaining three-dimensional point clouds and texture maps of all angles by adopting a three-frequency color phase-shift profilometry and a color fringe decoupling method.

The detailed steps are as follows and shown in fig. 3.

(6) And (4) automatically splicing the three-dimensional point clouds with different angles obtained in the step (5) into a complete model, wherein the automatic splicing method is as shown in the following and in fig. 4 and 5.

2. Automatic exposure adjustment

The high-frequency components (Zou H-H, Zhou X, ZHao H, et al. color front-project detection for measuring two dimensional object based on two dimensional spatial temporal composition [ J ]. Applied optics, 2012, 51 (16): 3622) 3630.) of the tri-band color stripes collected by the camera are obtained by empirical mode decomposition and two-dimensional Hilbert transform, the two-dimensional amplitude of the high-frequency components of the photos collected by the camera is extracted by two-dimensional Hilbert, the maximum value and the minimum value of the two-dimensional amplitude are calculated, the difference between the maximum value and the minimum value of the stripe pattern collected by the camera and the maximum value and the minimum value of the projection stripe pattern are calculated, and the exposure time is adjusted to minimize the sum of the difference, so as to achieve the purpose of adjusting the exposure.

3. Tri-frequency color phase shift profilometry and texture recovery

The hand-held three-dimensional scanning method of the invention adopts a color grating projection method, projects a three-frequency color sine stripe graph shown in figure 2 to the surface of an object to be measured, the ratio of RGB three-channel frequencies is 1: 4: 12, shoots a deformed stripe graph projected on the surface of the object by a color camera, separates three carrier frequencies (Takeda M, Mutoh K. fourier transform profile for the automatic measurement of 3-D objects shape [ J ]. Applied optics, 1983, 22 (24): 397 < 7 > 3982.) (hereinafter referred to as "three-frequency"), then spreads the three-frequency phases respectively, performs phase unwrapping (Zhao H, Chen W, Tan Y. unwrapped for the three-dimensional measurement of three-dimensional shapes [ J ]. application, 449, 33, 4500), and finally obtains a three-dimensional carrier frequency point cloud (4500 ) through a point cloud calibration X system, xi J, Jin Y, et al, accurate calibration for a camera-projector system based on structured lighting project [ J ]. Optics and Lasers in Engineering, 2009, 47 (3): 310-319.). The deformed fringe pattern adopts a frequency domain minimization method (Zou H-H, Zhou X, ZHao H, et al color fringe-projected technical for measuring dynamic object based on two dimensional empirical mode decomposition [ J ]. Applied optics, 2012, 51 (16): 3622 and 3630.) to obtain a three-frequency sine fringe component, and the texture pattern of the measured object surface is recovered by subtracting the three-frequency sine fringe component from the color deformed fringe pattern.

The specific steps for obtaining a three-dimensional point cloud are shown in fig. 3 and explained as follows.

(a) Generating a three-frequency color fringe pattern by a computer, and projecting the three-frequency color fringe pattern onto the surface of a measured object by a projection device, as shown in FIG. 2;

(b) shooting a measured object by a color camera to generate a monocular color deformation fringe pattern;

(c) separating high, medium and low three carrier frequencies by Fourier change to obtain three deformation fringe patterns of high, medium and low of a single unit;

(d) performing stripe self-adaptive analysis and phase extraction by using Hilbert-Huang transform (HHT) to obtain high, medium and low three-frequency wrapping phases;

(e) obtaining a high-precision absolute phase by a time domain variable precision unwrapping method;

(f) and (e) obtaining the three-dimensional point cloud of the measured object by the system calibration parameters according to the absolute phase obtained in the step (e).

(g) Obtaining a texture map: respectively subtracting the high, middle and low three-frequency components obtained in the step (c) from the obtained color deformation fringe pattern to obtain background components; and the obtained background component is separated and recovered through texture illumination to obtain the original texture information of the measured object.

(h) Obtaining three-dimensional point cloud containing texture information: and (d) assigning the texture map obtained in the step (g) to the point cloud obtained in the step (f) to obtain the high-precision texture-containing three-dimensional point cloud of the measured object.

4. Automatic splicing method

The hand-held three-dimensional scanning method can realize automatic seamless splicing without mark points, and the splicing mainly depends on texture information recovered from a deformed fringe pattern, takes the curvature and the normal direction of measured data as constraints, searches the position relation between two adjacent frames of measurement results, and finally realizes global splicing.

The coordinate system of the point cloud scanned at each angle is superposed with the camera coordinate system of the scanning position at that time, so if the point cloud coordinates scanned at different angles are not in the same coordinate system, the point clouds scanned at different angles are converted into the point cloud under the same coordinate system, and the process is the splicing of the point clouds. And if the world coordinate system is overlapped with the camera coordinate system under the angle 1, converting point clouds obtained by n angles measured each time into the world coordinate system, and realizing multi-angle splicing. The world coordinate system is expressed as (X)^w，Y^w，Z^w，O^w) The camera coordinate system at each angle is represented as (X)ⁱ，Yⁱ，Zⁱ，Oⁱ) Simultaneously (X)^w，Y^w，Z^w，O^w)＝(X¹，Y¹，Z¹，O¹) I.e. the world coordinates coincide with the camera coordinate system of the first location; the representation of the point cloud isWherein i represents the point cloud obtained by the ith measurement, j represents j points in the point cloud, and l represents coordinates under the ith angular coordinate system.

1) Splicing results of two adjacent angle scans

For the splicing of exposure measurement results of two adjacent angles, the splicing method of point clouds obtained by two adjacent scans is described in detail below by taking the splicing of three-dimensional point clouds measured at the ith angle and the (i + 1) th angle as an example. Wherein the point cloud obtained by the ith measurement is represented ask represents the number of points of the point cloud i; point cloud obtained by i +1 th measurementIs shown ass represents the number of points of the point cloud i + 1; the splicing method is shown in fig. 4, and the specific steps are as follows.

a) A method of extracting Feature points using Scale-invariant Feature Transform (SIFT) (Lowe dg. discrete image features from innovative keys [ J ]. International journal of computer vision, 2004, 60 (2): 91-110.) find corresponding feature points sparse in texture maps i and i +1, which are marked as { a1, a2,. said., am } and { b1, b2,. said., bm }, where m is the corresponding point number found in the texture maps;

b) random Sample Consensus (RANSAC) method (Fischler MA, balls RC. A partner for Model Fitting with Applications to image analysis and Automated graphics (reprinted in reading in computer vision, ed. MA Fischler, "[ J ]. Comm ACM, 1981, 24 (6): 381) 395) was used. Removing points with larger epipolar line errors, and recording the reserved points as { a1, a2,. multidot., ar } and { b1, b2,. multidot.,. br }, wherein r is the reserved corresponding points found in the texture map;

c) finding the spatial point coordinates { A1, A2, A3,. Ar } and { B1, B2, B3,. Br } corresponding to the matching feature points, wherein,representing the feature points in the ith point cloud,and (3) representing the corresponding point in the (i + 1) th point cloud, wherein r is less than or equal to s, k.

d) The iterative method solves the formula 1 to obtain p1, wherein p1 represents the (i + 1) th coordinate system (X)ⁱ⁺¹，Yⁱ⁺¹，Zⁱ⁺¹，Oⁱ¹) In the i-th coordinate system (X)ⁱ，Yⁱ，Zⁱ，Oⁱ) Pose potential matrix of (X)₀，Y₀，Z₀) And represents the position of the position, gamma, theta,representing the attitude, i.e. the (i + 1) th coordinate system (X)ⁱ⁺¹，Yⁱ⁺¹，Zⁱ⁺¹，Oⁱ⁺¹) With the i-th coordinate system (X)ⁱ，Yⁱ，Zⁱ，Oⁱ) The included angle of each coordinate axis.

Wherein p1 ═ R T]，R＝RotX·RotY·RotZ，T＝[X_0，Y_0，，Z_0，1]ⁱ，

e) All points in the point cloud i +1The matrix p1 is multiplied to the left, and the transformed matrix is converted into the coordinate system of the point cloud i, namely

f) According to the consistency of the curvature of the corresponding point and the normal direction, solving a space coordinate matrix p2, wherein the meaning represented by p2 is the same as that of p1, and the step of solving p2 is as follows:

the method comprises the following steps: initializing p2 to 0;

step two: respectively carrying out triangular patch division on the point cloud i and the point cloud i +1, namely constructing a triangular patch model with a connected topological relation; the point cloud i and the divided patch are marked as a model i and expressed asSimilarly, point cloud i +1 and its divided patch are denoted as model i +1, and are represented as

Step three: for model i +1, i.e.Performing a coordinate transformation of the transformation matrix to p2, i.e.And calculating the curvature and normal direction of each point after coordinate transformation, and expressing as r_i，1，r_i，2，r_i，3，...，r_i，k}，{r_i+1，1，r_i+1，2，r_i+1，3，...，r_i+1，sAnd { N }_i，1，N_i，2，N_i，3，...，N_i，k}，{N_i+1，1，N_i+1，2，N_i+1，3，...，N_i+1，s}；

Step four: each point of the surface model i +1 is along its normal direction { N_i+1，1，N_i+1，2，N_i+1，3，...，N_i+1，sFind the corresponding point on the surface model i. Define Q as a point on the surface model i +1If a point Q' on the surface model is found to satisfy the following two conditions, it is the corresponding point on the surface model i.

i.Q' connecting the Q line direction and the normal direction N of Q_n+1，jAt very small included angles, i.e.

angle[|Q′-Q|，N_n+1，j]≤ω；

The distance between Q 'and Q is less than a certain range, namely distance | Q' -Q | is less than or equal to d;

among the points satisfying the conditions of i and ii, the point having the closest curvature is selected as a matching point, that is, the point having the closest curvature is selected as the matching point

f(Q′)＝min|r_Q′-r_i+1，j|；

If the point meeting the conditions i and ii is not found, the point Q is considered to not find a matching point; wherein, omega and d are obtained by experimental calibration. Corresponding points on the model i found by the model i +1 are respectively represented as a point set { Q }₁，Q₂，...，Q_v- } and { Q₁′，Q₂′，...，Q_v' }, v denotes the number of found corresponding point pairs, the curvatures of which are expressed asAnd

step five, calculating the curvature variance of the corresponding points

Step six, if g is less than or equal to sigma, outputting p 2; if g > σ, change p2 and repeat steps two through six.

g) Iterative optimization is carried out on the splicing result by adopting an iterative Closest point ICP (iterative Closest point) iteration method, an optimization parameter p3 in a formula (2) is solved, the meaning represented by p3 is the same as that of p1, and coordinate transformation is carried out on the point cloud i +1 obtained in e) again;

f(p3)＝∑||M_i，j-p3·M_i+1，j||²＝min (2)

to this end, a transformation matrix p (i) ═ p3 · p2 · p1 is obtained by stitching the i +1 th point cloud and the i-th point cloud;

2) stitching of multiple exposure measurements

The stitching process for the multiple exposure measurements is shown in fig. 5.

According to the method described in 1), P (2), P (3), and P (n-1) can be obtained, and for the ith point cloud, the ith point cloud is spliced with the 1 st point cloud obtained by the first exposure measurement, and the required coordinate transformation is carried outWherein,representing transformation of the i +1 th angle measurement to the world coordinate system (X)^w，Y^w，Z^w，O^w) I.e., the coordinates of the 1 st measurement angle coordinate system,represents the coordinates of the (i + 1) th angle measurement result in the original coordinate system, i.e. the (i + 1) th measurement angle coordinate system, and is recorded as H_i＝P(1)·P(2)·P(3).....P(i-1)。

And (5) converting the 2 nd to n th angle measurement results into a world coordinate system according to the steps, completing the splicing of multi-angle measurement, and reconstructing a complete three-dimensional model.

Experimental cases:

the hand-held three-dimensional scanning equipment described by the invention is used for scanning the human face, and the human face has uneven reflectivity due to complex color, so that the problem is well solved by using the automatic exposure adjustment technology of the invention. Fig. 6 is a three-dimensional scan of a person with six angles, and fig. 7 is a multi-angle presentation of the results of the automatic stitching after scanning.

Claims (8)

1. A hand-held three-dimensional real-time scanning method comprises the following steps:

(1) selecting a measurement angle of a measured object, and enabling the coincidence rate of adjacent angles to be larger than 60%;

(2) selection of measuring mode, jog or continuous measurement

Inching measurement, namely, performing measurement once when triggering is performed once under the condition of meeting the measurement angle in the step (1); continuous measurement, namely moving the scanner at a constant speed around a target object, and simultaneously continuously exposing the camera to realize continuous measurement;

(3) the camera automatically adjusts the exposure time according to the information of the illumination and the reflectivity at different angles;

(4) the camera performs scanning measurement according to the selected measurement mode and the selected measurement angle, namely photographing;

(5) obtaining three-dimensional point clouds and texture maps of all angles by adopting a three-frequency color phase-shift profilometry and a color fringe decoupling method;

(6) and (5) automatically splicing the three-dimensional point clouds with different angles obtained in the step (5) into a complete model.

2. The handheld three-dimensional real-time scanning method as claimed in claim 1, wherein the trigger speed of continuous measurement is set to 5-10 frames/s during handheld measurement of a person.

3. The handheld three-dimensional real-time scanning method as claimed in claim 1, wherein the exposure automatic adjustment method of step (3) is as follows:

obtaining a high-frequency component of the three-frequency color stripes collected by the camera by adopting empirical mode decomposition and two-dimensional Hilbert transform;

extracting two-dimensional amplitude of a high-frequency component of a picture acquired by a camera by adopting two-dimensional Hilbert, and calculating the maximum value and the minimum value of the two-dimensional amplitude;

and calculating the difference between the maximum value and the minimum value of the fringe pattern acquired by the camera and the maximum value and the minimum value of the projection fringe pattern respectively, and adjusting the exposure time to ensure that the sum of the difference is minimum.

4. The handheld three-dimensional real-time scanning method as claimed in claim 1, wherein the tri-band color phase shift profilometry and texture retrieval method of step (5) is as follows:

projecting a three-frequency color sine fringe pattern on the surface of a measured object by adopting a color grating projection method;

setting RGB three-channel frequency ratio, and shooting a deformed fringe pattern projected on the surface of an object by using a color camera;

using Fourier transform to separate three carrier frequency fringe patterns of high, middle and low of the deformed fringe pattern, namely three frequencies;

respectively unfolding the phases of the three-frequency carrier frequency fringe pattern, and unwrapping the phases to recover the high-frequency carrier frequency phase;

obtaining three-dimensional point cloud through parameters calibrated by a system;

and obtaining a three-frequency sine fringe component by adopting a frequency domain minimization method for the deformed fringe pattern, and subtracting the three-frequency sine fringe component from the color deformed fringe pattern so as to recover the texture pattern of the surface of the measured object.

5. The handheld three-dimensional real-time scanning method of claim 4, wherein the RGB three-channel frequency ratio is 1: 4: 12.

6. The handheld three-dimensional real-time scanning method of claim 4, wherein the three-dimensional point cloud is obtained by:

(a) generating a three-frequency color fringe pattern by a computer, and projecting three-frequency color fringes onto the surface of a measured object by a projection device;

(b) shooting a measured object by a color camera to generate a monocular color deformation fringe pattern;

(c) separating high, medium and low three carrier frequencies by Fourier change to obtain three deformation fringe patterns of high, medium and low of a single unit;

(d) performing stripe self-adaptive analysis and phase extraction by using Hilbert-Huang transform HHT to obtain high, medium and low frequency wrapping phases;

(e) obtaining a high-precision absolute phase by a time domain variable precision unwrapping method;

(f) obtaining three-dimensional point cloud of the measured object by system calibration parameters according to the absolute phase obtained in the step (e);

(g) obtaining a texture map: respectively subtracting the high, middle and low three-frequency components obtained in the step (c) from the obtained color deformation fringe pattern to obtain background components; separating and recovering the obtained background components through texture illumination to obtain original texture information of the measured object;

(h) obtaining three-dimensional point cloud containing texture information: and (d) assigning the texture map obtained in the step (g) to the point cloud obtained in the step (f), so as to obtain the high-precision texture-containing three-dimensional point cloud of the measured object.

7. The handheld three-dimensional real-time scanning method as claimed in claim 1, wherein the automatic stitching method of step (6) is as follows:

assuming that a world coordinate system is coincident with a camera coordinate system under the 1 st angle, point clouds obtained by n angles measured each time are converted into the world coordinate system, and multi-angle splicing can be realized; the world coordinate system is expressed as (X)^w，Y^w，Z^w，O^w) The camera coordinate system at each angle is represented as (X)ⁱ，Yⁱ，Zⁱ，Oⁱ) At the same time (X)^w，Y^w，Z^w，O^w)＝(X¹，Y¹，Z¹，O¹) I.e. the world coordinates coincide with the camera coordinate system of the first location; the representation of the point cloud isWherein i represents the point cloud obtained by the ith measurement, j represents j points in the point cloud, and l represents coordinates under the l angular coordinate system;

1) splicing results of two adjacent angle scans

For the splicing of exposure measurement results of two adjacent angles, a method for splicing point clouds obtained by two adjacent scans is explained by taking the splicing of three-dimensional point clouds measured at the ith angle and the (i + 1) th angle as an example; wherein the point cloud obtained by the ith measurement is represented ask represents the number of points of the point cloud i; the point cloud obtained from the (i + 1) th measurement is represented ass represents the number of points of the point cloud i + 1; the specific splicing method comprises the following steps:

a) finding sparse corresponding feature points in the texture map i and i +1 by using a scale invariant feature conversion method, marking as { a1, a2, …, am } and { b1, b2, …, bm }, wherein m is the number of corresponding points found in the texture map;

b) points with larger epipolar line errors are removed by a random sampling consistency method, the reserved points are marked as { a1, a2, …, ar } and { b1, b2, …, br }, and r is the reserved corresponding point number found in the texture map;

c) finding the spatial point coordinates { A1, A2, A3, … Ar } and { B1, B2, B3, … Br } corresponding to the matching feature points, wherein,representing the feature points in the ith point cloud,representing the corresponding point in the (i + 1) th point cloud, wherein r is less than or equal to s, k;

d) solving the formula 1 by an iterative method to obtain p1, wherein p1 represents the (i + 1) th coordinate systemIn the i-th coordinate system (X)ⁱ，Yⁱ，Zⁱ，Oⁱ) Pose potential matrix of (X)₀，Y₀，Z₀) And represents the position of the position, gamma, theta,representing the attitude, i.e. the (i + 1) th coordinate system (X)ⁱ⁺¹，Yⁱ⁺¹，Zⁱ⁺¹，Oⁱ⁺¹) With the i-th coordinate system (X)ⁱ，Yⁱ，Zⁱ，Oⁱ) Angle between coordinate axes

$\left[\begin{array}{c}A1\\ A2\\ \&CenterDot;\\ \&CenterDot;\\ \&CenterDot;\\ Ar\end{array}\right]=p1\left[\begin{array}{c}B1\\ B2\\ \&CenterDot;\\ \&CenterDot;\\ \&CenterDot;\\ Br\end{array}\right]---\left(1\right)$

Wherein p1 ═ R T]，R＝RotX·RotY·RotZ，T＝[X_0.Y_0.，Z_0.1]^l，

$RotX=\left[\begin{array}{ccc}1& 0& 0\\ 0& cos\mathrm{\&gamma;}& \sin \mathrm{\&gamma;}\\ 0& -\sin \mathrm{\&gamma;}& cos\mathrm{\&gamma;}\end{array}\right],$ $RotY=\left[\begin{array}{ccc}cos\mathrm{\&theta;}& 0& -sin\mathrm{\&theta;}\\ 0& 1& 0\\ \sin \mathrm{\&theta;}& 0& \cos \mathrm{\&theta;}\end{array}\right],$

e) All points in the point cloud i +1The matrix p1 is multiplied to the left, and the transformed matrix is converted into the coordinate system of the point cloud i, namely

f) Solving a space coordinate matrix p2 according to the consistency of the curvature of the corresponding point and the normal direction, wherein the meaning represented by p2 is the same as that of p 1;

g) iterative optimization is carried out on the splicing result by adopting an iterative closest point ICP iterative method, an optimization parameter p3 in a formula (2) is solved, the meaning of p3 is the same as that of p1, and coordinate transformation is carried out on the point cloud i +1 obtained in e) again;

f(p3)＝Σ||M_i，j-p3·M_i+1，j||²＝min (2)

to this end, a transformation matrix p (i) ═ p3 · p2 · p1 is obtained by stitching the i +1 th point cloud and the i-th point cloud;

2) stitching of multiple exposure measurements

Obtaining P (1), P (2), P (3), and P (n-1) according to the method described in 1), splicing the ith point cloud with the 1 st point cloud obtained by the first exposure measurement, and performing coordinate transformationWherein,representing transformation of the i +1 th angle measurement to the world coordinate system (X)^w，Y^w，Z^w，O^w) I.e., the coordinates of the 1 st measurement angle coordinate system,represents the coordinates of the (i + 1) th angle measurement result in the original coordinate system, i.e. the (i + 1) th measurement angle coordinate system, and is recorded as H_i＝P(1)·P(2)·P(3)……P(i-1)；

And (5) converting the 2 nd to n th angle measurement results into a world coordinate system according to the steps, completing the splicing of multi-angle measurement, and reconstructing a complete three-dimensional model.

8. The handheld three-dimensional real-time scanning method as recited in claim 7, wherein the step of solving for p2 in step f) is:

the method comprises the following steps: initializing p2 to 0;

step two: respectively carrying out triangular patch division on the point cloud i and the point cloud i +1, namely constructing a triangular patch model with a connected topological relation; the point cloud i and the divided patch are marked as a model i and expressed asSimilarly, point cloud i +1 and its divided patch are denoted as model i +1, and are represented as

Step three: for model i +1, i.e.Performing a coordinate transformation of the transformation matrix to p2, i.e.And calculating the curvature and normal direction of each point after coordinate transformation, and expressing as r_i，1，r_i，2，r_i，3，…，r_i，k}，{r_i+1，1，r_i+1，2，r_i+1，3，…，r_i+1，sAnd { N }_i，1，N_i，2，N_i，3，…，N_i，k}，{N_i+1，1，N_i+1，2，N_i+1，3，…，N_i+1，s}；

Step four: each point of the surface model i +1 is along its normal direction { N_i+1，1，N_i+1，2，N_i+1，3，…，N_i-1，sFinding the corresponding point on the surface model i;

define Q as a point on the surface model i +1If a point Q' on the surface model is found to satisfy the following two conditions, it is the corresponding point on the surface model i.

The direction of the line connecting Q' and Q and the normal direction N of Q_n+1，jAt very small included angles, i.e.

angle[|Q′-Q|，N_n+1，j]≤ω；

The distance between Q 'and Q is less than a certain range, namely distance | Q' -Q | is less than or equal to d;

of the points satisfying the conditions i and ii, the point having the closest curvature is selected as the matching point, i.e., f (Q') ═ min | r_Q′-r_i+1，j|；

If the point meeting the conditions i and ii is not found, the point Q is considered to not find a matching point; wherein, omega and d are obtained by experimental calibration; corresponding points on the model i found by the model i +1 are respectively represented as a point set { Q }₁，Q₂，...，Q_vAnd { Q }₁′，Q₂′，...，Q_v' }, v denotes the number of found corresponding point pairs, the curvatures of which are expressed asAnd

step five, calculating the curvature variance of the corresponding points

Step six, if g is less than or equal to sigma, outputting p 2; if g > σ, change p2 and repeat steps two through six.