Three-dimensional point cloud automatic registration method based on characteristic information change degree
Technical Field
The invention relates to a three-dimensional point cloud automatic registration method based on characteristic information change degree, and belongs to the field of three-dimensional information reconstruction.
Background
Three-dimensional reconstruction of object surfaces has been an important topic of research in the field of machine vision. The point cloud data of the surface of the measured object can be quickly acquired by an optical scanner, but due to the linear propagation characteristic of light, the complete data of the surface of the measured object needs to be acquired by measuring for many times under multiple view angles, so that the acquired data are not under the same coordinate system, and in order to acquire a complete model of the object, the data acquired from each view angle need to be subjected to coordinate transformation and finally combined into a unified coordinate system, which is point cloud registration. The point cloud registration technology is widely applied to the fields of robot navigation, reverse engineering, object surface shape detection, virtual reality and the like.
According to the difference of input and output results of the depth map to be registered, the existing depth map registration methods can be roughly divided into two categories: coarse registration and fine registration. The rough registration refers to finding a group of approximate coordinate transformation relations without any prior knowledge, and unifying the depth maps under two viewpoints into the same coordinate system. Since there is no information about the relative positional relationship between the depth maps before registration, the coarse registration algorithm is usually developed around how to establish the matching relationship of corresponding elements between two depth maps at the same position of the model, and many scholars propose different methods for this. These methods can be divided into point-to-point correspondence, line-to-line correspondence, plane-to-plane correspondence, and body-to-body correspondence according to the difference in corresponding elements; if the matching modes are different, the equal relation based on Euclidean distance and the equal relation based on descriptors can be divided; the search range by matching can be divided into non-feature matching and feature matching. However, due to the diversity and complexity of the measurement model, the coarse registration algorithm is often dependent on some specific applications, and further research on the coarse registration algorithm aims to improve the accuracy, efficiency, robustness and applicability of the coarse registration. The result of the coarse registration typically provides only an approximation of the exact coordinate transformation between the depth maps, so that the overlapping regions of the two depth maps have a certain degree of fit. However, due to the approximate position transformation, the overlapping regions of the depth maps are difficult to be accurately fitted, some interleaving and layering phenomena often exist, and the subsequent fusion of the depth map data is not facilitated, so that the position of the depth map needs to be further adjusted to improve the registration accuracy of the depth map, and the process is called as fine registration. The fine registration is to realize more accurate registration transformation by continuously iterating and minimizing the distance between corresponding points on two depth maps on the basis that the approximate value of the coordinate transformation between the depth maps is obtained by coarse registration, and the representative algorithm is the classic ICP algorithm.
Disclosure of Invention
The invention provides an automatic registration method based on feature information change degree, aiming at the problems of identification degree of feature description of a three-dimensional point cloud automatic registration midpoint, registration precision and robustness. The method includes the steps that point cloud data are simplified by introducing Sine of Normal (SiN) on the basis of solution vector, a feature point set is extracted, so that the subsequent calculation amount is reduced, corresponding point pairs are searched through three geometric features, effective accurate matching point pairs are obtained by a method of combining rigid distance constraint and a RANSAC algorithm and used for calculating initial registration parameters, and finally, secondary splicing is conducted through an improved ICP algorithm. The method can improve the precision and speed of registration and has higher robustness.
The invention adopts the following technical scheme for solving the technical problems:
the invention provides a three-dimensional point cloud automatic registration method based on characteristic information change degree, which comprises the following steps:
step 1, acquiring feature point sets Pt and Qt of two point cloud data P and Q to be registered based on normal vector information of different neighborhood radii, wherein Q is reference point cloud;
step 2, establishing the feature description of each feature point in Pt and Qt;
step 3, based on the characteristic description of Pt and Qt obtained in step 2, finding the most similar point for each point in Pt in Qt as a matching point to obtain an initial matching point pair set;
step 4, combining the initial matching point pair set obtained in the step 3 with the distance constraint condition between points, and using a self-adaptive RANSAC algorithm to obtain an accurate matching point pair set;
step 5, calculating an initial registration parameter by using a four-element method;
and 6, performing point cloud registration by adopting an improved iterative closest point ICP algorithm.
As a further technical solution of the present invention, step 1 specifically comprises:
step 1.1, the characteristic degree of a certain point p in the point cloud data is
Where θ is p in different neighborhood radius r
1、r
2Normal vector of
Angle of (d) r
1≠r
2;
Step 1.2, extracting characteristic degree of point cloud data greater than set threshold epsilon1The points of (2) are used as characteristic points of the point cloud data;
step 1.3, feature point extraction is performed on P and Q by the methods of step 1.1 and step 1.2, respectively, and the set of feature points of P is obtained as Pt ═ { Pt ═ Pt1,pt2,pt3,…ptm'Q has a feature point set of Qt ═ Qt1,qt2,qt3,…qtn'Where m 'and n' are the numbers of characteristic points of P and Q, respectively.
As a further technical solution of the present invention, step 2 specifically is:
step 2.1, for the ith' point Pt in Pti′The feature degree of the point is taken as the first feature quantity f at the point1(pti′);
Step 2.2: for pti′In P with pti′The point in the spherical region having the origin and the radius of γ is denoted as pti′To calculate pti′Neighborhood center of gravity O (pt)i′) In pti′To O (pt)i′) As the second characteristic quantity f of the point2(pti′)=||O(pti′)-pti′||;
Step 2.3: for pt
i′In pt
i′And O (pt)
i′) The line between pt and
i′taking the inverse cosine value of the included angle of the normal vector as the third characteristic quantity of the point
Step 2.4: for pti′If there is a jth point Qt at Qtj′Meet the following three conditions, then qtj′Is pti′The corresponding points of (1):
|f1(pti′)-f1(qtj′)|/(f1(pti′)+f1(qtj′))≤ε2
|f2(pti′)-f2(qtj′)|/(f2(pti′)+f2(qtj′))≤ε3
|f3(pti′)-f3(qtj′)|/(f3(pti′)+f3(qtj′))≤ε4
wherein epsilon2、ε3、ε4All are set thresholds.
As a further technical solution of the present invention, step 3 specifically is:
based on the information obtained in step 2, finding a corresponding point for each point in Pt in Qt as its initial matching point, and the initial matching point pair is set as:
where num (matchdots) is the number of initial matching point pairs.
As a further technical solution of the present invention, step 4 specifically is:
step 4.1: for point pairs in Matchdots
And
if it satisfies
Then
And
according to the constraint condition of the distance between the points, and the sum in Matchdots is calculated
Number num of matching point pairs satisfying distance constraint between points
gWherein h is 1,2,3 … num (Matchdots), dist (·,) represents the distance between two points, ε
5Setting a threshold;
step 4.2: selecting the matching point pair corresponding to the larger N values in the calculation result of the step 4.1, and checking whether the N matching point pairs are correct matching point pairs by using a RANSAC algorithm, wherein the specific checking method comprises the following steps:
1) randomly selecting 3 matching point pairs from the N matching point pairs as a sample;
2) setting 3 selected matching point pairs in the step 1) as correct matching point pairs, and calculating a rigid body transformation matrix U;
3) judging whether the rest N-3 matching points are correctly matched under the rigid body transformation matrix U in the step 2): for any point(s) in the remaining N-3 matching points1,s2) If | | | U(s)1),s2If | is less than the predefined threshold value, then(s) is considered1,s2) If the matching point pair is correct, marking as an inner point; otherwise,(s)1,s2) The matching point pairs are wrong and are marked as outer points, and all inner points form a consistent set of the sampling; wherein, U(s)1) Denotes s1Calculating a new coordinate point through a rigid body transformation matrix;
4) repeating the steps 1) to 3) until the iteration times reaches the upper limit of the sampling times, finally, taking the rigid body transformation matrix corresponding to the consistent set with the maximum number of the internal points obtained by iteration as a correct rigid body transformation matrix, and obtaining the internal points in Matchdots as final accurate matching point pairs according to the correct rigid body transformation matrix, wherein the accurate matching point pair set is as follows:
Matchdots1={(me,m'e)|me∈P,m'e∈Q,e=1,2,3…t}
wherein (m)e,m'e) To exactly match a pair of points, t is the number of exactly matching pairs of points.
As a further technical solution of the present invention, the calculation of the initial registration parameter in step 5 specifically includes:
1): separately compute a set of points { me|meE.g. P, e 1, 2.e|m'eE.g., centroid of Q, e 1, 2.
Wherein μ, μ' are each point me,m'eThe center of mass of;
2): respectively set points { me|meE.g. P, e 1, 2.e|m'eE Q, e 1,2, t, do a translation relative to the centroid:
wherein s ise,s'eAre respectively a point me,m'eNew feature points after the relative centroid translation;
3): according to the point set seAnd { s'eComputing a correlation matrix K:
4): by the elements K in the correlation matrix KbcConstructing a four-dimensional symmetric matrix Ka:
Wherein b, c is 1,2,3, 4;
5): calculating KaThe unit feature vector q corresponding to the maximum feature root:
q=[q0q1q2q3]T
6): calculating a rotation matrix R0:
7): solving translation vector T0:
T0=μ'-Rμ。
As a further technical solution of the present invention, step 6 specifically is:
step 6.1, utilizing the initial registration parameter R obtained in step 50And T0Transforming each point in the target point cloud P to a coordinate system of the reference point cloud Q, and obtaining an initial target point cloud set P in the ICP algorithm after transformation0={p01,p02,…,p0mIn which p is0i=R0·pi+T0,piIs the ith point in the target point cloud P, i is 1,2, …, m;
step 6.2, for P0At any point in the four points, the gravity center P of a tetrahedron formed by four points with the nearest Euclidean distance on Q is obtained through a polyhedron gravity center formula0The ith point and its corresponding center of gravity constitute the initial closest point pair (p) in the ICP algorithm0i,q0i);
Step 6.3, according to an iterative formula P
s+1=R
s+1P
s+T
s+1Performing iterative calculation until a set convergence condition d is met
s-d
s+1If the registration parameter is less than E, the obtained final registration parameter is used for registration of the target point cloud P to obtain an accurate registration result; wherein E is a set threshold value,
(p
(s-1)i,q
(s-1)i)、(p
si,q
si) Respectively the s-1 th iteration and the nearest neighbor point pair, R, obtained by the registration parameter obtained by the iteration after the s iteration according to the method in the steps 6.1 and 6.2
s、R
s+1Respectively obtaining rotation matrixes after the s-th iteration and the s + 1-th iteration; t is
s、T
s+1Respectively obtaining translation vectors after the s-th iteration and the s + 1-th iteration; p
s、P
s+1Respectively after s iterations and s +1 iterations of the target point cloud PPoint cloud collection of d
s、d
s+1Are respectively P
s、P
s+1And the average distance of Q.
Has the advantages that:
compared with other feature-based registration algorithms which only use a single feature value (such as curvature, gravity center distance and the like) to describe the point cloud, the invention adopts various geometric feature features to describe the point cloud, so that the point identification degree is improved, the anti-noise interference capability of the method is improved, the influence of the point cloud topological structure is small, and the registration precision and speed are improved to a certain extent. Compared with other feature-based registration techniques, the present invention has the following advantages:
firstly, the method selects the characteristic point set based on the change degree of the normal vectors of different neighborhood radiuses of the point cloud instead of using all the points for point pair matching, saves the processing time, removes the points with unobvious relative characteristics, improves the accuracy of point pair matching, and avoids the occurrence of wrong matched point pairs caused by too many similar point matches.
Secondly, various geometric characteristics are designed to carry out characteristic point pair acquisition, the identification degree of characteristic description is improved, the anti-noise interference capability of the characteristics is strong, and the influence of a point cloud topological structure is small, so that the accuracy of point pair matching is effectively improved.
And finally, combining rigid distance constraint and RANSAC algorithm to obtain accurate matching point pairs, and further iteratively solving an accurate registration result by using an improved ICP algorithm, wherein the algorithm is quick and effective.
In conclusion, the method can realize automatic registration of partially overlapped point cloud data, the point cloud does not need any predicted information, manual intervention is not needed in the registration process, the initial registration effect is good, the secondary registration result is more accurate, and the registration requirement of multi-view point cloud data is met.
Drawings
FIG. 1 is a flow chart of the overall process of the present invention.
FIG. 2 is a schematic diagram of the bunny point cloud with normal information in whole and in part.
FIG. 3 is a schematic diagram of bunny point cloud feature point extraction.
FIG. 4 is a schematic diagram of two pairwise view-angle initial matching point-to-connecting lines of the bunny point cloud.
FIG. 5 is a schematic diagram of a connecting line of matching point pairs of two pairwise viewing angles of the bunny point cloud after removing error point pairs.
Fig. 6 is the initial registration result of two view angles of bunny point cloud.
Fig. 7 is a comparison diagram of the initial registration and the accurate registration result of two viewing angles of the bunny point cloud.
Detailed Description
The following further describes the embodiments of the present invention with reference to the drawings. And selecting Visual Studio2017 as a programming tool under a Windows operating system, and performing registration processing on multi-view point cloud data acquired by the three-dimensional measuring equipment. In the example, the bunny point cloud data of Stanford university is adopted, and a relatively accurate registration result is finally obtained.
FIG. 1 is a flow chart of the overall process of the present invention.
The invention provides a point cloud automatic registration algorithm based on feature information change degree, which aims at the problems of low feature identification degree, sensitivity to noise, high requirements on a point cloud topological structure and the like in the existing feature-based automatic registration algorithm. On the basis of solving vector, point cloud data is simplified by introducing Sine of Normal (SiN) to extract a feature point set, so that the subsequent calculation amount is reduced, corresponding point pairs are sought through three geometric features, then accurate matching point pairs are obtained by utilizing rigid distance constraint conditions and applying RANSAC algorithm, initial registration parameters are calculated by adopting a four-element method, and finally, the point cloud is accurately registered by adopting an improved ICP algorithm.
The method comprises the following specific implementation steps:
step 1: the method comprises the following steps of obtaining feature point sets in two point clouds to be matched based on normal vector information of different neighborhood radiuses:
step 1.1: the existing three-dimensional scanner is adopted to obtain multi-view point cloud data with normal vector information, and overlapping parts exist between point clouds obtained from adjacent views, so that the point p of a certain point in the point cloud can be calculated at different positionsNeighborhood radius r
1、r
2(r
1≠r
2) Normal vector of
Defining the characteristic degree f of the point p as its normal vector
Absolute value of the included angle sine | sin θ |:
step 1.2: selecting a suitable threshold value epsilon1Removing the relatively flat part in the point cloud, and reserving f > epsilon1Point (2) of (c).
Step 1.3: setting two point cloud data to be registered as P and Q respectively, wherein Q is a reference point cloud, and respectively extracting the feature points of the two point clouds by using the feature point extraction methods provided in the step 1.1 and the step 1.2 to obtain a feature point set Pt ═ Pt of P1,pt2,pt3,…ptm'Q has a feature point set of Qt ═ Qt1,qt2,qt3,…qtn'Where m 'and n' are the numbers of characteristic points of P and Q, respectively.
FIG. 2 is a schematic diagram of the bunny point cloud with normal information in whole and in part.
FIG. 3 is a schematic diagram of bunny point cloud feature point extraction.
Step 2: establishing the feature description of the feature point set, wherein the method comprises the following steps:
step 2.1: for the ith' point Pt in the point set Pti′The feature degree of the point is taken as the first feature quantity f at the point1(pti′);
Step 2.2: for the ith' point Pt in the point set Pti′In the origin set P, pti′The point in the spherical region having the origin and the radius of γ is denoted as pti′To calculate pti′Neighborhood center of gravity O (pt)i′) In pti′To O (pt)i′) The distance value of (2) is taken as the second special characteristic of the pointCharacterization, noted as:
f2(pti′)=||O(pti′)-pti′||
step 2.3: for the ith' point Pt in the point set Pt
i′In pt
i′And O (pt)
i′) The line between pt and
i′normal vector
The inverse cosine value of the included angle of (b) is taken as a third characteristic quantity of the point and is recorded as:
step 2.4: for the ith' point Pt in the point set Pti′For it, search for the corresponding point in the set of points Qt, if Qtj′Is pti′Should the three geometrical characteristics proposed above be the same or approximately the same for the two points, and therefore for Pt in the point set Pti′And Qt in the set of points Qtj′We consider these three conditions to have correspondence as long as they are satisfied, and the three conditions are as follows:
|f1(pti′)-f1(qtj′)|/(f1(pti′)+f1(qtj′))≤ε2
|f2(pti′)-f2(qtj′)|/(f2(pti′)+f2(qtj′))≤ε3
|f3(pti′)-f3(qtj′)|/(f3(pti′)+f3(qtj′))≤ε4
wherein take epsilon2=ε3=ε4And (5) filtering by using the three relations and initially establishing the point correspondence relation, wherein the point correspondence relation is 0.01.
And step 3: based on the information obtained in step 2, finding a corresponding point for each point in Pt in Qt as its initial matching point, and the initial matching point pair is set as:
where num (matchdots) is the number of initial matching point pairs.
FIG. 4 is a schematic diagram of two pairwise view-angle initial matching point-to-connecting lines of the bunny point cloud.
And 4, step 4: combining the distance constraint condition between points, using a self-adaptive RANSAC algorithm to obtain an accurate matching point pair, and the method specifically comprises the following steps:
step 4.1: any two point pairs in Matchdots
And
if all are correct matching point pairs, then it is easy to get the following according to the constraint condition of distance between points:
since it is difficult to find the exact corresponding point pair in two discrete point clouds, the correct matching point pair here is only an approximate corresponding point pair, i.e. it only needs to satisfy:
selecting an appropriate ε
5>0(ε
5Taken as 0.02), for point pairs in Matchdots
And
if it satisfies
Then
And
and meeting the constraint condition of the distance between the points.
To pair
Calculating the number num of point pairs of Matchdots, except for the point pairs meeting the point-to-point distance constraint condition
g。
Step 4.2: because a certain number of correct matching point pairs exist in the matching point pairs obtained after the initial matching, in general, the number num of the point pairs conforming to the distance constraint is obtained by the correct matching point pairsgIs relatively large; the num proposed above is calculated for each point pairgValue, then in numgSorting Matchdots from large to small, and selecting the first N point pairs; the calculation of the rigid body transformation matrix requires at least 3 pairs of matching points which are not collinear in the three-dimensional space.
The RANSAC algorithm is used herein to check whether the N point pairs are correct matching point pairs, where N is generally 50 in the experiment, and the specific checking method is as follows:
1) randomly selecting 3 point pairs from the N point pairs as a sample;
2) considering the 3 point pairs as correct matching point pairs, and calculating a rigid body transformation matrix U;
3) judging whether the rest N-3 point pairs are correctly matched under the rigid body transformation matrix U: for any point(s) in the remaining N-3 matching points1,s2) If | | | U(s)1),s2If | is less than a predefined threshold (this threshold is typically set to 2 times the average distance between points of the point cloud), then(s) is considered1,s2) If the matching point pair is correct, marking as an inner point; otherwise,(s)1,s2) And the matching point pairs are wrong and are marked as outer points, and all the inner points form a consistent set of the sampling. Wherein, U(s)1) Denotes s1And calculating a new coordinate point through the rigid body transformation matrix.
4) Repeating the steps 3 until the cycle times reach the upper limit of the sampling times, and finally taking the rigid body transformation matrix corresponding to the consistent set with the maximum number of the internal points as a correct rigid body transformation matrix. And obtaining inner points in Matchdots as final accurate matching point pairs according to the transformation matrix, wherein the accurate matching point pairs are recorded as:
Matchdots1={(me,m'e)|me∈P,m'e∈Q,e=1,2,3…t}
wherein t is the number of precisely matched point pairs;
fig. 5 is a schematic diagram of the effect of the matching point pairs obtained after distance constraint of two pairwise viewing angle matching point pairs of bunny.
And 5: calculating an initial registration parameter by using a four-element method according to the accurate matching point pair obtained in the step 4, and specifically comprising the following steps:
1): separately compute a set of points { me|meE.p, e 1,2, n, and m'e|m'eE.g., centroid of Q, e 1, 2.
Wherein μ, μ' are each point me,m'eThe center of mass of;
2): respectively set points { me|meE.p, e 1,2, n, and m'e|m'eE Q, e 1,2, n, do a translation relative to the centroid:
wherein s ise,s'eAre respectively a point me,m'eNew feature points after the relative centroid translation;
3): according to the point set seAnd { s'eComputing a correlation matrix K:
4): by the elements K in the correlation matrix KbcConstructing a four-dimensional symmetric matrix Ka:
Wherein b, c is 1,2,3, 4;
5): calculating KaThe unit feature vector q corresponding to the maximum feature root:
q=[q0q1q2q3]T
6): calculating a rotation matrix R0:
7): solving translation vector T0:
T0=μ'-Rμ。
Fig. 6 is a result diagram obtained after initial registration of the bunny four-view point cloud image, and it can be seen from the result diagram that a satisfactory result can be obtained by the initial registration of the method.
Step 6: the method adopts an improved ICP algorithm to accurately register the point cloud, and comprises the following specific steps:
step 6.1, utilizing the initial registration parameter R obtained in step 50And T0Transforming each point in the target point cloud P to a coordinate system of the reference point cloud Q, and obtaining an initial target point cloud set P in the ICP algorithm after transformation0={p01,p02,…,p0mIn which p is0i=R0·pi+T0,piIs the ith point in the target point cloud P, i is 1,2, …, m;
step 6.2, for P0At any point in the four points, the gravity center P of a tetrahedron formed by four points with the nearest Euclidean distance on Q is obtained through a polyhedron gravity center formula0The ith point and its corresponding center of gravity constitute the initial closest point pair (p) in the ICP algorithm0i,q0i);
Step 6.3, according to an iterative formula P
s+1=R
s+1P
s+T
s+1Performing iterative computation onUntil a set convergence condition d is satisfied
s-d
s+1If the registration parameter is less than E, the obtained final registration parameter is used for registration of the target point cloud P to obtain an accurate registration result; wherein E is a set threshold (E is 0.001 in the present invention),
(p
(s-1)i,q
(s-1)i)、(p
si,q
si) Respectively the s-1 th iteration and the nearest neighbor point pair, R, obtained by the registration parameter obtained by the iteration after the s iteration according to the method in the steps 6.1 and 6.2
s、R
s+1Respectively obtaining rotation matrixes after the s-th iteration and the s + 1-th iteration; t is
s、T
s+1Respectively obtaining translation vectors after the s-th iteration and the s + 1-th iteration; p
s、P
s+1Respectively is a point cloud set of the target point cloud P after s iterations and s +1 iterations, d
s、d
s+1Are respectively P
s、P
s+1And the average distance of Q.
Fig. 7 is a comparison graph of results obtained after initial and accurate registration of bunny two-view point cloud images, and the effectiveness and good registration effect of the method can be seen from the comparison graph.
Based on the normal vector information of the point cloud, the invention selects points with larger local normal vector change as feature points, utilizes three kinds of geometric information as feature description, sets and compares the similarity degree of the feature points through a threshold value to obtain initial matching point pairs, obtains accurate matching point pairs by using a method of combining rigid distance constraint and RANSAC algorithm, obtains initial registration parameters by using a four-element method, and further obtains accurate registration results by using improved ICP algorithm iteration.
The above description is only an embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can understand that the modifications or substitutions within the technical scope of the present invention are included in the scope of the present invention, and therefore, the scope of the present invention should be subject to the protection scope of the claims.