CN114037743A - A Robust Registration Method for 3D Point Clouds of Terracotta Warriors Based on Dynamic Graph Attention Mechanism - Google Patents

Abstract

The invention discloses a method for robust registration of three-dimensional point clouds of terracotta figurines based on a dynamic graph attention mechanism. Net network, replace the convolutional layer with B‑NHN‑Conv, replace the deconvolutional layer with B‑NHN‑ConvTr, and embed the residual module and dynamic graph attention mechanism into the U‑Net network to get the point Cloud registration network; step 3, input the 3D point cloud of Terracotta Warriors with different resolutions into the point cloud registration network, and train it under the supervision of the circle loss loss function and the overlap loss loss function; step 4, use the training The completed point cloud registration network extracts the 3D point cloud features of the Terracotta Warriors, and combines the RANSAC algorithm to estimate the change matrix between the source point cloud and the target point cloud to complete the registration of the Terracotta Warriors 3D point cloud. The registration method provided by the present invention can still learn robust features under the condition that the resolution of the point cloud does not match and contains a lot of noise, and can better complete the registration of the point cloud under low overlap.

Description

A Robust Registration Method for 3D Point Clouds of Terracotta Warriors Based on Dynamic Graph Attention Mechanism

technical field

The invention relates to a three-dimensional point cloud model registration technology, in particular to a method for robust registration of three-dimensional point clouds of Qin figurines based on a dynamic graph attention mechanism.

Background technique

The point cloud registration technology plays an important role in projects such as the virtual restoration of the Terracotta Warriors and the Qinling Wisdom Museum. The accuracy of the point cloud registration results is the key to the subsequent 3D reconstruction. The purpose of point cloud registration is to obtain a perfect coordinate transformation through calculation, and to uniformly integrate point cloud data from different perspectives into a specified coordinate system through rigid transformations such as rotation and translation.

At present, most of the registration methods of Qin figurines and related cultural relics are traditional registration methods based on optimization, among which the more classic method is the Iterative Closest Point algorithm. The method mainly consists of two stages: correspondence search and transform estimation. These two stages are repeated to find the best transition between point clouds. However, when dealing with scenes with large initial position differences, mismatched point cloud resolutions, strong noise interference, and small overlap, the above methods are prone to fall into local optimum. In recent years, some researchers have proposed methods based on deep learning to learn robust feature calculation corresponding points, and finally determine the transformation matrix through RANSAC or SVD algorithm without iteration between correspondence estimation and transformation estimation. However, such algorithms cannot be fast and robust in the face of challenges such as partial overlap, density variation, and noise.

SUMMARY OF THE INVENTION

In view of the deficiencies in the prior art, the purpose of the present invention is to provide a robust registration method for 3D point clouds of Terracotta Warriors based on dynamic graph attention mechanism, which can robustly handle partial overlap and density changes.

In order to achieve the above object, the present invention adopts the following technical solutions to realize:

A robust registration method for 3D point cloud of Terracotta Warriors based on dynamic graph attention mechanism, comprising the following steps:

Step 1, obtain 3D point clouds of Terracotta Warriors with different resolutions through a 3D scanner;

Step 2, in the U-Net network, replace the convolutional layer with B-NHN-Conv, replace the deconvolutional layer with B-NHN-ConvTr, and embed the residual module and dynamic graph attention mechanism into U- Net network, get point cloud registration network;

Step 3: Input the three-dimensional point clouds of the Terracotta Warriors with different resolutions into the point cloud registration network, and train the constructed point cloud registration network under the joint supervision of the circle loss loss function and the overlap loss loss function;

Step 4: Use the trained point cloud registration network to extract the 3D point cloud features of the Terracotta Warriors, and combine the RANSAC algorithm to estimate the change matrix between the source point cloud and the target point cloud to complete the registration of the Terracotta Warriors 3D point cloud.

Further, the U-Net network in the step 2 also includes an encoder module and a decoder module, and the encoder module obtains multiple scale features through multiple downsampling, and fully connects all the multi-layer perceptrons. The layers are converted into a series of fully convolutional layers with a kernel size of 1 × 1 × 1, and the number of channels is (64, 128, 256, 512); each time the decoder module is up-sampled, the skip connection structure and the feature extraction part correspond to the same scale of the number of channels. The features are fused to restore the information loss caused by downsampling.

Further, the residual module in the step 2 is formed by connecting a plurality of residual blocks in series, and is used to jump-connect the input information to the output information.

Further, in the step 2, the dynamic graph attention mechanism includes a combination module of a multi-layer self-attention module and a cross-attention module, and the dynamic graph attention mechanism finds the top k for each query node based on the attention weight a _ij . edges, and only the first k edges and corresponding nodes are used to construct the graph, and a new graph is constructed according to the changed attention weights a _ij at each layer of the dynamic graph attention mechanism.

Compared with the prior art, the present invention has the following technical effects:

The present invention uses U-Net as the backbone architecture, and solves the problem of gradient disappearance or gradient explosion by embedding residual blocks; by embedding dynamic graph attention mechanism to aggregate local features and context features of point clouds, multi-level and richer features can be obtained. Semantic representation can also better help the network determine possible overlapping areas between point clouds; and reduce or remove potential changes in feature mean and standard deviation through the B-NHN-Conv convolution operation, which improves the sensitivity of 3D features to point density changes. Robustness; this enables the point cloud registration network constructed in the present invention to learn robust features and better complete the registration of point clouds under low overlap even when the resolution of point clouds does not match and contains a lot of noise .

Further, converting all fully connected layers in a multilayer perceptron into a series of fully convolutional layers with a kernel size of 1 × 1 × 1 can speed up the efficiency of network processing.

Description of drawings

1 is a flowchart of a method for robust registration of three-dimensional point clouds of Terracotta Warriors based on a dynamic graph attention mechanism of the present invention;

Fig. 2 is the network model diagram of point cloud registration;

Figure 3 is a schematic structural diagram of a dynamic graph attention mechanism;

FIG. 4 is a schematic structural diagram of a residual module;

Figure 5 is the initial pose diagram of the point cloud of the two heads of the Qin figurines;

Figure 6 is the result of point cloud registration of the two heads of the Qin figurines;

Figure 7 is the initial pose diagram of the point cloud of the two feet of the Qin figurines;

Figure 8 shows the result of point cloud registration on the feet of the Qin figurines.

Detailed ways

The specific content of the present invention will be further explained in detail below in conjunction with the embodiments.

1-4, this embodiment provides a method for robust registration of 3D point clouds of Terracotta Warriors based on a dynamic graph attention mechanism, including the following steps:

Step 1, obtain 3D point clouds of Terracotta Warriors with different resolutions through a 3D scanner;

Step 2, in the U-Net network, replace the convolutional layer with B-NHN-Conv, replace the deconvolutional layer with B-NHN-ConvTr, and embed the residual module and dynamic graph attention mechanism into U- Net network, the point cloud registration network is obtained; among them,

B-NHN-Conv is a combination of B-NHN normalization and subsequent three-dimensional sparse convolution operations, and the B-NHN normalization and three-dimensional sparse convolution are tightly coupled; B-NHN-ConvTr is a combination of B- The NHN-Conv convolution operation is transposed, that is, the B-NHN normalization operation is combined with the three-dimensional sparse transpose convolution function; the B-NHN normalization operation can reduce or remove the potential changes in the feature mean and standard deviation, and improve the three-dimensional Robustness of features to changes in point density.

The residual module is composed of multiple residual blocks in series, which is used to jump connect the input information to the output information; it can alleviate the problem of the loss of effective feature information, make the network easier to optimize, and help to establish the relationship between the two point clouds. Correspondence.

The dynamic graph attention mechanism includes a combination of multiple layers of self-attention and cross-attention, also known as dynamic graph attention. The dynamic graph attention mechanism finds the top k edges for each query node based on the attention weights a _ij , and uses only the top k edges and corresponding nodes to construct the graph, and uses the dynamic graph attention mechanism at each layer of the dynamic graph attention mechanism. Construct a new graph according to the changed attention weights a _ij .

键向量

和值向量

其中

表示实数集，b为特征的维数，用于图结构更新和注意力聚合，如下公式所示：The dynamic graph attention mechanism first calculates the query vector for the feature nodes in the graph through linear projection

key vector

sum value vector

in

Represents the set of real numbers, and b is the dimension of the feature, which is used for graph structure update and attention aggregation, as shown in the following formula:

q _i =W ₁ ^(l) f _i ^Q +b ₁

Among them, {Q,S}∈{X,Y} ² , when Q=S, it means self-attention; when Q≠S, it means cross-attention; W and b are learnable projection parameters, ^(l) f _i ^Q represents the feature of the key point located at the l layer of the dynamic graph attention module in the point cloud Q, which can be represented as a query node, and all nodes in the point cloud S can be represented as source nodes.

By calculating the similarity or correlation between qi and each k _j , the weight coefficient α _{ij of each k j corresponding to v j is obtained, and then the weighted summation of v j is performed ,} that is, the final Attention value is obtained. value, the calculation formula is as follows:

表示特征(l)f_i ^Q对特征

的关注程度；ε∈{ε_self,ε_cross}。where the weight factor

Represents feature (l)f _i ^Q pair feature

the degree of attention; ε∈{ε _self ,ε _cross }.

Once all layers are aggregated, the final features of nodes can be expressed as:

f _i =Wf _i +b

The U-Net network also includes an encoder module for extracting features and a decoder module for fusing features, the encoder module obtains multiple scale features through multiple downsampling, and combines the multi-layer perceptron All fully connected layers are converted into a series of fully convolutional layers with a kernel size of 1×1×1, and the number of channels is (64, 128, 256, 512). The features of the same scale are fused to restore the information loss caused by downsampling.

Step 3: Input the 3D point clouds of the Terracotta Warriors with different resolutions into the point cloud registration network, and train the constructed points under the joint supervision of the circle loss function (cycle loss function) and the overlap loss function (overlap loss function). cloud registration network;

Circle loss is a variant of triplet loss commonly used in point cloud registration. The loss function formula is as follows:

表示特征空间中两特征间的距离，Δ_x和Δ_n分别表示正负样本间隔，权重

由超参数γ、

以及样本正样本间隔Δ_x所决定，同样权重

由超参数γ、

以及样本负样本间隔Δ_n所决定，

计算方法同理，所以circle loss最终的损失函数为

Among them, the overlapping point cloud pairs X and Y have been aligned, n _x represents the number of point clouds randomly sampled from the point cloud X, ε _x (x _i ) represents the point x _i as the center, and all points within the radius r _x belong to the point cloud The point of Y, ε _n (x _i ) represents all the points belonging to the point cloud Y outside the radius r _x with the point x _i as the center,

Represents the distance between two features in the feature space, _Δx and _Δn represent the positive and negative sample intervals, respectively, and the weight

By the hyperparameter γ,

And the sample positive sample interval Δ _x is determined, the same weight

By the hyperparameter γ,

And the sample negative sample interval _Δn is determined,

The calculation method is the same, so the final loss function of circle loss is

The estimates of overlapping regions are converted to binary classification, supervised using overlap loss, and the loss function is as follows:

表示点x_i是否为重叠区域，

表示网络预测的标签。

计算方法同理。Among them, the true label

Indicates whether point _xi is an overlapping area,

Represents the label predicted by the network.

The calculation method is the same.

Step 4: Input the source point cloud and the target point cloud into the trained point cloud registration network, pass through the encoder module, the dynamic graph attention module in turn, and finally output the extracted features through the decoder module, and calculate with the RANSAC algorithm The transformation matrix completes the registration.

Figure 5 and Figure 7 are respectively the initial pose map of the point cloud of the two heads of the Qin figurines and the initial pose map of the point clouds of the two feet of the Qin figurines obtained by the 3D scanner; The registration result of the point cloud of the two heads of the Qin figurines and the point cloud registration result of the feet of the Qin figurines after the registration. It can be seen from the figure that the method of the present invention can still handle robustly when the point cloud is partially overlapped and the density changes.

Claims (4)

1. A three-dimensional point cloud robust registration method for Qin warriors based on a dynamic graph attention mechanism is characterized by comprising the following steps:

step 1, acquiring three-dimensional point clouds of Qinhong figures with different resolutions through a three-dimensional scanner;

step 2, replacing the convolution layer with B-NHN-Conv, replacing the deconvolution layer with B-NHN-ConvTr, and embedding a residual module and a dynamic graph attention mechanism into the U-Net network to obtain a point cloud registration network;

step 3, inputting three-dimensional point clouds of Qin warriors with different resolutions into a point cloud registration network, and training the constructed point cloud registration network under the joint supervision of a circle loss function and an overlap loss function;

and 4, extracting three-dimensional point cloud characteristics of the Qin warriors by using the trained point cloud registration network, and estimating a change matrix between the source point cloud and the target point cloud by combining a RANSAC algorithm to complete registration of the three-dimensional point cloud of the Qin warriors.

2. The robust registration method for the three-dimensional point cloud of Qin warriors based on the dynamic graph attention mechanism as claimed in claim 1, wherein the U-Net network in step 2 further comprises an encoder module and a decoder module, the encoder module obtains a plurality of scale features through a plurality of downsampling, and converts all full connection layers in the multi-layer sensor into a series of full convolution layers with the kernel size of 1 x 1, and the number of channels is (64,128,256, 512); and the decoder module fuses the features with the same scale and the number of corresponding channels through the jump connection structure and the feature extraction part every time sampling is carried out, so that the information loss caused by down-sampling is reduced.

3. The robust registration method for the three-dimensional point cloud of the Qin warriors based on the dynamic graph attention mechanism as claimed in claim 1, wherein the residual error module in step 2 is formed by connecting a plurality of residual error blocks in series, and is used for jumping and connecting the input information to the output information.

4. The robust registration method for the three-dimensional point cloud of the Qin warriors based on the dynamic graph attention mechanism as claimed in claim 1, wherein the dynamic graph attention mechanism in the step 2 comprises a combination module of a multilayer self-attention module and a cross-attention module, and the dynamic graph attention mechanism is based on the attention weight a_ijFinding the first k edges for each query node, and constructing a graph using only the first k edges and corresponding nodes, and based on the changed attention weight α at each level of the dynamic graph attention mechanism_ijA new graph is constructed.

Images