CN112488117B - Point cloud analysis method based on direction-induced convolution - Google Patents

Abstract

The invention discloses a point cloud analysis method based on direction-induced convolution. , construct the residual direction induced convolution module and the farthest point sampling residual direction induced convolution module; construct the direction induced convolution network according to the residual direction induced convolution module and the farthest point sampling residual direction induced convolution module; The input direction of point cloud data induces a convolutional network to obtain point cloud segmentation results and classification results. The present invention better captures the local spatial structure of the point cloud in an end-to-end manner, and improves the accuracy of point cloud classification and point cloud segmentation.

Description

A point cloud analysis method based on direction-induced convolution

technical field

The invention belongs to 3D point cloud analysis technology, in particular to a point cloud analysis method based on direction-induced convolution.

Background technique

As a simple and direct representation of 3D data, point cloud can well describe the geometric and topological information of objects in 3D space, and is widely used in understanding the surrounding environment. In recent years, point cloud analysis has attracted more and more attention. Unlike general images, point clouds in 3D space have an irregular and disordered topology, making it difficult to apply standard convolution operations. To remedy this problem in terms of data representation, some works directly perform convolution processing on regular grids by transforming raw point cloud data into a set of 2D images or 3D voxel representations. However, these transformations usually lose a lot of intrinsic geometric information, and at the same time have high complexity.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a point cloud analysis method based on direction-induced convolution.

The technical solution for realizing the object of the present invention is: a point cloud analysis method based on direction-induced convolution, comprising the following steps:

Step 1. Constructing a direction-induced convolution module, the direction-induced convolution module is used to extract the features of the disordered neighborhood of the point cloud;

Step 2. Based on the direction-induced convolution module, construct the residual direction-induced convolution module and the farthest point sampling residual direction-induced convolution module;

Step 3. Construct a direction-induced convolution network according to the residual direction-induced convolution module and the farthest point sampling residual direction-induced convolution module;

Step 4: Input the point cloud data into the direction-induced convolution network to obtain point cloud segmentation results and classification results.

Preferably, the specific steps that the direction-induced convolution module is used to extract the features of the disordered neighborhood of the point cloud are:

Calculate the spatial direction features of the local neighborhood points of the point cloud;

Calculate and normalize the spatial direction similarity weight between the neighborhood points and the items of the spatial direction set;

Convert the features of point cloud disordered neighborhood points into the direction set space;

In the orientation set space, standard convolution is used to encode the spatially transformed features projected onto the items of the spatial orientation set to complete feature extraction of the unordered neighborhood of the point cloud.

Preferably, the calculation formula of the spatial direction similarity weight is:

为每个点的领域点的空间方向特征，每项x_m∈D为3D空间中的一个方向。In the formula,

is the spatial orientation feature of the domain point for each point, and each term x _m ∈ D is an orientation in 3D space.

的空间方向特征

编码为：Preferably, the neighborhood points of each point p _i

The spatial orientation characteristics of

Encoded as:

Preferably, the feature transformation is expressed as:

是邻域点

的特征；

代表串接；

表示归一化的空间方向相似性；

表示邻域点

的相对位置编码，N(p_i)表示以点p_i为中心的局部点云区域。in,

is the neighborhood point

Characteristics;

represents concatenation;

represents the normalized spatial orientation similarity;

Represents a neighborhood point

The relative position encoding of , N( _pi ) represents the local point cloud area centered on point _pi .

Preferably, the specific process of using standard convolution to encode the spatial transformation features projected onto the items of the spatial direction set is as follows:

表示所有邻域点特征在方向集D中各项x_m上投影的加权和。In the formula, ψ _conv represents the convolution of the conversion feature, w ₁ ,...,w _M represents different learnable parameters,

Represents the weighted sum of the projections of all neighborhood point features on each item x _m in the direction set D.

Preferably, the method for generating the direction set space is:

Sampling S groups of point clouds in the point cloud data;

For each sampled point cloud P _s ={ps _,i ∈R ³ |i=1,...,N}, simulate the direction-induced convolutional network coding stage of each residual direction-induced convolution module and the farthest point sampling The residual direction induces the farthest point sampling and neighborhood construction process of the convolution module, and calculates the spatial direction features of all sampled point clouds at each layer

使用K均值算法对每一层的空间方向特征

进行聚类，生成方向诱导卷积网络各层的空间方向集D_l。Combining all the spatial direction features of the corresponding layers of the S groups of point clouds to obtain

Use the K-means algorithm to analyze the spatial orientation features of each layer

Clustering is performed to generate a spatial orientation set D _l of each layer of the orientation-inducing convolutional network.

Preferably, the residual direction-induced convolution module includes a first multi-layer perceptron, a direction-induced convolution branch, a second multi-layer perceptron and a third multi-layer perceptron; the first multi-layer perceptron is used for Extract the point-wise features of the input point cloud and input the direction-induced convolution branch and the second multilayer perceptron respectively;

The direction-induced convolution branch includes a first direction-induced convolution module and a fourth multi-layer perceptron. The first direction-induced convolution is used to extract features of the disordered neighborhood of point clouds. The fourth multi-layer perceptron The second multi-layer perceptron is used to increase the feature dimension so that it is the same as the feature dimension output by the direction-induced convolution branch; the direction-induced convolution branch and the second multi-layer perceptron output After the features are concatenated, the third multi-layer perceptron performs feature fusion to obtain the output of the residual direction induced convolution module.

Preferably, the farthest point sampling residual direction-induced convolution module includes a farthest point sampling module, a fifth multi-layer perceptron, a direction-induced convolution branch, a max-pooling branch and an eighth multi-layer perceptron, and the The farthest point sampling module is used to downsample the input point cloud; the fifth multi-layer perceptron is used to extract point-by-point features and input the direction-induced convolution branch and the maximum pooling branch respectively; the direction-induced convolution branch It includes a second-direction induced convolution and a sixth multi-layer perceptron, and the maximum pooling branch includes a maximum-pooling layer and a seventh multi-layer perceptron; the second-direction induced convolution is used to extract point cloud disordered neighbors Domain features, the max pooling branch is used to aggregate local features, the sixth and seventh multilayer perceptrons are used to increase the feature dimension, and make the dimensions of the output features of the two branches equal; The features output by the branches are concatenated and then fused by the eighth multi-layer perceptron, and the output of the convolution module induced by the direction of the farthest point sampling residual is obtained.

Compared with the prior art, the present invention has the following significant advantages: 1) the present invention better captures the local spatial structure of the point cloud, wherein the neighborhood information of each point can be projected into a standardized and ordered direction set space; 2) The present invention constructs a direction-induced convolution network by alternately stacking the residual direction-induced convolution module and the farthest point sampling residual direction-induced convolution module, and analyzes the point cloud in an end-to-end manner, improving the performance of the point cloud. Classification and accuracy of point cloud segmentation; 3) The present invention achieves better performance in both point cloud classification and semantic segmentation tasks.

Description of drawings

FIG. 1 is a schematic structural diagram of the direction-induced convolution of the present invention.

FIG. 2 is a schematic structural diagram of a residual direction induced convolution module of the present invention.

FIG. 3 is a schematic structural diagram of the farthest point sampling residual direction induced convolution module of the present invention.

FIG. 4 is a schematic structural diagram of a direction-inducing convolutional network according to the present invention.

Detailed ways

The solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

As shown in Figures 1-4, a point cloud analysis method based on direction-induced convolution includes the following steps:

Step 1. Constructing a direction-induced convolution module, the direction-induced convolution module is used to extract the features of the disordered neighborhood of the point cloud;

In a further embodiment, the specific process of extracting the features of the disordered neighborhood of the point cloud is:

Calculate the spatial direction features of the local neighborhood points of the point cloud;

表示一组点云输入，使用F＝{f_i|i＝1,……,N}∈R^d表示其对应特征集，其中N表示点的数量，p_i表示三维坐标(x，y，z)，f_i表示附加特征(例如RGB颜色、曲面法线、激光反射强度等)。对于每个点p_i∈P，使用N(p_i)表示一个以p_i为中心的局部点云区域，

其中r∈R为选定的半径。对每个点p_i的邻域点

其空间方向特征

编码如下：use

represents a set of point cloud inputs, and uses F={fi | _i =1,...,N}∈R ^d to represent its corresponding feature set, where N represents the number of points, and p _i represents the three-dimensional coordinates (x, y, z ), f _i represents additional features (such as RGB color, surface normal, laser reflection intensity, etc.). For each point p _i ∈ P, use N( _pi ) to denote a local point cloud region centered on p _i ,

where r∈R is the selected radius. Neighborhood points for each point _pi

its spatial orientation

The encoding is as follows:

Calculate and normalize the spatial direction similarity weight between the neighborhood points and the items of the spatial direction set;

表示一个含有M项的空间方向集，其中每项x_m∈D代表3D空间中的一个方向。对每个邻域点

计算其空间方向特征

与空间方向集中各项x_m的相似性，以获得每个邻域点

在x_m方向的相似性权重

该相似性权重计算过程表示为以下形式：use

represents a set of spatial directions with M terms, where each term x _m ∈ D represents a direction in 3D space. for each neighborhood point

Calculate its spatial orientation feature

similarity to the terms x _m in the set of spatial orientations to obtain each neighborhood point

Similarity weight in x _m direction

The similarity weight calculation process is expressed in the following form:

与方向集D各项所计算得到的相似性权重进行归一化，这里使用softmax进行归一化，以使得相似性权重更加稀疏，该相似性权重归一化计算过程表示为以下形式：the neighborhood point

It is normalized with the similarity weight calculated by each item of the direction set D. Here, softmax is used for normalization to make the similarity weight more sparse. The calculation process of the similarity weight normalization is expressed as the following form:

表示每个邻域点

在x_m项上的归一化相似性权重。in

represents each neighborhood point

Normalized similarity weights over the x _m terms.

Perform feature space conversion to convert the features of point cloud disordered neighborhood points into a standardized and ordered direction set space, specifically:

的特征投影到规范有序的方向集空间D中，该过程称为特征空间转换，表示为以下形式：Based on normalized similarity weights, all neighbor points of point _pi are

The features of are projected into a canonical ordered set of directions space D, a process called feature space transformation, expressed in the following form:

是邻域点

的特征；

代表串接；

表示该点的相对位置编码，表示为：in

is the neighborhood point

Characteristics;

represents concatenation;

Represents the relative position code of the point, expressed as:

表示所有邻域点特征在方向集D中各项x_m上投影的加权和。至此，点p_i的所有无序邻域点

的特征都被转换到规则有序的方向集空间中。feature space transformation

Represents the weighted sum of the projections of all neighborhood point features on each item x _m in the direction set D. So far, all the unordered neighbor points of point p _i

The features are transformed into a regular and ordered direction set space.

In the direction set space, standard convolution is used to encode the spatial transformation features projected onto the items of the spatial direction set to complete the feature extraction of the disordered neighborhood of the point cloud, which is more robust to the disordered local area of the point cloud. The specific process is expressed as follows:

Among them, ψ _conv represents the convolution of the transformation feature, and w ₁ ,...,w _M represents different learnable parameters.

Step 2. Based on the direction-induced convolution module, construct the residual direction-induced convolution module and the farthest point sampling residual direction-induced convolution module to retain richer point cloud feature information;

As shown in FIG. 2, in a further embodiment, the residual direction-induced convolution module includes a first multilayer perceptron, a direction-induced convolution branch, a second multilayer perceptron, and a third multilayer perceptron; The first multilayer perceptron is used to extract the point-wise features of the input point cloud, and then the direction-induced convolution branch and the second multilayer perceptron are respectively input,

The direction-induced convolution branch includes a first direction-induced convolution module and a fourth multi-layer perceptron. The first direction-induced convolution is used to extract features of the disordered neighborhood of point clouds. The fourth multi-layer perceptron The second multi-layer perceptron is used to increase the feature dimension so that it is the same as the feature dimension output by the direction-induced convolution branch. Finally, the output features of the two branches are concatenated, and the third multi-layer perceptron is used for feature fusion to obtain the output of the residual direction induced convolution module.

As shown in FIG. 3 , in a further embodiment, the farthest point sampling residual direction-induced convolution module includes a farthest point sampling module, a fifth multilayer perceptron, a direction-induced convolution branch, a maximum pooling branch and Eighth Multilayer Perceptron. First use the farthest point sampling module to downsample the input point cloud, reduce the number of points, and perform multi-scale analysis, then use the fifth multilayer perceptron to extract point-by-point features, and then input the direction-induced convolution branch and max pooling respectively. The direction-induced convolution branch includes a second direction-induced convolution and a sixth multi-layer perceptron, and the max-pooling branch includes a max-pooling layer and a seventh multi-layer perceptron. The second-direction induced convolution is used to extract the features of the point cloud disordered neighborhood, and the max-pooling operation is used in the max-pooling branch to aggregate local features, and then both branches use the sixth multilayer perceptron, the first The seven multi-layer perceptron increases the feature dimension to retain more feature information and make the dimensions of the output features of the two branches equal. Finally, the features output from the two branches are concatenated, and the eighth multi-layer perceptron performs feature fusion to obtain The furthest point samples the residual direction to induce the output of the convolution module.

Step 3. According to the residual direction-induced convolution module and the farthest point sampling residual direction-induced convolution module, a direction-induced convolution network is constructed to capture the hierarchical features of the input point cloud and perform point cloud analysis in an end-to-end manner .

As shown in Figure 4, the direction-inducing convolutional network consists of a point cloud segmentation sub-network and a classification sub-network.

The point cloud segmentation sub-network and the classification sub-network use the same encoder to encode the features of the input point cloud. The encoder is formed by alternately connecting the residual direction induced convolution module and the farthest point sampling residual direction induced convolution module, including the first residual direction induced convolution module and the first farthest point sampling residual direction induced convolution volume. The product module, the second residual direction induced convolution module, the second farthest point sampling residual direction induced convolution module, and the third residual direction induced convolution module. After the input point cloud P is extracted with features by the first residual direction induced convolution module, input the first farthest point sampling residual direction induced convolution module to obtain the down-sampling point cloud P ₁ and its features. Similarly, after the output P ₁ of the first farthest point sampling residual direction induced convolution module is extracted features through the second residual direction induced convolution module, it is input into the second farthest point sampling residual direction induced convolution module to obtain: _Downsampled point cloud P2 and its features. The first residual direction induced convolution module, the first farthest point sampling residual direction induced convolution module, the second residual direction induced convolution module, the second farthest point sampling residual direction induced convolution module, the third The residual direction induces the convolution modules to be connected alternately, and the local neighborhood features of the point cloud are extracted hierarchically for multi-scale analysis.

The decoder of the point cloud segmentation sub-network includes: first upsampling, second upsampling, ninth multilayer perceptron, tenth multilayer perceptron and first fully connected layer. The decoder uses K-nearest neighbor interpolation to upsample the downsampled point cloud encoded by the encoder, recovers the number of point clouds layer by layer, and propagates the features of the points. At the same time, cross-connection and multi-layer perceptrons are adopted to combine the features of corresponding layers in the encoder and decoder. Specifically, the output P ₂ of the third residual direction induced convolution module in the encoder is subjected to the first up-sampling to obtain the up-sampled point cloud P ₁ ' and its characteristics, the number of point clouds in P ₁ ' and the second residual direction The output point cloud P ₁ of the induced convolution module is consistent, the features of the P ₁ and P ₁ ' point clouds are concatenated, and then the ninth multi-layer perceptron is used for feature fusion to obtain a new P ₁ point cloud and its features. Similarly, the output point cloud P1 _of the ninth multilayer perceptron is upsampled to obtain the point cloud P' and its features, and the tenth multilayer perceptron is used to induce the convolution module of the point cloud P' and the first residual direction. The output point cloud P of P is feature fusion to obtain a new P point cloud and its features. Finally, the first fully connected layer is used to predict the label of each point to get the segmentation result.

The decoder of the point cloud classification sub-network includes global max pooling and a second fully connected layer. The output of the third residual direction-induced convolution module in the encoder is globally max-pooled to obtain the global features of the entire point cloud. Finally, the labels of the global features are predicted using the second fully connected layer to obtain the classification results.

Step 4: Input the point cloud data into the direction-induced convolution network to obtain point cloud segmentation results and classification results.

Orientation-inducing convolutional networks are trained on point cloud segmentation and classification datasets using an Adaptive Moment Estimation (Adam) optimizer with a learning rate of 0.001 and a decay rate of 0.7, while the network parameters are optimized using a cross-entropy loss. All multilayer perceptrons in the network are composed of 1×1 convolution, batch normalization (BN) and nonlinear activation (Relu) concatenated. In the testing phase, the point cloud data is input into the trained direction-induced convolutional network to obtain point cloud segmentation results and classification results.

In a further embodiment, the method for generating the spatial direction set is:

将S组点云相应层的所有空间方向特征合并得到

使用K均值算法对每一层的空间方向特征

进行聚类，生成方向诱导卷积网络各层的空间方向集Dl。According to different point cloud data sets, S groups of point clouds are sampled in the training point cloud data, namely {P _s }, s=1,...,S. For each sampled point cloud P _s ={ps _,i ∈R3|i=1,...,N}, each residual direction-induced convolution module and the farthest point sampling residual in the simulation direction-induced convolutional network coding stage The farthest point sampling and neighborhood construction process of the differential direction induced convolution module, and the spatial direction characteristics of all sampled point clouds of each layer are calculated

Combining all the spatial direction features of the corresponding layers of the S groups of point clouds to obtain

Use the K-means algorithm to analyze the spatial orientation features of each layer

Clustering is performed to generate the spatial direction set D1 of each layer of the direction-inducing convolutional network.

The present invention utilizes a new direction-induced convolution to improve the performance of point cloud analysis. For dense point clouds, most points in the local neighborhood have similar features, such as spatial location and color information, so it is difficult to use these features to sparse quantization of neighborhood points. Since the local neighborhood points of the point cloud are in different directions of the center point, their spatial directions contain a lot of useful information. To this end, the present invention proposes to sparse and quantify the spatial direction features of the local neighborhood points of the point cloud, and use direction-induced convolution to project the features of the disordered neighborhood points into a canonical and ordered direction set space and extract the features. Based on the direction-induced convolution, the residual direction-induced convolution module and the farthest point sampling residual direction-induced convolution module are constructed to obtain a richer point cloud representation. By alternately stacking residual direction-induced convolution modules and farthest point sampling residual direction-induced convolution modules to extract point cloud hierarchical features, a direction-induced convolutional network with a widely used cross-connection encoder-decoder architecture is constructed for to complete point cloud classification and segmentation tasks.

Example

In order to verify the effectiveness of the scheme of the present invention, the following experiments were carried out.

To verify the effectiveness of the orientation-induced convolution method, the performance of the orientation-induced convolutional network on various tasks (such as point cloud classification, part segmentation, large-scale scene segmentation) is evaluated. Use different network configurations for different tasks. For the point cloud classification task, 3 residual direction induced convolution modules and 2 residual direction induced convolution modules are alternately stacked; for the shape part segmentation task, 4 residual direction induced convolution modules and 3 farthest point sampling residual direction induced convolution modules; for semantic segmentation tasks, 5 residual direction induced convolution modules and 4 farthest point sampling residual direction induced convolution modules are alternately stacked. In all classification and segmentation tasks, the learning rate of the Adaptive Moment Estimation (Adam) optimizer is set to 0.001 and the decay rate is 0.7, while the network parameters are optimized with cross-entropy loss. All multilayer perceptrons in the network are composed of 1×1 convolution, batch normalization (BN) and nonlinear activation (Relu) concatenated. The number of items in the direction set is set to 15 (ie, M=15). All experiments are performed on a single Titan Xp GPU.

Table 1: Comparison of shape classification performance on ModelNet40 dataset

method enter points OA(%) Pointwise-CNN xyz 1k 86.1 DeepSets xyz 1k 87.1 ECC xyz 1k 87.4 PointNet xyz 1k 89.2 SCN xyz 1k 90.0 Kd-Net (depth=10) xyz 1k 90.6 PointNet++ xyz 1k 90.7 KCNet xyz 1k 91.0 MRTNet xyz 1k 91.2 Spec-GCN xyz 1k 91.5 PointCNN xyz 1k 92.2 DGCNN xyz 1k 92.2 PCNN xyz 1k 92.3 SO-Net xyz 2k 90.9 Kd-Net (depth=15) xyz 32k 91.8 DICNet(Ours) xyz 1k 91.5 O-CNN xyz,nor - 90.6 Spec-GCN xyz,nor 1k 91.8 Geo-CNN xyz,nor 1k 93.4 PointNet++ xyz,nor 5k 91.9 SpiderCNN xyz,nor 5k 92.4 SO-Net xyz,nor 5k 93.4

For the point cloud classification task, the present invention is evaluated on the ModelNet40 dataset. ModelNet40 includes 40 classes of objects, of which 9843 are used for training and 2468 are used for testing. The point cloud data of ModelNet40 is taken as input, where each point cloud is uniformly sampled with 1024 points. The experiments only use the geometric coordinates (x, y, z) of the sampling points. Normalizing the coordinates transforms the 3D object into the unit sphere. At the same time, simple point cloud data augmentation techniques are used, including random scaling, translation, and perturbation of the original point cloud. Finally, each test instance is tested 10 times, the average of the 10 times is calculated as the final predicted value, and the overall accuracy (OA) of the 40 classes is used to evaluate the performance of different methods. The present invention is compared with several state-of-the-art methods (divided into two groups according to the number of input point clouds and whether they have additional features, xyz denotes 3D coordinates, nor denotes normal vectors), and the results are shown in Table 1.

The invention obtains an OA value of 91.5% in the point cloud classification task, and achieves a remarkable effect. Compared with other methods that take 3D coordinates of point cloud as input, the proposed orientation-induced convolutional network achieves superior results. The proposed method is still competitive compared to other methods that utilize more input information. The experimental results show that the orientation-induced convolution has the ability to model the entire point cloud.

Table 2: Comparison of shape part segmentation performance on ShapeNet parts dataset

Method mcIoU (%) mIoU(%) aero bag cap car chair eare. guit. knif lamp lap. moto. mug pist rock skate. table Kd-Net 77.4 82.3 80.1 74.6 74.3 70.3 88.6 73.5 90.2 87.2 81.0 94.9 57.4 86.7 78.1 51.8 69.9 80.3 PointNet 80.4 83.7 83.4 78.7 82.5 74.9 89.6 73.0 91.5 85.9 80.8 95.3 65.2 93.0 81.2 57.9 72.8 80.6 SO-Net 81.0 84.9 82.8 77.8 88.0 77.3 90.6 73.5 90.7 83.9 82.8 94.8 69.1 94.2 80.9 53.1 72.9 83.0 RS-Net 81.4 84.9 82.7 86.4 84.1 78.2 90.4 69.3 91.4 87.0 83.5 95.4 66.0 92.6 81.8 56.1 75.8 82.2 PCNN 81.8 85.1 82.4 80.1 85.5 79.5 90.8 73.2 91.3 86.0 85.0 95.7 73.2 94.8 83.3 51.0 75.0 81.8 PointNet++ 81.9 85.1 82.4 79.0 87.7 77.3 90.8 71.8 91.0 85.9 83.7 95.3 71.6 94.1 81.3 58.7 76.4 82.6 SPLATNet 82.0 84.6 81.9 83.9 88.6 79.5 90.1 73.5 91.3 84.7 84.5 96.3 69.7 95.0 81.7 59.2 70.4 81.3 KCNet 82.2 84.7 82.8 81.5 86.4 77.6 90.3 76.8 91.0 87.2 84.5 95.5 69.2 94.4 81.6 60.1 75.2 81.3 SynSpecCNN 82.0 84.7 81.6 81.7 81.9 75.2 90.2 74.9 93.0 86.1 84.7 95.6 66.7 92.7 81.6 60.6 82.9 82.1 DGCNN 82.3 85.1 84.2 83.7 84.4 77.1 90.9 78.5 91.5 87.3 82.9 96.0 67.8 93.3 82.6 59.7 75.5 82.0 SpiderCNN 82.4 85.3 83.5 81.0 87.2 77.5 90.7 76.8 91.1 87.3 83.3 95.8 70.2 93.5 82.7 59.7 75.8 82.8 SubSparseCNN 83.3 86.0 84.1 83.0 84.0 80.8 91.4 78.2 91.6 89.1 85.0 95.8 73.7 95.2 84.0 58.5 76.0 82.7 PointCNN 84.6 86.1 84.1 86.5 86.0 80.8 90.6 79.7 92.3 88.4 85.3 96.1 77.2 95.3 84.2 64.2 80.0 83.0 DICNet(Ours) 82.9 85.7 83.0 84.0 86.0 80.4 91.0 72.1 91.5 87.2 83.9 95.8 73.1 95.0 83.6 59.3 77.1 83.2

Part segmentation is a challenging task in shape analysis. To this end, the effectiveness of the inventive scheme is evaluated on the ShapeNet parts dataset. The dataset contains 16,881 3D objects covering 16 shape categories. Each point cloud instance has 2 to 6 part label annotations, for a total of 50 part labels. 2048 points are randomly selected as input and one-hot encoding of object labels is concatenated in the last feature layer. In the experiment, the class average intersection ratio (mcIoU) and instance average intersection ratio (mIoU) of the segmentation results of point cloud parts are used as evaluation indicators.

As shown in Table 2, the present invention outperforms the state-of-the-art methods on the task of segmentation of shape parts. 82.9% mcIoU and 85.7% mIoU were achieved. All experimental results fully demonstrate that the present invention can achieve more excellent performance, which also proves the effectiveness of the proposed direction-induced convolution for part modeling tasks.

The present invention is evaluated on the large-scale indoor dataset S3DIS. The S3DIS dataset contains 273 million points from 6 large indoor areas in 3 different building complexes, each of which is annotated with semantic labels for 13 categories. The Area-5 scene is used for testing, and the other scenes are used for training. In the preparation of training and testing datasets, each room is divided into multiple blocks, each with a size of 1m × 1m and a step size of 0.5m. The dataset spanning six regions contains a total of 23,585 blocks. In each block, 4096 points are sampled for training. Each point is represented by a 9-dimensional vector (XYZ, RGB and normalized XYZ). The experiments use the mean intersection ratio (mIoU), mean class accuracy (mAcc), and overall accuracy (OA) of 13 classes as evaluation metrics.

Table 3: Comparison of semantic segmentation performance on S3DISArea-5 dataset

The quantitative results are shown in Table 3, and the present invention has achieved the best results in all three metrics. At the same time, the mIoU score of the model significantly exceeds PointNet by 21.4% and PointCNN by 5.3%. Compared with the previous best-performing point cloud semantic segmentation method, it achieves 1.1%, 2.2% and 2.2% improvement on OA, mAcc and mIoU, respectively. Experiments show that direction-induced convolution can better model the semantics of point clouds. Additionally, the models achieve state-of-the-art performance in the wall, window, door, and bookcase categories. Usually windows, doors and bookcases are embedded in the wall and indistinguishable, and the direction-induced convolutional network can still perform accurate segmentation and get better results. This shows that direction-induced convolution can capture subtle differences in local regions and accurately segment complex scenes, which further validates the ability of direction-induced convolution to model complex scene semantics.

The above-mentioned embodiments are only preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, without departing from the principles of the present invention, several improvements and equivalent replacements can also be made. The technical solutions required to be improved and equivalently replaced all fall within the protection scope of the present invention.

Claims (8)

1. a point cloud analysis method based on direction-induced convolution, is characterized in that, comprises the following steps: Step 1. Constructing a direction-induced convolution module, the direction-induced convolution module is used to extract the features of the disordered neighborhood of the point cloud; Step 2. Based on the direction-induced convolution module, construct the residual direction-induced convolution module and the farthest point sampling residual direction-induced convolution module; The residual direction-induced convolution module includes a first multi-layer perceptron, a direction-induced convolution branch, a second multi-layer perceptron and a third multi-layer perceptron; the first multi-layer perceptron is used to extract input points Point-wise features of the cloud and input to the direction-induced convolution branch and the second multilayer perceptron, respectively; The direction-induced convolution branch includes a first direction-induced convolution module and a fourth multi-layer perceptron. The first direction-induced convolution is used to extract features of the disordered neighborhood of point clouds. The fourth multi-layer perceptron The second multi-layer perceptron is used to increase the feature dimension so that it is the same as the feature dimension output by the direction-induced convolution branch; the direction-induced convolution branch and the second multi-layer perceptron output After the features are concatenated, the third multi-layer perceptron performs feature fusion to obtain the output of the residual direction induced convolution module; The farthest point sampling residual direction-induced convolution module includes a farthest point sampling module, a fifth multi-layer perceptron, a direction-induced convolution branch, a maximum pooling branch and an eighth multi-layer perceptron. The sampling module is used for down-sampling the input point cloud; the fifth multi-layer perceptron is used for extracting point-by-point features and inputting the direction-induced convolution branch and the maximum pooling branch respectively; the direction-induced convolution branch includes a second Direction-induced convolution, sixth multi-layer perceptron, the maximum pooling branch includes maximum pooling layer, seventh multi-layer perceptron; the second direction-induced convolution is used to extract the features of point cloud disordered neighborhoods , the maximum pooling branch is used to aggregate local features, the sixth and seventh multilayer perceptrons are both used to increase the feature dimension, and make the dimensions of the output features of the two branches equal; the two branches output After the features are concatenated, the features are fused by the eighth multilayer perceptron, and the output of the convolution module induced by the direction of the farthest point sampling residual is obtained; Step 3. Construct a direction-induced convolution network according to the residual direction-induced convolution module and the farthest point sampling residual direction-induced convolution module; Step 4: Input the point cloud data into the direction-induced convolution network to obtain point cloud segmentation results and classification results. 2. the point cloud analysis method based on direction-induced convolution according to claim 1, is characterized in that, the concrete steps that described direction-induced convolution module is used to extract the feature of point cloud disordered neighborhood are: Calculate the spatial direction features of the local neighborhood points of the point cloud; Calculate and normalize the spatial direction similarity weight between the neighborhood points and the items of the spatial direction set; Convert the features of point cloud disordered neighborhood points into the direction set space; In the orientation set space, standard convolution is used to encode the spatially transformed features projected onto the items of the spatial orientation set to complete feature extraction of the unordered neighborhood of the point cloud. 3. the point cloud analysis method based on direction-induced convolution according to claim 2, is characterized in that, the calculation formula of spatial direction similarity weight is:

In the formula,

is the spatial orientation feature of the domain point for each point, and each term x _m ∈ D is an orientation in 3D space. 4. the point cloud analysis method based on direction-induced convolution according to claim 2, is characterized in that, the neighborhood point of each point p _i

The spatial orientation characteristics of

Encoded as:

5. the point cloud analysis method based on direction-induced convolution according to claim 2, is characterized in that, feature conversion is expressed as:

in,

is the neighborhood point

The characteristics of ; ⊕ represents concatenation;

represents the normalized spatial orientation similarity;

Represents a neighborhood point

The relative position encoding of , N( _pi ) represents the local point cloud area centered on point _pi . 6. the point cloud analysis method based on direction-induced convolution according to claim 2, is characterized in that, the concrete process that uses standard convolution to encode the spatial transformation feature projected on each item of spatial direction set is:

In the formula, ψ _conv represents the convolution of the conversion feature, w ₁ ,...,w _M represents different learnable parameters,

Represents the weighted sum of the projections of all neighborhood point features on each item x _m in the direction set D. 7. The point cloud analysis method based on direction-induced convolution according to claim 2, is characterized in that, the generation method of described direction set space is: Sampling S groups of point clouds in the point cloud data; For each sampled point cloud P _s ={ps _,i ∈R ³ |i=1,...,N}, simulate the direction-induced convolutional network coding stage of each residual direction-induced convolution module and the farthest point sampling The residual direction induces the farthest point sampling and neighborhood construction process of the convolution module, and calculates the spatial direction features of all sampled point clouds at each layer

Combining all the spatial direction features of the corresponding layers of the S groups of point clouds to obtain

Use the K-means algorithm to analyze the spatial orientation features of each layer

Clustering is performed to generate a spatial orientation set D _l of each layer of the orientation-inducing convolutional network. 8. The point cloud analysis method based on direction-induced convolution according to claim 1, wherein the direction-induced convolution network comprises: a point cloud segmentation sub-network and a classification sub-network, the point cloud segmentation sub-network The same encoder is used as the classification sub-network to encode the input point cloud; the encoder is formed by alternately connecting the residual direction induced convolution module and the farthest point sampling residual direction induced convolution module; The decoder of the point cloud segmentation sub-network includes: the first upsampling, the second upsampling, the ninth multi-layer perceptron, the tenth multi-layer perceptron and the first fully connected layer; the decoder uses K-nearest neighbor interpolation to encode the encoder The down-sampled point cloud is up-sampled, the number of point clouds is restored layer by layer, and the features of the points are propagated; at the same time, cross-connection and multi-layer perceptrons are used to combine the features of the corresponding layers in the encoder and decoder; The decoder of the point cloud classification sub-network includes: global maximum pooling and the second fully connected layer; global maximum pooling is performed on the output of the encoder to obtain the global features of the entire point cloud; the second fully connected layer is used to predict the global features label to get the classification result.

Images