CN114170410A - Point cloud part level segmentation method based on PointNet graph convolution and KNN search - Google Patents
Point cloud part level segmentation method based on PointNet graph convolution and KNN search Download PDFInfo
- Publication number
- CN114170410A CN114170410A CN202111442718.2A CN202111442718A CN114170410A CN 114170410 A CN114170410 A CN 114170410A CN 202111442718 A CN202111442718 A CN 202111442718A CN 114170410 A CN114170410 A CN 114170410A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- net
- network
- point
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24143—Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/02—Affine transformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Architecture (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- Image Analysis (AREA)
Abstract
本发明公开了基于PointNet的图卷积与KNN搜索的点云零件级分割方法，首先，通过一个微型网络T‑Net保证点云旋转不变性；然后，计算每两两点对之间的欧氏距离，选取距离最近的k个点，以点pi为中心点构造k近邻域图来提取局部特征，在后续的模型训练中，对每一层动态更新k邻域图，得到局部特征信息；接下来，通过设计新的T‑Net微型网络对齐不同点云局部特征数据；最后，将输出的多层局部特征与全局特征进行拼接，再通过多个MLP操作，得到三维点云物体的零件级分割结果，本发明能够提高三维点云物体的零件级分割精度。
The invention discloses a point cloud part-level segmentation method based on PointNet graph convolution and KNN search. First, a micro network T-Net is used to ensure the point cloud rotation invariance; then, the Euclidean value between every two point pairs is calculated. Distance, select the k points with the closest distance, and construct a k-nearest neighborhood graph with point p i as the center point to extract local features. In the subsequent model training, the k neighborhood graph is dynamically updated for each layer to obtain local feature information; Next, the local feature data of different point clouds is aligned by designing a new T-Net micro-network; finally, the output multi-layer local features and global features are spliced, and then through multiple MLP operations, the part level of the 3D point cloud object is obtained As a result of the segmentation, the present invention can improve the part-level segmentation accuracy of the three-dimensional point cloud object.
Description
Technical Field
The invention belongs to the technical field of point cloud object segmentation, and particularly relates to a point cloud part level segmentation method based on PointNet graph convolution and KNN search.
Background
Analysis and understanding of three-dimensional shapes has been an important research topic in computer graphics. In recent years, the popularization of depth sensors and three-dimensional laser scanners has promoted the rapid development of three-dimensional point cloud processing methods. The part-level segmentation of the three-dimensional point cloud object is used as the basis of 3D scene understanding and analysis, becomes the research focus in the fields of navigation positioning, medical image analysis, mode recognition and the like, and has important research value and wide application prospect.
The point cloud part level segmentation is to divide the point cloud into a plurality of regions with detailed semantic categories. The basic point cloud contains 6-dimensional features, XYZ coordinates and RGB color information, respectively. With the appearance of a large-scale data set, the reduction of computer hardware cost and the improvement of GPU parallel computing capability, deep learning gradually occupies an absolute dominant position in the point cloud segmentation field, deep learning taking a PointNet network as a pioneer is researched and used by more scholars, the network model is very simple, the original point cloud data can be directly operated according to the characteristics of the point cloud data, meanwhile, MaxPoint is used as a symmetric function to process the orderless of the point cloud, and two T-net networks are adopted to carry out rotation invariance processing on the model.
The PointNet network mainly processes points independently on a local scale, arrangement invariance is kept, but interaction relation of adjacent points is not considered, and therefore local characteristics of geometric relation among nodes are lost. Based on the method, the PointNet network is optimized, and the neighborhood characteristics of the point cloud are more efficiently aggregated by the part level segmentation method based on the graph convolution and KNN search of PointNet.
Disclosure of Invention
The invention aims to provide a point cloud part level segmentation method based on graph convolution and KNN search of PointNet, which solves the problem of low semantic segmentation precision caused by extracting the characteristics of single points only by a PointNet network.
The technical scheme adopted by the invention is that the point cloud part level segmentation method based on PointNet graph convolution and KNN search is implemented according to the following steps:
step 1, point cloud space alignment: inputting point cloud data, predicting an affine transformation matrix by using a T-Net micro network, and performing coordinate alignment on the input point cloud data by using the matrix;
step 2, local feature extraction: in order to capture local features, each point in the step 1 is taken as a vertex, k points with the nearest distance are selected to construct a k nearest neighborhood graph, edge convolution is applied to edges connecting adjacent point pairs to obtain edge feature information between every two point pairs, and then local feature matrix information is extracted through MLP operation of a multilayer perceptron network;
step 4, point cloud feature transformation: designing a new T-Net mini network, inputting the updated local feature matrix information into the new T-Net network for coordinate alignment, and ensuring the rotation invariance of feature points;
and 5, fusing local features and global features: and carrying out MaxPolling maximum pooling operation on the aligned and updated local feature matrix information, and processing the global feature information obtained by the operation through a plurality of MLPs to obtain the category fraction of each object, thereby realizing the part-level segmentation of the three-dimensional object.
The present invention is also characterized in that,
the step 1 specifically comprises the following steps:
step 1.1, inputting point cloud data P ═ { P) of a three-dimensional object1,p2,…,pi,…,pNIn which p isiIndicating the position information of the ith point, where N is point _ num, which is the number of points of the object, N is 2048, the batch size is 4, and the one-time training period epoch is 200;
step 1.2, in order to make the model have arrangement invariance to the input, the information of each point is aggregated by using a symmetric function to obtain an affine matrix A:
f({p1,…,pN})≈g(h(p1),…,h(pN)), (1)
in the formula (1), h represents a multilayer perceptron network, g represents a maximum pooling function, and f represents the characteristic information of a captured point cloud set;
and step 1.3, after the input point cloud data is multiplied by the affine matrix A, the input point cloud coordinate alignment is realized through a plurality of MLP processing.
The step 2 specifically comprises the following steps:
step 2.1, selecting k nearest neighbor points k as 20 by using Euclidean distance as a measurement standard;
step 2.2, each point p of the point cloud data obtained after the alignment in the step 1 is processediAnd its k nearest neighbors qjJ-1, 2, …, k constructs a k-neighborhood domain map, where
RFRepresenting an F-dimensional input point cloud matrix.
Step 2.3, the k-neighborhood map structure is G ═ (V, E), where V ═ { p ═ piI | ═ 1, …, N } represents a set of vertices, E ═ Ei＝(ei1,ei2,…,eij,…eik) I | ═ 1,2, …, N } represents the set of edges between vertices, eijRepresents a point piDirected edges with k neighbor points;
and 2.4, performing MLP convolution processing on the k near neighborhood map obtained after updating in the step 2.3 to obtain local characteristic matrix information of the three-dimensional point cloud object.
The step 3 is as follows:
step 3.1, the MLP comprises two convolution layers, a batch processing normalization layer and an activation layer, wherein the sizes of convolution kernels of the two convolution layers are 64 and 64 from left to right, and output point cloud data of each layer of the MLP network is
l represents the l-th layer of the MLP network;
step 3.2, obtaining a different output k near-neighborhood graph G for each MLP layer according to the output point cloud data of each MLP network layerl＝(Vl,El)，GlK-neighborhood graph, V, representing the output of the l-th layerlRepresents GlCorresponding set of vertices, ElRepresents GlA set of edges between corresponding vertices;
and 3.3, acquiring local feature matrix information of each layer according to the k near neighborhood graph output by each layer.
The new designed T-Net network is characterized in that a regularization item shown in a formula (3) is added in the softmax training loss of a conventional T-Net network framework, and a characteristic transformation matrix is limited to be close to an orthogonal matrix;
in the formula (3), A is an affine matrix predicted by the T-Net network, so that input information cannot be lost by orthogonal transformation, I is an identity matrix, and F represents a point cloud dimension.
The step 5 is as follows:
step 5.1, using the aligned and updated local feature matrix information as input, and performing convolution operation processing through 3 spatial _ transform modules to respectively obtain three local feature information nets1，net2，net3；
Step 5.2, respectively converting three local features corresponding to the 3 spatial _ transform modules: net1，net2，net3Splicing, and outputting out;
step 5.3, carrying out MaxPolling maximum pooling operation on the output out data to obtain the global characteristics of the current layer;
step 5.4, splicing the global features of the current layer with the tag information of the input point cloud in the data set to obtain global feature global _ feature of the whole network;
step 5.5, global _ feature and net of global feature1，net2，net3Splicing is carried out;
and 5.6, carrying out MLP multi-layer perceptron processing on the data spliced in the step 5.5 to obtain the category score conditions of all the objects, and realizing the part-level segmentation of the three-dimensional objects.
The invention has the beneficial effects that:
the invention relates to a point cloud part level segmentation method based on PointNet graph convolution and KNN search.A point cloud rotation invariance is ensured through a micro network T-Net (input _ transform); then, calculating Euclidean distance between every two point pairs, selecting k points with the nearest distance, and taking point piConstructing a k near neighborhood graph for the central point to extract local features, and dynamically updating the k neighborhood graph for each layer in subsequent model training to obtain local feature information; next, aligning the local feature data of different point clouds through a T-Net (feature _ transform) micro network; and finally, splicing the output multilayer local features and the global features, and then obtaining a part grade segmentation result of the three-dimensional point cloud object through a plurality of MLP (multilayer perceptron) operations.
Drawings
FIG. 1 is a network architecture diagram of a point cloud part level segmentation method based on PointNet's graph convolution and KNN search;
FIG. 2 is a graph convolution based on PointNet and KNN search point cloud part level segmentation method improved graph convolution and KNN search edge feature (edge _ feature) extraction network graph;
FIG. 3 is a result of part-level segmentation of a three-dimensional object in a ShapeNetCore dataset;
FIG. 4(a) is a graph of loss during ShapeNetCore dataset training
Figure 4(b) is a graph of accuracy during the training of the ShapeNetCore dataset.
Detailed Description
The present invention will be described in detail with reference to the following embodiments.
Examples
The embodiment provides a point cloud part level segmentation method based on PointNet graph convolution and KNN search, which is specifically implemented according to the following steps as shown in FIG. 1:
step 1, point cloud space alignment: inputting point cloud data, predicting an affine transformation matrix by using a T-Net micro network, and performing coordinate alignment on the input point cloud data by using the matrix;
step 1.1, inputting point cloud data P ═ { P) of a three-dimensional object1,p2,…,pi,…,pNIn which p isiIndicating the position information of the ith point, where N is point _ num, which is the number of points of the object, N is 2048, the batch size is 4, and the one-time training period epoch is 200;
step 1.2, in order to make the model have arrangement invariance to the input, the information of each point is aggregated by using a symmetric function to obtain an affine matrix A:
f({p1,…,pN})≈g(h(p1),…,h(pN)), (1)
in the formula (1), h represents a multilayer perceptron network, g represents a maximum pooling function, and f represents the characteristic information of a captured point cloud set;
and step 1.3, after the input point cloud data is multiplied by the affine matrix A, the input point cloud coordinate alignment is realized through a plurality of MLP processing.
Step 2, local feature extraction: in order to capture local features, each point in the step 1 is taken as a vertex, k points with the nearest distance are selected to construct a k nearest neighborhood graph, edge convolution is applied to edges connecting adjacent point pairs to obtain edge feature information between every two point pairs, and then local feature matrix information is extracted through MLP operation of a multilayer perceptron network;
step 2.1, selecting k nearest neighbor points k as 20 by using Euclidean distance as a measurement standard;
step 2.2, each point p of the point cloud data obtained after the alignment in the step 1 is processediAnd its k nearest neighbors qjJ-1, 2, …, k constructs a k-neighborhood domain map, where
RFRepresenting an F-dimensional input point cloud matrix.
Step 2.3, the k-neighborhood map structure is G ═ (V, E), where V ═ { p ═ piI | ═ 1, …, N } represents a set of vertices, E ═ Ei＝(ei1,ei2,…,eij,…eik) I | ═ 1,2, …, N } represents the set of edges between vertices, eijRepresents a point piDirected edges with k neighbor points;
and 2.4, performing MLP convolution processing on the k near neighborhood map obtained after updating in the step 2.3 to obtain local characteristic matrix information of the three-dimensional point cloud object.
step 3.1, the MLP comprises two convolution layers, a batch processing normalization layer and an activation layer, wherein the sizes of convolution kernels of the two convolution layers are 64 and 64 from left to right, and output point cloud data of each layer of the MLP network is
l represents the l-th layer of the MLP network;
step 3.2, obtaining a different output k near-neighborhood graph G for each MLP layer according to the output point cloud data of each MLP network layerl＝(Vl,El)，GlK-neighborhood graph, V, representing the output of the l-th layerlRepresents GlA set of corresponding vertices is created and is,Elrepresents GlA set of edges between corresponding vertices;
and 3.3, acquiring local feature matrix information of each layer according to the k near neighborhood graph output by each layer.
Step 4, point cloud feature transformation: designing a new T-Net mini network, inputting the updated local feature matrix information into the new T-Net network for coordinate alignment, and ensuring the rotation invariance of feature points;
the new designed T-Net network is characterized in that a regularization item shown in a formula (3) is added in the softmax training loss of a conventional T-Net network framework, and a characteristic transformation matrix is limited to be close to an orthogonal matrix;
in the formula (3), A is an affine matrix predicted by the T-Net network, so that input information cannot be lost by orthogonal transformation, I is an identity matrix, and F represents a point cloud dimension.
And 5, fusing local features and global features: and carrying out MaxPolling maximum pooling operation on the aligned and updated local feature matrix information, and processing the global feature information obtained by the operation through a plurality of MLPs to obtain the category fraction of each object, thereby realizing the part-level segmentation of the three-dimensional object.
Step 5.1, using the aligned and updated local feature matrix information as input, and performing convolution operation processing through 3 spatial _ transform modules to respectively obtain three local feature information nets1，net2，net3The spatial _ transform module is shown in fig. 2, wherein the spatial _ transform module is composed of an edge convolution module and a plurality of MLP multi-layer perceptron modules;
step 5.2, respectively converting three local features corresponding to the 3 spatial _ transform modules: net1，net2，net3Splicing, and outputting out;
step 5.3, carrying out MaxPolling maximum pooling operation on the output out data to obtain the global characteristics of the current layer;
step 5.4, splicing the global features of the current layer with the tag information of the input point cloud in the data set to obtain global feature global _ feature of the whole network;
step 5.5, global _ feature and net of global feature1，net2，net3Splicing is carried out;
and 5.6, carrying out MLP multi-layer perceptron processing on the data spliced in the step 5.5 to obtain the category score conditions of all the objects, and realizing the part-level segmentation of the three-dimensional objects as shown in FIG. 3.
In the training process, the cross entropy loss function is used for learning parameters, so that the model reaches a convergence state, and the error of the predicted value of the model is reduced. As shown in fig. 4, fig. 4(a) shows a loss curve during the training process, fig. 4(b) shows an accuracy curve, and as the accuracy of the training set increases with the increase of epoch, loss steadily decreases, which indicates that the learning model of the present invention has good adaptability to the training set.
According to the method, due to the fact that the point cloud is lack of topological information essentially, the graph convolution neural network can run on the graph structure directly and can capture dependency relations in the graph by means of information transfer among nodes in the graph, and aiming at the limitation of local feature extraction in the PointNet framework, the method utilizes the graph convolution neural network to extract features of a central point and edge vectors of the central point and k adjacent domain points of the central point to obtain local features of the point cloud, and the problem that the PointNet network cannot extract the local structure is effectively solved.
Claims (6)
1. The point cloud part level segmentation method based on PointNet graph convolution and KNN search is characterized by comprising the following steps:
step 1, point cloud space alignment: inputting point cloud data, predicting an affine transformation matrix by using a T-Net micro network, and performing coordinate alignment on the input point cloud data by using the matrix;
step 2, local feature extraction: in order to capture local features, each point in the step 1 is taken as a vertex, k points with the nearest distance are selected to construct a k nearest neighborhood graph, edge convolution is applied to edges connecting adjacent point pairs to obtain edge feature information between every two point pairs, and then local feature matrix information is extracted through MLP operation of a multilayer perceptron network;
step 3, dynamically updating the local neighborhood map: calculating and updating a k neighbor graph of each layer of the MLP according to the embedding sequence, and extracting updated local feature matrix information;
step 4, point cloud feature transformation: designing a new T-Net mini network, inputting the updated local feature matrix information into the new T-Net network for coordinate alignment, and ensuring the rotation invariance of feature points;
and 5, fusing local features and global features: and carrying out MaxPolling maximum pooling operation on the aligned and updated local feature matrix information, and processing the global feature information obtained by the operation through a plurality of MLPs to obtain the category fraction of each object, thereby realizing the part-level segmentation of the three-dimensional object.
2. The point cloud part level segmentation method based on PointNet graph convolution and KNN search of claim 1, wherein the step 1 specifically comprises:
step 1.1, inputting point cloud data P ═ { P) of a three-dimensional object1,p2,…,pi,…,pNIn which p isiIndicating the position information of the ith point, where N is point _ num, which is the number of points of the object, N is 2048, the batch size is 4, and the one-time training period epoch is 200;
step 1.2, in order to make the model have arrangement invariance to the input, the information of each point is aggregated by using a symmetric function to obtain an affine matrix A:
f({p1,…,pN})≈g(h(p1),…,h(pN)), (1)
in the formula (1), h represents a multilayer perceptron network, g represents a maximum pooling function, and f represents the characteristic information of a captured point cloud set;
and step 1.3, after the input point cloud data is multiplied by the affine matrix A, the input point cloud coordinate alignment is realized through a plurality of MLP processing.
3. The point cloud part level segmentation method based on PointNet's atlas convolution and KNN search of claim 2, wherein the step 2 specifically is:
step 2.1, selecting k nearest neighbor points k as 20 by using Euclidean distance as a measurement standard;
step 2.2, each point p of the point cloud data obtained after the alignment in the step 1 is processediAnd its k nearest neighbors qjJ-1, 2, …, k constructs a k-neighborhood domain map, where
RFRepresenting an F-dimensional input point cloud matrix;
step 2.3, the k-neighborhood map structure is G ═ (V, E), where V ═ { p ═ piI | ═ 1, …, N } represents a set of vertices, E ═ Ei＝(ei1,ei2,…,eij,…eik) I | ═ 1,2, …, N } represents the set of edges between vertices, eijRepresents a point piDirected edges with k neighbor points;
and 2.4, performing MLP convolution processing on the k near neighborhood map obtained after updating in the step 2.3 to obtain local characteristic matrix information of the three-dimensional point cloud object.
4. The point cloud part level segmentation method based on PointNet graph convolution and KNN search of claim 1, wherein the step 3 specifically comprises:
step 3.1, the MLP comprises two convolution layers, a batch processing normalization layer and an activation layer, wherein the sizes of convolution kernels of the two convolution layers are 64 and 64 from left to right, and output point cloud data of each layer of the MLP network is
l represents the l-th layer of the MLP network;
step 3.2, obtaining a different output k near-neighborhood graph G for each MLP layer according to the output point cloud data of each MLP network layerl＝(Vl,El)，GlDenotes the l-th layerOutput k-neighborhood graph, VlRepresents GlCorresponding set of vertices, ElRepresents GlA set of edges between corresponding vertices;
and 3.3, acquiring local feature matrix information of each layer according to the k near neighborhood graph output by each layer.
5. The point cloud part level segmentation method based on PointNet graph convolution and KNN search as claimed in claim 1, wherein the new T-Net network designed in step 4 is to add a regularization term as shown in formula (3) to the softmax training loss of the conventional T-Net network framework, and limit the feature transformation matrix to be close to the orthogonal matrix;
in the formula (3), A is an affine matrix predicted by the T-Net network, so that input information cannot be lost by orthogonal transformation, I is an identity matrix, and F represents a point cloud dimension.
6. The point cloud part level segmentation method based on PointNet graph convolution and KNN search of claim 1, wherein the step 5 is as follows:
step 5.1, using the aligned and updated local feature matrix information as input, and performing convolution operation processing through 3 spatial _ transform modules to respectively obtain three local feature information nets1，net2，net3；
Step 5.2, respectively converting three local features corresponding to the 3 spatial _ transform modules: net1，net2，net3Splicing, and outputting out;
step 5.3, carrying out MaxPolling maximum pooling operation on the output out data to obtain the global characteristics of the current layer;
step 5.4, splicing the global features of the current layer with the tag information of the input point cloud in the data set to obtain global feature global _ feature of the whole network;
step 5.5, global _ feature and net of global feature1，net2，net3Splicing is carried out;
and 5.6, carrying out MLP multi-layer perceptron processing on the data spliced in the step 5.5 to obtain the category score conditions of all the objects, and realizing the part-level segmentation of the three-dimensional objects.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111442718.2A CN114170410B (en) | 2021-11-30 | 2021-11-30 | Point cloud part classification method based on PointNet graph convolution and KNN search |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111442718.2A CN114170410B (en) | 2021-11-30 | 2021-11-30 | Point cloud part classification method based on PointNet graph convolution and KNN search |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114170410A true CN114170410A (en) | 2022-03-11 |
| CN114170410B CN114170410B (en) | 2024-09-06 |
Family
ID=80481933
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111442718.2A Active CN114170410B (en) | 2021-11-30 | 2021-11-30 | Point cloud part classification method based on PointNet graph convolution and KNN search |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114170410B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114693932A (en) * | 2022-04-06 | 2022-07-01 | 南京航空航天大学 | Large aircraft large component point cloud semantic segmentation method |
| CN114743123A (en) * | 2022-04-29 | 2022-07-12 | 电子科技大学 | Scene understanding method based on implicit function three-dimensional representation and graph neural network |
| CN114758061A (en) * | 2022-03-16 | 2022-07-15 | 南京理工大学 | Method for constructing temperature field based on three-dimensional model point cloud grid data |
| CN114973422A (en) * | 2022-07-19 | 2022-08-30 | 南京应用数学中心 | Gait recognition method based on three-dimensional human body modeling point cloud feature coding |
| CN115049833A (en) * | 2022-06-30 | 2022-09-13 | 西安理工大学 | Point cloud component segmentation method based on local feature enhancement and similarity measurement |
| CN116128835A (en) * | 2023-02-02 | 2023-05-16 | 南方电网数字电网研究院有限公司 | Point cloud analysis-based power transmission wire sag measurement method, device and equipment |
| CN116164648A (en) * | 2023-04-20 | 2023-05-26 | 武汉嘉晨电子技术有限公司 | Automatic detection method for BDU automobile harness connector terminal |
| CN116563310A (en) * | 2023-05-16 | 2023-08-08 | 澳门大学 | Point cloud segmentation method, device, equipment and storage medium |
| CN117033686A (en) * | 2023-08-04 | 2023-11-10 | 大连海事大学 | Mechanical part three-dimensional model retrieval method based on feature extraction |
| CN118898751A (en) * | 2024-09-30 | 2024-11-05 | 山东大学 | Detection method and system based on weak projection mapping and dual-modal sequential enhancement fusion |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020173297A1 (en) * | 2019-02-25 | 2020-09-03 | 腾讯科技（深圳）有限公司 | Point cloud segmentation method, computer-readable storage medium and computer device |
| CN112488210A (en) * | 2020-12-02 | 2021-03-12 | 北京工业大学 | Three-dimensional point cloud automatic classification method based on graph convolution neural network |
| CN113628217A (en) * | 2021-08-12 | 2021-11-09 | 江南大学 | Three-dimensional point cloud segmentation method based on image convolution and integrating direction and distance |
| CN115049833A (en) * | 2022-06-30 | 2022-09-13 | 西安理工大学 | Point cloud component segmentation method based on local feature enhancement and similarity measurement |
-
2021
- 2021-11-30 CN CN202111442718.2A patent/CN114170410B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020173297A1 (en) * | 2019-02-25 | 2020-09-03 | 腾讯科技（深圳）有限公司 | Point cloud segmentation method, computer-readable storage medium and computer device |
| CN112488210A (en) * | 2020-12-02 | 2021-03-12 | 北京工业大学 | Three-dimensional point cloud automatic classification method based on graph convolution neural network |
| CN113628217A (en) * | 2021-08-12 | 2021-11-09 | 江南大学 | Three-dimensional point cloud segmentation method based on image convolution and integrating direction and distance |
| CN115049833A (en) * | 2022-06-30 | 2022-09-13 | 西安理工大学 | Point cloud component segmentation method based on local feature enhancement and similarity measurement |
Non-Patent Citations (5)
| Title |
|---|
| YAODONG CUI 等: "Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review", IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, vol. 23, no. 2, 17 March 2021 (2021-03-17), pages 722 - 739 * |
| 宁小娟 等: "结合语义分割与模型匹配的室内场景重建方法", 中国图象图形学报, vol. 28, no. 10, 16 October 2023 (2023-10-16), pages 3149 - 3162 * |
| 张新良;付鹏飞;赵运基;谢恒;王琬如;: "融合图卷积和差异性池化函数的点云数据分类分割模型", 中国图象图形学报, no. 06, 16 June 2020 (2020-06-16), pages 137 - 144 * |
| 点云PCL博主: "结合代码理解Pointnet网络结构", Retrieved from the Internet <URL:https://cloud.tencent.com/developer/article/1605590> * |
| 韩姗姗;黄远程;白穆;: "基于DGCNN语义分割的倾斜摄影测量三维点云建筑物分类", 测绘标准化, no. 03, 25 September 2020 (2020-09-25), pages 25 - 30 * |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114758061A (en) * | 2022-03-16 | 2022-07-15 | 南京理工大学 | Method for constructing temperature field based on three-dimensional model point cloud grid data |
| CN114693932A (en) * | 2022-04-06 | 2022-07-01 | 南京航空航天大学 | Large aircraft large component point cloud semantic segmentation method |
| CN114743123A (en) * | 2022-04-29 | 2022-07-12 | 电子科技大学 | Scene understanding method based on implicit function three-dimensional representation and graph neural network |
| CN115049833A (en) * | 2022-06-30 | 2022-09-13 | 西安理工大学 | Point cloud component segmentation method based on local feature enhancement and similarity measurement |
| CN114973422A (en) * | 2022-07-19 | 2022-08-30 | 南京应用数学中心 | Gait recognition method based on three-dimensional human body modeling point cloud feature coding |
| CN116128835B (en) * | 2023-02-02 | 2023-09-15 | 南方电网数字电网研究院有限公司 | Point cloud analysis-based power transmission wire sag measurement method, device and equipment |
| CN116128835A (en) * | 2023-02-02 | 2023-05-16 | 南方电网数字电网研究院有限公司 | Point cloud analysis-based power transmission wire sag measurement method, device and equipment |
| CN116164648A (en) * | 2023-04-20 | 2023-05-26 | 武汉嘉晨电子技术有限公司 | Automatic detection method for BDU automobile harness connector terminal |
| CN116563310A (en) * | 2023-05-16 | 2023-08-08 | 澳门大学 | Point cloud segmentation method, device, equipment and storage medium |
| CN116563310B (en) * | 2023-05-16 | 2025-07-11 | 澳门大学 | Point cloud segmentation method, device, equipment and storage medium |
| CN117033686A (en) * | 2023-08-04 | 2023-11-10 | 大连海事大学 | Mechanical part three-dimensional model retrieval method based on feature extraction |
| CN118898751A (en) * | 2024-09-30 | 2024-11-05 | 山东大学 | Detection method and system based on weak projection mapping and dual-modal sequential enhancement fusion |
| CN118898751B (en) * | 2024-09-30 | 2025-01-24 | 山东大学 | Detection method and system based on weak projection mapping and dual-modal sequential enhancement fusion |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114170410B (en) | 2024-09-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114170410A (en) | Point cloud part level segmentation method based on PointNet graph convolution and KNN search | |
| CN111489358B (en) | Three-dimensional point cloud semantic segmentation method based on deep learning | |
| CN112633350B (en) | A multi-scale point cloud classification implementation method based on graph convolution | |
| CN105740894B (en) | A Semantic Labeling Method for Hyperspectral Remote Sensing Images | |
| CN112364730B (en) | Method and system for automatic classification of hyperspectral objects based on sparse subspace clustering | |
| CN113361485B (en) | A hyperspectral image classification method based on spectral spatial attention fusion and deformable convolutional residual network | |
| CN110443818A (en) | A kind of Weakly supervised semantic segmentation method and system based on scribble | |
| CN111079847B (en) | Remote sensing image automatic labeling method based on deep learning | |
| CN109508654A (en) | Merge the human face analysis method and system of multitask and multiple dimensioned convolutional neural networks | |
| CN111507222A (en) | Three-dimensional object detection framework based on multi-source data knowledge migration | |
| CN112052783A (en) | High-resolution image weak supervision building extraction method combining pixel semantic association and boundary attention | |
| CN109086777B (en) | Saliency map refining method based on global pixel characteristics | |
| CN113989340A (en) | Point cloud registration method based on distribution | |
| CN115311502A (en) | A small sample scene classification method for remote sensing images based on multi-scale dual-stream architecture | |
| CN113011568A (en) | Model training method, data processing method and equipment | |
| CN110263855A (en) | A method of it is projected using cobasis capsule and carries out image classification | |
| WO2020252746A1 (en) | Method for performing image classification by using common-base capsule projection | |
| CN116935486A (en) | Sign language identification method and system based on skeleton node and image mode fusion | |
| CN118485932A (en) | UAV cross-modal fusion detection method based on CFT-EfficientDet | |
| Wu et al. | Sc-wls: Towards interpretable feed-forward camera re-localization | |
| CN110598746A (en) | Adaptive scene classification method based on ODE solver | |
| CN116468895A (en) | Similarity matrix guided few-sample semantic segmentation method and system | |
| CN114373177A (en) | Three-dimensional target detection method for intermediate supervision by using label coding | |
| CN114663880A (en) | Three-dimensional target detection method based on multi-level cross-modal self-attention mechanism | |
| CN111368733A (en) | A 3D hand pose estimation method, storage medium and terminal based on label distribution learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |