CN117333677B - A method for extracting building vector outlines - Google Patents
A method for extracting building vector outlines Download PDFInfo
- Publication number
- CN117333677B CN117333677B CN202311629268.7A CN202311629268A CN117333677B CN 117333677 B CN117333677 B CN 117333677B CN 202311629268 A CN202311629268 A CN 202311629268A CN 117333677 B CN117333677 B CN 117333677B
- Authority
- CN
- China
- Prior art keywords
- contour
- vector
- building
- vertex
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
- G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
- G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
- G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/176—Urban or other man-made structures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Biodiversity & Conservation Biology (AREA)
- Image Analysis (AREA)
Abstract
本发明提供一种建筑物矢量轮廓提取方法，将遥感影像输入矢量多边形轮廓提取模型，获取遥感影像中建筑物目标的矢量多边形轮廓；其中，矢量多边形轮廓提取模型包括轮廓提取模型和顶点分类模型，所述轮廓提取模型，用于提取遥感影像中建筑物目标的矢量多边形轮廓，所述顶点分类模型，用于对所述轮廓提取模型提取的建筑物目标的矢量多边形轮廓的顶点进行分类，基于有效的顶点构成最终的建筑物目标的矢量多边形轮廓。本发明通过构建轮廓提取模型和顶点分类模型，并构成矢量多边形轮廓提取模型，只需要向矢量多边形轮廓提取模型输入遥感影像，即可直接输出遥感影像中建筑物目标的矢量多边形轮廓，构建的模型准确度高，提取过程简便。
The present invention provides a method for extracting vector contours of buildings, wherein a remote sensing image is input into a vector polygon contour extraction model to obtain the vector polygon contours of building targets in the remote sensing image; wherein the vector polygon contour extraction model includes a contour extraction model and a vertex classification model, wherein the contour extraction model is used to extract the vector polygon contours of building targets in the remote sensing image, and the vertex classification model is used to classify the vertices of the vector polygon contours of the building targets extracted by the contour extraction model, and the final vector polygon contours of the building targets are formed based on valid vertices. The present invention constructs a contour extraction model and a vertex classification model, and forms a vector polygon contour extraction model. It only needs to input the remote sensing image into the vector polygon contour extraction model to directly output the vector polygon contours of the building targets in the remote sensing image. The constructed model has high accuracy and a simple extraction process.
Description
Technical Field
The invention relates to the field of building contour extraction, in particular to a building vector contour extraction method.
Background
The vector outline of the building is extracted from the remote sensing image, and the method has important significance in the application fields of map making, city planning, space analysis, change detection and the like. However, because the background of the remote sensing image is complex, the outline shape of the building is different and the scale is different, the automatic extraction of the regular vector outline of the building from the remote sensing image still faces the challenges of low accuracy, large calculation amount and the like.
With the continuous development of computer vision and other technologies, a deep learning method based on semantic segmentation is gradually applied to remote sensing image building extraction tasks. The current mainstream method is to conduct pixel-by-pixel dense prediction on the remote sensing image, divide the remote sensing image into two types of buildings and backgrounds, obtain a binary image extracted from the building, and output vector outline polygon information of the building through further processing. However, this method has problems of complicated operation steps, large loss of contour accuracy, serious aliasing of the pattern, and the like, and the pixel-by-pixel convolution calculation requires more calculation amount.
Aiming at the remote sensing image building contour extraction task, a novel solution is provided by a general image target contour extraction scheme represented by DEEPSNAKE. However, the outline polygon output by such a scheme has redundant vertexes and smooth outline, and is not suitable for a plurality of building geometries with prominent right angle characteristics, so that the extraction accuracy is low. Therefore, there is still a need to find a more accurate end-to-end remote sensing image building vector contour extraction method.
Disclosure of Invention
The invention provides a building vector contour extraction method aiming at the technical problems in the prior art, which comprises the following steps:
inputting the remote sensing image into a vector polygon contour extraction model to obtain a vector polygon contour of a building target in the remote sensing image;
The vector polygonal contour extraction model comprises a contour extraction model and a vertex classification model, wherein the contour extraction model is used for extracting vector polygonal contours of building targets in remote sensing images, and the vertex classification model is used for classifying vertexes of the vector polygonal contours of the building targets extracted by the contour extraction model and forming final vector polygonal contours of the building targets based on effective vertexes.
According to the building vector contour extraction method, the contour extraction model and the vertex classification model are constructed, the vector polygonal contour extraction model is formed, the vector polygonal contour of a building target in the remote sensing image can be directly output only by inputting the remote sensing image into the vector polygonal contour extraction model, the accuracy of the constructed model is high, and the extraction process is simple and convenient.
Drawings
FIG. 1 is a flow chart of a building vector contour extraction method provided by the invention;
FIG. 2 is a schematic diagram of a vertex shift network;
FIG. 3 is a schematic illustration of the outline of an circumscribing polygon taken from different building shapes;
fig. 4 is a schematic diagram of steps for extracting various contours in the building vector contour extraction method provided by the invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention. In addition, the technical features of each embodiment or the single embodiment provided by the invention can be combined with each other at will to form a feasible technical scheme, and the combination is not limited by the sequence of steps and/or the structural composition mode, but is necessarily based on the fact that a person of ordinary skill in the art can realize the combination, and when the technical scheme is contradictory or can not realize, the combination of the technical scheme is not considered to exist and is not within the protection scope of the invention claimed.
In order to solve the problems that the traditional semantic segmentation method needs secondary processing, the operation flow is complex (end-to-end can not be achieved), and the calculation cost is high and the accuracy is high. According to the end-to-end building vector contour extraction method constructed by the invention, through the deep learning network, the input remote sensing image can directly obtain the vector polygonal contour of the building, and compared with a semantic segmentation mode needing secondary processing, the operation flow is more convenient; compared with a pixel-by-pixel dense calculation model, the method only needs to carry out convolution transformation on the points on the outline, and has the advantages of lower calculation cost and higher execution efficiency.
Fig. 1 is a flow chart of a method for extracting a vector outline of a building, which is provided by the invention, as shown in fig. 1, and the method comprises the following steps:
Step 1, a contour extraction model and a vertex classification model are constructed, wherein the contour extraction model is used for extracting vector polygonal contours of building targets in remote sensing images, the vertex classification model is used for classifying vertices of the vector polygonal contours of the building targets extracted by the contour extraction model so as to form final vector polygonal contours of the building targets based on effective vertices in the vector polygonal contours of the building targets, and the contour extraction module and the vertex classification module form the vector polygonal contour extraction model.
The profile extraction model comprises a feature extraction module, a profile initialization module and a profile evolution module, wherein the feature extraction module uses a depth residual error network (DLA 34 model) with a multi-level fusion structure, a rectangular detection frame , for extracting a remote sensing image feature matrix F and a building target represents the abscissa of the ith vertex of the rectangular detection frame, and a/> represents the ordinate of the ith vertex of the rectangular detection frame.
The contour initialization module comprises a vertex offset network, which is used for acquiring the external polygonal contour of the building target, wherein the model reasoning value of the external polygonal contour is recorded as , i is an integer, the actual value of the external polygonal contour of the building target is recorded as/> , i is an integer, and is the vertex number of the external polygonal contour of the building target.
The contour evolution module comprises three vertex offset networks connected in series, and is used for acquiring the vector polygonal contour of the building target, wherein the model reasoning value of the vector polygonal contour is recorded as , i is an integer, the actual value of the vector polygonal contour of the building target is recorded as/> , i is an integer, and/> is the number of vertices of the vector polygonal contour of the building target.
The vertex offset network is shown in fig. 2, and the vertex offset network structure comprises 7 serial ring convolution layers, 1 single convolution layer and 2 serial convolution layers, wherein the 7 serial ring convolution layers are used for extracting detailed information, local information and global information of the remote sensing image, the 1 convolution layer is used for fusing information extracted by the ring convolution layers, and the 2 serial convolution layers are used for obtaining reasoning results.
The torus convolution layer, cirConv shown in fig. 2, is a special convolution operation. Compared with the traditional convolution layer, cirConv copies adj features from the head and tail of the input feature map respectively, and adds the copied head features to the tail, and the copied tail features to the head, so that the head and tail features are connected to form a feature map like a circular ring. The input of the vertex offset network is polygon vertex coordinate array and corresponding feature matrix, and the output is the offset of the vertex coordinates.
Wherein, the adj is set to 4 in the invention. The vertex shift network structure as shown in fig. 2, cirConv in the figure represents a torus convolution; reLU represents a ReLU activation function; BN represents normalization calculation; conv1d represents a one-dimensional convolution calculation. D in the figure represents the expansion parameter (dilation) of the convolution layer, k represents the convolution kernel size parameter (kernel) of the convolution layer, and the convolution kernel sizes of CirConv layers without the k parameter are all 9.
As an embodiment, the training process of the contour extraction model includes: constructing a first training data set of the contour initialization module and a second training data set of the contour evolution module; training the contour initialization module based on the first training data set, and training the contour evolution module based on the second training data set.
The process of constructing the first training data set of the contour initialization module and the second training data set of the contour evolution module mainly comprises the following steps:
Acquiring a remote sensing image and cutting the remote sensing image into a plurality of tile images, and respectively marking a rectangular detection frame P box of a building target and a target vector polygonal contour of the building target on each tile image, wherein/> is the number of vertexes of the target vector polygonal contour;
Acquiring an external polygonal contour P ex of the building target according to the target vector polygonal contour ;
Adopting a uniform interpolation method for the external polygonal contour P ex of the building target, expanding the number of vertexes of the external polygonal contour P ex of the building target to N init, and acquiring an actual external polygonal contour P init of the building target to construct a first training data set of the contour initialization module;
And (3) for the target vector polygonal contour P gt of the building target, adopting a uniform interpolation method, expanding the number of vertexes of the target vector polygonal contour P gt of the building target to N poly, and acquiring the actual vector polygonal contour P poly of the building target so as to construct a second training data set of the contour evolution module.
Specifically, the method for acquiring the first training data set and the second training data set comprises the following steps:
(1) And acquiring a remote sensing image and cutting the remote sensing image into tile images with the width W and the height H, and respectively labeling a target detection frame rectangle P box of a building and a target vector polygonal contour of a building target on the tile images, wherein i is an integer, and/> is the number of vertexes of the target vector polygonal contour.
(2) Obtaining an external polygonal outline P ex of a building target according to P gt, wherein the external polygonal outline P ex is specifically:
First, four extrema of are obtained: x-coordinate minimum/> , y-coordinate minimum/> , x-coordinate maximum/> , y-coordinate maximum/> .
For extremum , > , a set of vertices where the difference between the x-coordinate and/> is less than or equal to t is obtained. The two vertices with the largest and smallest y coordinates in/> are taken as the polar edges of the extremum/> .
For extremum , > , a set of vertices where the difference between the y coordinate and/> is less than or equal to t is obtained. The two vertices with the largest and smallest x-coordinates in/> are taken as the polar edges of the extremum/> .
And obtaining extreme values and extreme values/> in the same mode, and finally performing de-duplication operation on eight vertexes on four extreme edges of the four extreme values to obtain the external polygonal contour/> of the building target, wherein i is an integer, and/> is the number of vertexes of the external polygonal contour of the building target. The outline of the circumscribed polygon of different building shapes can be seen in fig. 3, and t is set to be 0.01, and the letter,/> is set to be 4-8.
(3) Obtaining an actual value P init of the external polygonal contour of the building target according to the external polygonal contour P ex of the building target, specifically, adopting a uniform interpolation method to expand the number of the vertexes of the external polygonal contour P ex of the building target to N init to obtain an actual value P init of the external polygonal contour of the building target so as to construct a first training data set. The invention sets N init to 40.
(4) According to the marked target vector polygonal contour P gt of the building target, the actual value P poly of the vector polygonal contour of the building target is obtained, specifically, a uniform interpolation method is adopted, the number of the vertexes of the target vector polygonal contour P gt of the building target is expanded to N poly, and the actual value P poly of the vector polygonal contour of the building target can be obtained, so that the second training data set is constructed. The present invention sets N poly to 128.
The first training data set of the contour initialization model and the second training data set of the contour evolution module are constructed, and then the contour initialization module and the contour evolution module in the contour extraction model are respectively trained, wherein the training process of the contour initialization module is as follows:
Performing data enhancement operation on the remote sensing tile images in a rotating, zooming mode and the like, inputting the remote sensing images in a disordered order into the feature extraction module, and outputting a remote sensing image feature matrix F and rectangular detection frames P box of a plurality of building targets;
Expanding the number of vertexes of the rectangular detection frames P box of each building target to N init by adopting a uniform interpolation method;
Extracting a feature matrix F box of a rectangular detection frame P box of the building target from the remote sensing image feature matrix F according to the vertex coordinate value of the rectangular detection frame P box of the building target;
Inputting the combined result of F box and P box (F box,Pbox) into the contour initialization module to obtain an offset O box, and calculating an inference value of the circumscribed polygon contour of the building target as follows:
；
And calculating a first error between an inferred value of the circumscribed polygonal contour of the building target and an actual circumscribed polygonal contour P init of the building target based on the smoothing loss function, and obtaining a contour initialization module when the first error is minimum.
The training process for the contour evolution module is as follows:
Expanding the number of vertexes of the inferred value of the circumscribed polygonal outline of the building target to N poly by adopting a uniform interpolation method;
Extracting a feature matrix/> of the inferred value/> of the circumscribed polygonal contour of the building target from the remote sensing image feature matrix F according to the vertex coordinate value of the inferred value of the circumscribed polygonal contour of the building target;
inputting the result of and/> combination/> into the contour evolution module to obtain an offset O init, and calculating the reasoning value/> of the vector polygonal contour of the building target as follows:
；
And calculating a second error between the inferred value of the vector polygonal contour of the building target and the actual vector polygonal contour P poly of the building target based on the bidirectional nearest loss function, and acquiring a trained contour evolution model converged by the second error.
Wherein calculating a second error between the inferred value of the vector polygonal contour of the building target and the actual vector polygonal contour P poly of the building target based on the bi-directional nearest loss function comprises:
Let the i-th point in be the nearest Manhattan distance L 1 between the i-th point in/> , and/> of/> , and the distance L 1 between/> , and/> be/> , respectively
；
The average nearest distance between the inferred value of the vector polygonal contour of the building target and the actual vector polygonal contour P poly of the building target is/> :
；
Let the j-th point in be the closest L 1 distance from the/> point in/> , to/> , and the L1 distance from/> , to/> be/> :
；
The average nearest distance between the actual vector polygonal contour P poly of the building target and the inferred value of the vector polygonal contour of the building target is/> , then:
；
Defining the bi-directional nearest loss function as the average of and/> , denoted/> , there are:
。
Wherein, in order to conveniently demonstrate the calculation result of the bidirectional nearest loss function, the invention provides an embodiment, and N poly is set to 4.
Assuming that the model inference value = { (0, 0), (-2, 2), (1, 1), (1, 0) }, the actual value/> = { (0, 0), (0, 1), (1, 0) }, the/> = 0.5 is calculated according to the above-mentioned two-way nearest loss function calculation formula.
The contour extraction model is constructed, and a vertex classification model is constructed, wherein the vertex classification model comprises a vertex offset network and a sigmoid activation layer, the output of the vertex classification model is the classification of the vertex, the value range is 0-1, 0 indicates that the vertex is invalid, and 1 indicates that the vertex is valid.
And obtaining a vertex classification labeling set of the training vertex classification model, specifically, obtaining a vector polygon contour of a building target in the remote sensing image extracted by the contour extraction model, labeling the type of each vertex in the vector polygon contour/> , wherein the type of each vertex is effective or ineffective, the actual value of an ineffective classification Score is 0, and the actual value of an effective classification Score is 1, so as to obtain a vertex classification labeling set Score gt of the vertex classification model.
Wherein labeling the type of each vertex in the vector polygonal contour comprises:
Obtaining a vector polygonal contour of the building target extracted by the contour evolution module, wherein the number of vertexes of the known/> is/> , and the number of vertexes of a target vector polygonal contour/> of the marked building target is/> , and is larger than/> ;
Carrying out optimal matching on the vertexes in and the vertexes in/> based on a weighted bipartite graph optimal matching algorithm, taking the Euclidean distance between the two vertexes as a weight value to enable the sum of the Euclidean distances of a matching result to be minimum, wherein the matching result is a vertex pair of and/> ,/> is a subset of/> , the number of the vertexes of/> is equal to that of/> , and marking the vertexes in as 1 to represent effective vertexes; labeling the vertex of/> as 0, representing an invalid point;
The constructed vertex classification label set is denoted Score gt.
Training the vertex classification model based on a third training data set, wherein the specific training process comprises the following steps:
extracting a feature matrix F of the remote sensing image based on a feature extraction module;
extracting a feature matrix/> of a vector polygonal contour/> of the building target from the feature matrix F according to the vertex coordinate values of the vector polygonal contour of the building target;
Inputting and/> combined results/> into the vertex classification model to obtain classification Score reasoning values Score pre of each vertex in/> ;
Calculating a third error between the classification Score reasoning value Score pre and the classification Score actual value Score gt in the vertex classification annotation set based on the Focal Loss function;
and obtaining the vertex classification model when the third error is minimum.
And 2, inputting the remote sensing image into a vector polygon contour extraction model to obtain the vector polygon contour of the building target in the remote sensing image.
It can be understood that in step 1, a contour extraction model and a vertex classification model are respectively constructed, and the trained contour extraction model and the trained vertex classification model are connected to form a remote sensing image vector polygon contour extraction model.
In reasoning, the input remote sensing image firstly obtains the vector polygon outline of the building target through the outline extraction module, then inputs the vector polygon outline of the building target into the vertex classification module to obtain the classification score of each vertex, regards the vertices with the score smaller than the preset score value score as invalid vertices, and deletes the invalid vertices to obtain the final building target vector polygon outline. In the present invention, the preset score value score is set to 0.85.
Referring to fig. 4, for a schematic diagram of each contour extraction step in a building vector contour provided by the invention, firstly, a remote sensing image is input into a feature extraction module in a contour extraction model to output a rectangular detection frame of a building target, then an external polygonal contour of the building target is output through a contour initialization module, then a vector polygonal contour of the building target is output through a contour evolution module, finally, the validity classification is carried out on the vertexes in the vector polygonal contour of the building target through a vertex classification model, and the final vector polygonal contour of the building target is formed according to the valid vertexes.
The invention provides a building vector contour extraction method, which has the following advantages:
(1) The invention constructs the deep learning network structure, realizes the end-to-end remote sensing image building vector contour extraction process, can directly obtain building vector contour data by inputting one remote sensing image, avoids secondary processing, and has lower calculation cost and higher accuracy.
(2) When the method adopts an external polygonal initializing mode, the shape of the building target can be better wrapped, and the accuracy of the initializing stage is improved. In the contour evolution stage, two-way nearest loss functions are used for promoting each point on the contour to move towards the vertex direction which is closer to the contour, so that convergence is quickened. In the contour simplification stage, a weighted bipartite graph optimal matching algorithm is adopted to construct a training data set, so that a trained module can distinguish whether vertexes are redundant or not, the contour accuracy is further improved, and the output vector polygons can express geographic information more closely.
In the foregoing embodiments, the descriptions of the embodiments are focused on, and for those portions of one embodiment that are not described in detail, reference may be made to the related descriptions of other embodiments.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (8)
1. A building vector contour extraction method, comprising:
inputting the remote sensing image into a vector polygon contour extraction model to obtain a vector polygon contour of a building target in the remote sensing image;
The vector polygonal contour extraction model comprises a contour extraction model and a vertex classification model, wherein the contour extraction model is used for extracting a vector polygonal contour of a building target in a remote sensing image, and the vertex classification model is used for classifying vertexes of the vector polygonal contour of the building target extracted by the contour extraction model and forming a final vector polygonal contour of the building target based on the effective vertexes;
The contour extraction model comprises a feature extraction module, a contour initialization module and a contour evolution module;
The feature extraction module is a depth residual error network with a multi-level fusion structure, and is used for extracting a remote sensing image feature matrix F and a rectangular detection frame of a building target, wherein the abscissa of the ith vertex of the rectangular detection frame is represented, and the ordinate of the ith vertex of the rectangular detection frame is represented by/> ;
the outline initialization module comprises a vertex offset network, which is used for obtaining the external polygonal outline of the building target according to the remote sensing image feature matrix F and the rectangular detection frame P box of the building target
The contour evolution module comprises three vertex offset networks connected in series, and is used for acquiring the vector polygonal contour/>, of the building target according to the circumscribed polygonal contour of the building target
The training process of the contour extraction model comprises the following steps:
Constructing a first training data set of the contour initialization module and a second training data set of the contour evolution module;
Training the contour initialization module based on the first training data set, and training the contour evolution module based on the second training data set;
wherein constructing the first training data set of the contour initialization module and constructing the second training data set of the contour evolution module includes:
acquiring a remote sensing image and cutting the remote sensing image into a plurality of tile images, and respectively marking a rectangular detection frame P box of a building target and a target vector polygonal contour i of the building target on each tile image as integers, wherein N gt is the number of vertexes of the target vector polygonal contour;
obtaining an external polygonal contour P ex of the building target according to the target vector polygonal contour P gt;
Adopting a uniform interpolation method for the external polygonal contour P ex of the building target, expanding the number of vertexes of the external polygonal contour P ex of the building target to N init, and acquiring an actual external polygonal contour P init of the building target to construct a first training data set of the contour initialization module;
And (3) for the target vector polygonal contour P gt of the building target, adopting a uniform interpolation method, expanding the number of vertexes of the target vector polygonal contour P gt of the building target to N poly, and acquiring the actual vector polygonal contour P poly of the building target so as to construct a second training data set of the contour evolution module.
2. The building vector contour extraction method according to claim 1, wherein the vertex offset network has an input of a vertex coordinate array and a corresponding feature matrix, and an output of an offset of vertex coordinates, and the vertex offset network includes 7 serially connected circular convolution layers, 1 single convolution layer and 2 serially connected convolution layers;
the circular convolution layer CirConv is used for extracting detailed information, local information and global information of the remote sensing image;
The single convolution layer is used for fusing the detail information, the local information and the global information extracted by the circular convolution layer;
2 serially connected convolution layers for obtaining reasoning results according to the fusion information;
The circular convolution layer CirConv copies adj features from the head and tail of the input feature map, and adds the copied head features to the tail, and the copied tail features to the head, so that the head and tail features are connected to form a feature map like a circular ring.
3. The building vector outline extraction method according to claim 1, wherein obtaining an circumscribing polygon outline P ex of a building object from the object vector polygon outline P gt includes:
Four extreme values of the target vector polygonal contour P gt are acquired: x-coordinate minimum y-coordinate minimum/> x-coordinate maximum/> y-coordinate maximum/>
Traversing P gt for the extremum , and acquiring a set of vertexes with differences between x coordinates and/> in P gt being less than or equal to t, and taking two vertexes with the largest y coordinates and the smallest y coordinates in/> as polar edges of the extremum/> ;
Traversing P gt for the extremum , and acquiring a set of vertexes with differences between y coordinates and/> in P gt smaller than t, wherein the two vertexes with the largest x coordinates and the smallest x coordinates in/> are taken as polar edges of the extremum/> ;
Based on the same mode, extreme values and extreme values/> are obtained, the eight vertexes on the four extreme values are subjected to deduplication operation, the external polygonal contour/> i of the building target is obtained as an integer, and N ex is the number of vertexes of the external polygonal contour of the building target.
4. The building vector contour extraction method of claim 1, wherein said training said contour initialization module based on said first training data set comprises:
performing data enhancement operation on the remote sensing images, inputting the remote sensing images in a disordered order into the feature extraction module, and outputting a remote sensing image feature matrix F and rectangular detection frames P box of a plurality of building targets;
Expanding the number of vertexes of the rectangular detection frames P box of each building target to N init by adopting a uniform interpolation method;
Extracting a feature matrix F box of a rectangular detection frame P box of the building target from the remote sensing image feature matrix F according to the vertex coordinate value of the rectangular detection frame P box of the building target;
Inputting the combined result of F box and P box (F box,Pbox) into the contour initialization module to obtain an offset O box, and calculating an inference value of the circumscribed polygon contour of the building target as follows:
And calculating a first error between an inferred value of the circumscribed polygonal contour of the building target and an actual circumscribed polygonal contour P init of the building target based on the smoothing loss function, and obtaining a contour initialization module when the first error is minimum.
5. The building vector contour extraction method of claim 4, wherein said training said contour evolution module based on said second training data set comprises:
expanding the number of vertexes of the inferred value of the circumscribed polygonal outline of the building target to N poly by adopting a uniform interpolation method;
Extracting the feature matrix/> of the inferred value/> of the circumscribed polygonal contour of the building target from the remote sensing image feature matrix F according to the vertex coordinate value of the inferred value of the circumscribed polygonal contour of the building target
Inputting the result of and/> combination/> into the contour evolution module to obtain an offset O init, and calculating the reasoning value/> of the vector polygonal contour of the building target as follows:
And calculating a second error between the inferred value of the vector polygonal contour of the building target and the actual vector polygonal contour P poly of the building target based on the bidirectional nearest loss function, and obtaining a contour evolution module when the second error is minimum.
6. The method of claim 5, wherein calculating a second error between the inferred value of the vector polygonal contour of the building object and the actual vector polygonal contour P poly of the building object based on the bi-directional nearest loss function comprises:
Let the i-th point in be the closest Manhattan distance L 1 between the x i -th point in/> Ppoly and/> , and let the distance L 1 between/> and/> be/>
The average nearest distance L pre2gt between the inferred value of the vector polygonal contour of the building object and the actual vector polygonal contour P poly of the building object is:
Let the j-th point in P poly be the closest distance between the x j -th point in and L 1 of P j poly, and the L1 distance between Pj poly and/> be L 1j gt2pre:
the average nearest distance between the actual vector polygonal contour P poly of the building target and the inferred value of the vector polygonal contour of the building target is L gt2pre, then:
Defining the bi-directional nearest loss function as the average of L pre2gt and L gt2pre, denoted as L poly, there are:
7. The building vector contour extraction method of claim 1, wherein said vertex classification model comprises a vertex shift network and a sigmoid activation layer, said training process of said vertex classification model comprising:
Acquiring a vector polygonal contour of a building target in the remote sensing image extracted by the contour extraction model, marking the type of each vertex in the vector polygonal contour/> , wherein the type of each vertex is effective or ineffective, the actual value of an ineffective classification Score is 0, and the actual value of an effective classification Score is 1, so as to construct a vertex classification marking set Score gt;
extracting a feature matrix F of the remote sensing image based on a feature extraction module;
Extracting a feature matrix/> of a vector polygonal contour/> of the building target from the feature matrix F according to the vertex coordinate values of the vector polygonal contour of the building target
Inputting and/> combined results/> into the vertex classification model to obtain classification Score reasoning values Score pre of each vertex in/> ;
Calculating a third error between the classification Score reasoning value Score pre and the classification Score actual value Score gt in the vertex classification annotation set based on the Focal Loss function;
and obtaining the vertex classification model when the third error is minimum.
8. The method of claim 7, wherein labeling the type of each vertex in the vector polygonal contour comprises:
Obtaining the known/> vertex number N poly of the vector polygonal contour of the building target extracted by the contour evolution module, wherein the labeled vertex number N gt,Npoly of the target vector polygonal contour P gt of the building target is larger than N gt;
Performing optimal matching on the vertexes in and the vertexes in P gt based on a weighted bipartite graph optimal matching algorithm, and taking the Euclidean distance between the two vertexes as a weight value to minimize the sum of the Euclidean distances of a matching result, wherein the matching result is a vertex pair of/> and P gt,/> is a subset of/> , the number of the vertexes of/> is equal to that of P gt, and the vertexes in/> are marked as 1 and represent effective vertexes; labeling the vertex of/> as 0, representing an invalid point;
The constructed vertex classification label set is denoted Score gt.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311629268.7A CN117333677B (en) | 2023-12-01 | 2023-12-01 | A method for extracting building vector outlines |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311629268.7A CN117333677B (en) | 2023-12-01 | 2023-12-01 | A method for extracting building vector outlines |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117333677A CN117333677A (en) | 2024-01-02 |
| CN117333677B true CN117333677B (en) | 2024-04-16 |
Family
ID=89279679
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311629268.7A Active CN117333677B (en) | 2023-12-01 | 2023-12-01 | A method for extracting building vector outlines |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117333677B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107328371A (en) * | 2017-05-22 | 2017-11-07 | 四川大学 | Sub-pix contours extract based on Gaussian and the optimization using Softmax recurrence in the case where metal plate detects scene |
| CN113658224A (en) * | 2021-08-18 | 2021-11-16 | 中国人民解放军陆军炮兵防空兵学院 | Target contour tracking method and system based on correlated filtering and Deep Snake |
| CN114820668A (en) * | 2022-03-09 | 2022-07-29 | 武汉大学 | End-to-end building regular outline automatic extraction method based on concentric ring convolution |
| CN116245901A (en) * | 2023-02-03 | 2023-06-09 | 武汉大学 | A method and system for extracting regular vector contours of buildings based on edge learning |
| CN116682021A (en) * | 2023-05-22 | 2023-09-01 | 中国地质大学（武汉） | A Method for Extracting Building Vector Outline Data from High Resolution Remote Sensing Image |
| CN116704518A (en) * | 2022-03-30 | 2023-09-05 | 中国电信股份有限公司 | A text recognition method and device, electronic equipment, and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU2017221222B2 (en) * | 2016-02-15 | 2022-04-21 | Pictometry International Corp. | Automated system and methodology for feature extraction |
-
2023
- 2023-12-01 CN CN202311629268.7A patent/CN117333677B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107328371A (en) * | 2017-05-22 | 2017-11-07 | 四川大学 | Sub-pix contours extract based on Gaussian and the optimization using Softmax recurrence in the case where metal plate detects scene |
| CN113658224A (en) * | 2021-08-18 | 2021-11-16 | 中国人民解放军陆军炮兵防空兵学院 | Target contour tracking method and system based on correlated filtering and Deep Snake |
| CN114820668A (en) * | 2022-03-09 | 2022-07-29 | 武汉大学 | End-to-end building regular outline automatic extraction method based on concentric ring convolution |
| CN116704518A (en) * | 2022-03-30 | 2023-09-05 | 中国电信股份有限公司 | A text recognition method and device, electronic equipment, and storage medium |
| CN116245901A (en) * | 2023-02-03 | 2023-06-09 | 武汉大学 | A method and system for extracting regular vector contours of buildings based on edge learning |
| CN116682021A (en) * | 2023-05-22 | 2023-09-01 | 中国地质大学（武汉） | A Method for Extracting Building Vector Outline Data from High Resolution Remote Sensing Image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117333677A (en) | 2024-01-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zheng et al. | Weakly supervised contrastive learning | |
| Elich et al. | 3d bird’s-eye-view instance segmentation | |
| Liu et al. | 3D Point cloud analysis | |
| CN110889318B (en) | Lane detection method and device using CNN | |
| CN113129311B (en) | A label-optimized point cloud instance segmentation method | |
| CN113570713B (en) | A semantic map construction method and device for dynamic environments | |
| CN114693923B (en) | A context- and attention-based 3D point cloud semantic segmentation method | |
| Ning et al. | Various realization methods of machine-part classification based on deep learning | |
| Lin et al. | Efficient training of semantic point cloud segmentation via active learning | |
| CN116310128A (en) | Monocular Multi-Object SLAM Method Based on Instance Segmentation and 3D Reconstruction in Dynamic Environment | |
| CN114972947A (en) | Depth scene text detection method and device based on fuzzy semantic modeling | |
| CN117523306A (en) | Laser point cloud location recognition method based on semantic graph and semantic scanning context | |
| CN115409896A (en) | Pose prediction method, pose prediction device, electronic device and medium | |
| CN116796798A (en) | A method and system for building image data recognition models | |
| WO2023206935A1 (en) | Person re-identification method, system and device, and computer-readable storage medium | |
| US20220318637A1 (en) | Machine learning techniques for automating tasks based on boundary representations of 3d cad objects | |
| Dalara et al. | Entity recognition in Indian sculpture using CLAHE and machine learning | |
| CN112560954A (en) | Geometric model non-rigid matching method and device using local and global information | |
| Han et al. | Novel methods for noisy 3D point cloud based object recognition | |
| CN117333677B (en) | A method for extracting building vector outlines | |
| CN112652059B (en) | Improved target detection and 3D reconstruction method based on Mesh R-CNN model | |
| CN116858233A (en) | Path generation method, path generation device, server and storage medium | |
| US20220318466A1 (en) | Techniques for generating uv-net representations of 3d cad objects for machine learning models | |
| CN110533781B (en) | An automatic labeling method for multi-category 3D model parts | |
| CN114677508A (en) | A point cloud instance semantic segmentation method based on dynamic filtering and point-by-point correlation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |