CN117649600A - Multi-category road automatic extraction method and system combining direction and semantic features - Google Patents

Abstract

The invention discloses a multi-category road automatic extraction method and system that combines direction and semantic features. First, remote sensing image data is obtained and data enhancement operations are performed; then the enhanced remote sensing image data is input into a multi-category road extraction network to output accurate The multi-category road extraction result map; the multi-category road extraction network includes a dense feature sharing encoder, a direction-guided feature stacking module, a semantic feature enhancement decoder and a directional feature decoder. The invention can significantly alleviate the situation of road extraction fragmentation and category confusion, and can be effectively applied to large-scale, multi-category road extraction tasks, thereby significantly improving the update efficiency of various road networks, and can extract railways and highways from remote sensing images with high quality. , paths and bridges.

Description

Multi-category road automatic extraction method and system combining direction and semantic features

Technical field

The invention relates to the technical fields of remote sensing image information processing and machine learning, and to a multi-category road automatic extraction method and system. Specifically, it relates to a multi-category road automatic extraction method and system that combines direction and semantic features, and can be applied to extracting data from high space. Extract and update multiple types of road network information in real time from high-resolution remote sensing images.

Background technique

Road information extraction and update is an important task in the field of surveying and mapping. It is of extremely important significance to smart city construction, unmanned autonomous driving, urban construction planning and other fields. Traditional road data is mainly realized through manual visual identification and labeling, which is time-consuming, low-efficiency, and has large subjective errors. It is far from meeting the real-time update requirements of road network information in today's society. With the development of high-resolution earth observation technology and machine learning, extracting road information from high-spatial-resolution remote sensing images can greatly reduce labor costs and achieve higher extraction accuracy.

Existing road extraction methods can generally be divided into methods based on expert knowledge and methods based on deep learning. Methods based on expert knowledge mainly extract road information by using texture information, morphological features, etc. This type of method can handle remote sensing images with simple backgrounds and clear road networks, but has great limitations in handling road extraction tasks in complex scenes; Methods based on deep learning have been a hot research topic in recent years. They can automatically obtain surface texture features and deep abstract features of images through deep neural networks, often with high extraction accuracy.

With the deepening of research, the above methods have certain limitations in practical applications: first, road extraction is often regarded as a pixel-level classification task, and detailed shapes are easily lost in common encoder structures, making it difficult to characterize connectivity. Not good; in addition, existing research only roughly divides the background and roads in the image into road extraction. There is no research that can handle the real-time extraction of multiple different types of roads at the same time. Existing methods are unable to handle multi-type road extraction tasks. At this time, there is a serious category confusion problem. Therefore, how to improve the morphological mining and semantic understanding capabilities of existing models when extracting road targets is still a difficult problem facing many challenges.

Contents of the invention

In order to overcome the shortcomings of the existing technology, the present invention proposes a multi-category road automatic extraction method and system that combines direction and semantic features to realize the simultaneous extraction of road network information such as railways, highways, paths and bridges in complex scenarios, and ensure the extraction results Class accuracy and road connectivity.

The technical solution adopted by the method of the present invention is: a multi-category road automatic extraction method that combines direction and semantic features, including the following steps:

Step 1: Obtain remote sensing image data and perform data enhancement operations;

Step 2: Input the enhanced remote sensing image data into the multi-category road extraction network and output an accurate multi-category road extraction result map;

The multi-category road extraction network includes a dense feature sharing encoder, a direction-guided feature stacking module, a semantic feature enhancement decoder and a directional feature decoder;

The dense feature sharing encoder enables users to obtain multi-scale refined road feature maps; the direction-guided feature stacking module is used to introduce road direction features and fully integrate them with road semantic features to obtain directional information fusion The feature map; the feature map after fusion of direction information is used for dual-branch decoding using the semantic feature enhancement decoder and the direction feature decoder respectively.

Preferably, in step 1, the data enhancement operations include random flipping, random rotation, random erasing and other operations to obtain enhanced data.

Preferably, in step 2, the dense feature sharing encoder is composed of 1 initialization convolution layer, 2 dense connection blocks and 1 max pooling layer; the dense connection block is composed of several intermediate layers, and the The intermediate layer consists of a normalization layer, a nonlinear activation layer, and a convolution layer. The input of each intermediate layer comes from the output of all previous intermediate layers. The relationship with the previous feature map is obtained by Equation (1):

X _N =H _N ([X ₀ ,X ₁ ,…,X _N-1 ]) (1)

In the _{formula ,} [X _{0 ,}

Preferably, in step 2, the direction-guided feature stacking module is constructed from two dual-branch attention modules and a fusion module;

The dual-branch attention module includes two network branches with the same structure, namely a semantic branch and a direction branch, which are used to learn road semantic information and direction information respectively; the two branches share the downsampling part of the entire module and the stride connection part, with a separate network propagation path in the upsampling part;

The downsampling part includes three groups of max pooling layers and residual blocks set in series; the max pooling layer of each group uses a 2×2 kernel to reduce the redundant information of the feature map, and then uses the residual block to extract different Road information at scale;

The input of the upsampling part is the high-dimensional road feature information finally obtained by the downsampling part, and then a series of residual modules and nearest neighbor interpolation upsampling operation feature maps are used to gradually restore the road features to the original size;

The stride connection part includes three sets of parallel coordinate attention modules and residual blocks; the input of the coordinate attention module is a feature map with a channel number of C, a width of W, and a height of H, with length and height respectively. Average pooling is performed in two wide dimensions, and then the channel splicing operation is performed. The spliced feature map is passed through a convolution layer, a batch normalization layer and a sigmoid layer in order to achieve feature dimensionality reduction; then the feature map is re-separated. , the two feature maps with embedded direction-specific information are encoded into attention maps in two directions through a convolution layer and a sigmoid layer respectively. Each attention map captures the long-range dependence of the feature map along the horizontal or vertical direction. relationship; then the attention maps in both directions are applied to the input feature map through multiplication to emphasize the spatial feature representation of the road area; finally, through a residual block and a convolution layer, the feature map is finally restored to the original resolution Rate;

The fusion module includes an intermediate supervision part and a branch fusion part; after the feature maps of the semantic branch and the direction branch are restored to the original resolution in the upsampling part, a rough road category prediction is obtained through a classification convolution layer in the intermediate supervision part Map or road direction prediction map, directly calculate the loss value with the ground truth map of the corresponding scale to achieve intermediate supervision; the branch fusion part, based on the semantic branch and direction branch output, uses a convolution layer to restore the rough road category prediction map to Initial channel number, and preliminary fusion of this feature with the output feature through the convolutional layer; in the first dual-branch attention module, the direction feature will be fused with the semantic branch feature, ultimately realizing information flow sharing between tasks.

Preferably, in step 2, the semantic feature enhancement decoder includes a deconvolution-based decoder, a deep supervision module and a category attention module;

The deconvolution-based decoder gradually restores the feature map to the input size through a series of deconvolution and transition convolution;

The category attention module first passes the input feature map through a convolution layer with a convolution kernel of 1×1, and adjusts the number of channels to C'; then the feature map is subjected to matrix transposition and deformation operations to obtain a size of HW ×C' two-dimensional feature map; then perform matrix deformation on the input rough prediction map to obtain a category attention matrix P of size N×HW, and perform matrix multiplication of the category attention matrix P and the above two-dimensional feature map, Obtain the class center of N×C'; then transpose the class center into a two-dimensional matrix of C'×N and multiply it with the category attention matrix of N×HW. After matrix deformation, the size is C'×H×W. The feature map; finally, the feature map is passed through another convolution layer with a convolution kernel of 1×1, and the final output is a category optimization feature with a size of C×H×W;

The category-optimized features that pass through the category attention module will be fused with the features in the deconvolution-based decoder, and finally the refined road category extraction results will be obtained through the classification layer.

Preferably, in step 2, the direction feature decoder is a deconvolution-based decoder, which gradually restores the feature map to the input size through a series of deconvolution and transitional convolution.

Preferably, the multi-category road extraction network is a well-trained network;

The training process includes the following sub-steps:

Step 2.1: Create a multi-category road data set;

The high-resolution remote sensing image data containing roads are randomly divided into training sets, verification sets, and test sets according to preset proportions. At the same time, the categories of road pixels in the corresponding areas are manually calibrated. The categories are railways, highways, roads, and bridges, and the road category labels are obtained. ; Then use the road direction calculation method and use the binarized road category labels to generate road direction labels; finally, perform data enhancement operations on the remote sensing images, road category labels, and road direction labels in the training set data set to complete the creation of the data set;

Step 2.2: Input the training set data into the multi-category road extraction network and perform network training;

During the training process, parameter update optimization is achieved by adjusting the learning rate using the SGD optimizer; given the road category labels and road direction labels in the training set, the joint loss function is used to train the network; after the training is completed, the optimal Network parameters.

Preferably, in step 2.1, the binarized road category label is used to generate the road direction label. The specific process is as follows:

(1) Use the skeleton refinement algorithm to extract the road centerline and its key points, and arrange the key point coordinates from top to bottom and from left to right;

(2) Connect the key points (p ₁ , p ₂ ,..., p _i ) in sequence, and calculate the unit direction vector formed by connecting adjacent key points Then convert the direction vector to the polar coordinate system to calculate its corresponding road direction angle value;

(3) Afterwards, a certain pixel point M is connected to the nearest key point p _i to form a connection vector, and the vector in the unit direction is calculated. If the projection length is within the automatically calculated road directional width threshold, the pixel of the point pixel is assigned the same direction value as the road unit direction vector;

(4) Traverse all pixels in the image and finally obtain the corresponding road direction label.

As a preference, in step 2.2, the joint loss function consists of two parts: road direction loss and road semantic loss, where road direction loss Cross-entropy loss is used to calculate the loss value; for road semantic loss/> Calculated using SoftIOU loss based on class balance;

where s represents the scale, and its value is {(H,W),(H/2,W/2),(H/4,W/4)}, H is the height of the feature map, and W is the width of the feature map; and/> They are the predicted road category results and the real road category results respectively;/> is the number of valid samples of class i, E _all is the total number of all samples;

Finally, the total loss function L _total is defined as follows:

The technical solution adopted by the system of the present invention is: a multi-category road automatic extraction system that combines direction and semantic features, including:

one or more processors;

A storage device configured to store one or more programs, which when executed by the one or more processors, causes the one or more processors to implement the joint direction and semantic features A multi-category road automatic extraction method.

The advantages of the present invention compared with the prior art are:

① Unlike other road network extraction methods that can only extract a single type of road, this invention can simultaneously extract multiple types of road networks including railways, highways, paths, bridges, etc. with high precision. It has strong fine-grained classification capabilities and can be adapted to cities, Extraction of various types of roads in various complex scenes such as rural areas, water bodies, mountainous areas, etc.

② This invention adopts a multi-task joint learning model and introduces prior information of road direction. Compared with other semantic segmentation methods, it effectively improves the connectivity of road extraction results.

③ This invention constructs a multi-category road data set, which can provide reliable training data for other road extraction methods.

④ This invention uses a joint loss function, including a direction loss function and a class-balanced SoftIoU loss function, which can effectively alleviate the problem of imbalance in the number of pixels of different road categories in the training data in real scenarios, and further improve the network model. road classification capabilities.

Description of drawings

Examples and specific implementations are used below to further illustrate the technical solutions herein. In addition, in the process of explaining the technical solution, some drawings are also used. For those skilled in the art, other drawings and the intention of the present invention can also be obtained based on these drawings without exerting creative efforts.

Figure 1 is an overall structural diagram of a network model according to an embodiment of the present invention;

Figure 2 is a structural diagram of a direction guidance stack module according to an embodiment of the present invention;

Figure 3 is a semantic enhancement branch structure diagram according to the embodiment of the present invention;

Figure 4 is a network training flow chart according to the embodiment of the present invention;

Figure 5 is a sample image of remote sensing images and road category labels in the data set according to the embodiment of the present invention;

Figure 6 is a flow chart for generating road direction labels in the data set according to the embodiment of the present invention;

Figure 7 is a visual comparison result diagram of road extraction tasks in remote sensing images using the method of the present invention and other methods.

Detailed ways

In order to facilitate those of ordinary skill in the art to understand and implement the present invention, the present invention will be described in further detail below in conjunction with the drawings and examples. It should be understood that the implementation examples described here are only used to illustrate and explain the present invention and are not intended to limit it. this invention.

This embodiment provides a multi-category road automatic extraction method that combines direction and semantic features, including the following steps:

Step 1: Obtain remote sensing image data and perform data enhancement operations;

In one implementation, the data enhancement operations include random flipping, random rotation, random erasing and other operations to obtain enhanced data.

Step 2: Input the enhanced remote sensing image data into the multi-category road extraction network and output an accurate multi-category road extraction result map;

In one implementation, please see Figure 1. The multi-category road extraction network includes a dense feature sharing encoder, a direction-guided feature stacking module, a semantic feature enhancement decoder and a directional feature decoder;

The dense feature sharing encoder enables users to obtain multi-scale refined road feature maps; the direction-guided feature stacking module is used to introduce road direction features and fully integrate them with road semantic features to obtain directional information fusion The feature map; the feature map after fusion of direction information is used for dual-branch decoding using the semantic feature enhancement decoder and the direction feature decoder respectively.

In one implementation, please see Figure 1. The dense feature sharing encoder consists of 1 initialization convolution layer, 2 dense connection blocks and 1 max pooling layer; the dense connection block consists of several intermediate The intermediate layer is composed of a normalization layer, a nonlinear activation layer, and a convolution layer. The input of each intermediate layer comes from the output of all previous intermediate layers. The relationship with the previous feature map is obtained by Equation (1):

X _N =H _N ([X ₀ ,X ₁ ,…,X _N-1 ]) (1)

In the _{formula ,} [X _{0 ,} correction—convolutional layer). This architecture can effectively increase the information interaction between layers, improve the information flow and gradient flow of feature extraction, and realize the full mining of fine road features.

In one implementation, please see Figure 2. The direction-guided feature stacking module is constructed from two dual-branch attention modules and a fusion module;

The dual-branch attention module includes two network branches with the same structure, namely a semantic branch and a direction branch, which are used to learn road semantic information and direction information respectively. The two branches share the downsampling part and the stride connection part of the entire module, and have separate network propagation paths in the upsampling part.

The downsampling part includes three sets of max pooling layers and residual blocks set in series. The maximum pooling layer of each group uses a 2×2 kernel to reduce the redundant information of the feature map, and then uses the residual block to extract road information at different scales.

The input of the upsampling part is the high-dimensional road feature information finally obtained by the downsampling part, and then a series of residual modules and nearest neighbor interpolation upsampling operation feature maps are used to gradually restore the road features to the original size.

The stride connection part is used to retain the detailed spatial information lost in the original features during the downsampling process, and includes three sets of parallel coordinate attention modules and residual blocks. The coordinate attention module is used to further highlight important spatial features. Its input is a feature map with a channel number of C, a width of W, and a height of H. It performs average pooling in the two dimensions of length and width. The purpose of this operation is In order to avoid the loss of position information caused by traditional two-dimensional global pooling. Then the channel splicing operation is performed, and the spliced feature map is passed through a convolution layer, a batch normalization layer and a sigmoid layer in sequence to achieve feature dimensionality reduction. The feature maps are then re-separated, and the two feature maps with embedded direction-specific information are encoded into attention maps in two directions through a convolutional layer and a sigmoid layer respectively. Each attention map captures features along the horizontal or vertical direction. Graph long-range dependencies. The attention maps in both directions are then applied to the input feature map via multiplication to emphasize the spatial feature representation of the attention road region. The multi-scale features passed through each residual module are added to the same size feature map corresponding to each task branch in the upsampling part using a stride connection, and finally through a residual block and a convolution layer, the feature map is finally restored to the original resolution.

The fusion module includes an intermediate supervision part and a branch fusion part. The purpose of intermediate supervision is to enable the intermediate layer of the road extraction network to directly obtain real semantic or direction information from the ground truth map during training, so as to train the network more stably. After the feature maps of the semantic branch and the direction branch are restored to the original resolution in the upsampling part, a rough road category prediction map or road direction prediction map is obtained through a classification convolution layer in the intermediate supervision part, which is directly compared with the true value of the corresponding scale. Graph calculates the loss value and implements intermediate supervision. In the branch fusion part, taking the semantic branch as an example, a convolution layer is used to restore the rough road category prediction map to the initial number of channels, and the features are initially fused with the output features of the convolution layer. The features of the direction branch are The operation is the same. In the first dual-branch attention module, the direction feature will be integrated with the semantic branch feature, ultimately realizing information flow sharing between tasks and helping to improve road connectivity.

In one implementation, please see Figure 3. The semantic feature enhancement decoder includes a deconvolution-based decoder, a deep supervision module and a category attention module; the deconvolution-based decoder , through a series of deconvolution and transitional convolution, the feature map is gradually restored to the input size; the deep supervision module is carried out using the feature maps of different scales in the deconvolution-based decoder and the true values of the category labels of the corresponding scales Supervise and generate rough classification maps;

The category attention module is used to enhance the subdivision ability of road categories. By checking the consistency of each pixel with the calculated class center, the network can understand the overall presentation of each class from a global perspective. This module first passes the input feature map through a convolution layer with a convolution kernel of 1×1, and adjusts the number of channels to C’. The feature map is then subjected to matrix transposition and deformation operations to obtain a two-dimensional feature map with a size of HW×C’. Perform matrix deformation on the input rough prediction map to obtain a category attention matrix P of size N×HW. Matrix multiply the category attention matrix P and the above two-dimensional feature map to obtain the class center of N×C’. The class center is then transposed into a C’×N two-dimensional matrix and multiplied by the N×HW category attention matrix. After matrix deformation, a feature map of size C’×H×W is obtained. Finally, the feature map is passed through another convolution layer with a convolution kernel of 1×1, and the final output is a category optimized feature with a size of C×H×W. The category-optimized features that pass through the category attention module will be fused with the features in the deconvolution-based decoder, and finally the refined road category extraction results will be obtained through the classification layer.

In one implementation, the directional feature decoder is a deconvolution-based decoder, which gradually restores the feature map to the input size through a series of deconvolution and transition convolution.

In one implementation, please see Figure 4. The multi-category road extraction network is a trained network; the training process includes the following sub-steps:

Step 2.1: Create data set;

Obtain high-resolution remote sensing image data including roads. After pre-processing such as cropping, the data is divided according to the ratio of 80% of the training set, 10% of the verification set and 10% of the test set. The road pixel categories of each image block are then manually divided into four categories: railways, highways, paths, and bridges, and the corresponding road category labels are obtained as shown in Figure 5. Then the binarized road category label, that is, the road mask, is used to generate the road direction label. The calculation steps are shown in Figure 6: ① Use the skeleton refinement algorithm to obtain the road centerline, its inflection points, end points and other key points. ② Connect the key points (p ₁ , p ₂ ,..., p _i ) in sequence, and calculate the unit direction vector formed by connecting adjacent key points Then the direction vector is converted to the polar coordinate system to calculate its corresponding road direction angle value ∠θ; ③ After that, a certain pixel point M is connected to the nearest key point P ₁ to form a connection vector Use formula (2) to calculate the projection length on the unit road direction vector. If the projection length is within the given road directional width threshold λ _Road , then the pixel of the point pixel is assigned the same direction value O as the road unit direction vector. _r , and for all other pixels, assign the non-road direction angle O _b . ④ Then traverse all pixels in the image to complete the above operation, and finally obtain the corresponding road direction label.

Finally, synchronized data enhancement operations are performed on the remote sensing images, road category labels, and road direction labels in the training set data set to complete the creation of the data set.

Step 2.2: Train the network model and save the optimal parameters;

Input the remote sensing road image training data in step 1 into the multi-category road extraction network model, and then train the network model. During the process, the joint loss function is used for loss calculation, and the SGD optimizer is used to implement parameter update and optimization. During training, the validation set is used to calculate the average intersection and union ratio accuracy every two epochs. If the accuracy is better, the current network parameters are saved. The training process continues until the network converges, and the optimal network model parameters can be obtained.

The joint loss function consists of two parts: road direction loss and road semantic loss, where road direction loss Cross entropy loss is used to calculate the loss value. For road semantic loss/> Calculation using SoftIOU loss based on category balance, as shown in Equation (3), can effectively solve the problem of imbalance in the number of categories in the data set:

where s represents the scale, and its value is {(H,W),(H/2,W/2),(H/4,W/4)}, H is the height of the feature map, and W is the width of the feature map. and/> They are the predicted road category results and the real road category results respectively. /> is the number of valid samples of class i, and E _all is the total number of all samples.

Finally, the total loss function L _total is defined as follows:

In this embodiment, for a given remote sensing image containing a road, the network simultaneously extracts the road area and the corresponding direction angle in an end-to-end manner. The shared encoder uses densely connected convolution blocks as the main feature extractor, and strengthens feature reuse through dense connections, thereby facilitating the mining and extraction of deep road features; the direction guidance stacking module uses the dual-branch attention module as the main component , which is similar to a mini codec structure. While enhancing the multi-scale capture of spatial context information, it uses the coordinate attention module to calculate the horizontal and vertical spatial attention weights of input features at each scale, and then inputs them into the residual Difference blocks generate refined feature maps to achieve road feature correction and rough prediction. The fusion layer in the two dual-branch attention modules can realize feature interaction between direction and semantics; the semantic enhancement decoding branch uses deconvolution in addition to , in addition to restoring the image size of the output result of the semantic branch to the original image size, it also uses the deep supervision module and the category attention module.

Among them, the deep supervision module consists of a transposed convolution layer, a convolution layer and a classification layer. It uses feature maps of different scales in the original decoder to generate the fusion features required for rough prediction and category attention modules. In addition, the deep supervision method can Let the intermediate layers at different scales learn road features directly from the category ground truth labels, mitigating the adverse effects of unstable gradients. The category attention module can effectively increase the feature distinction of different road classes, and by calculating the category center, the network can optimize features from a global perspective of the category. In actual calculations, a feature map with a channel number of C, a height of H, and a width of W is given. Finally, a convolutional layer with a convolution kernel of 1×1 is applied to reduce the number of channels to C'. Then through transposition and deformation, with the coarse prediction map/> Characteristics after deformation/> Multiply together to get the class center. Inspired by the attention mechanism, this embodiment treats the deformed rough prediction map as an attention map, and performs matrix multiplication with the transposed category center. The category-specific attention features of each pixel can be obtained by calculation/> Calculated as follows:

The calculated category attention features are further adjusted to a feature size of C×H×W through a convolutional layer with a convolution kernel of 1×1, and then fused with the features of the semantic decoder. Finally, fine road segmentation results can be obtained through classification.

In order to further verify the effectiveness and feasibility of this method, the following experiments were conducted in this example.

This experiment uses Pytorch1.10 as the deep learning network construction framework, and the development environment is Python3.6. The data set uses a self-built multi-type road data set. In order to fully collect all types of road data, the remote sensing images of the data set come from Google Earth and three public road data sets: Deepglobe data set, RoadNet data set, CHN6-CUG data set. For the evaluation of extraction results, the commonly used indicators of semantic segmentation accuracy are intersection-union ratio, average intersection-union ratio, and frequency-weighted intersection-union ratio for accuracy evaluation.

Table 1 shows the quantitative comparison results between the method of the present invention and other methods in the road extraction task of remote sensing images. Figure 7 shows the visual comparison results.

Table 1 Comparison of road extraction accuracy between the method of the present invention and other methods

Judging from the results shown in Table 1 and Figure 7, the method described in the present invention is effective in extracting multiple types of roads. Not only does the overall accuracy of extraction reach the optimal level on the four types of road types, but it also improves road connectivity and classification. Compared with other methods, it has significant advantages in accuracy.

It should be understood that the above description of the preferred embodiments is relatively detailed and cannot therefore be considered to limit the scope of patent protection of the present invention. A person of ordinary skill in the art, under the inspiration of the present invention, may not deviate from the claims of the present invention. Within the scope of protection, substitutions or modifications can be made, all of which fall within the scope of protection of the present invention. The scope of protection claimed by the present invention shall be determined by the appended claims.

Claims (10)

1. A multi-category road automatic extraction method that combines direction and semantic features, which is characterized by including the following steps: Step 1: Obtain remote sensing image data and perform data enhancement operations; Step 2: Input the enhanced remote sensing image data into the multi-category road extraction network and output an accurate multi-category road extraction result map; The multi-category road extraction network includes a dense feature sharing encoder, a direction-guided feature stacking module, a semantic feature enhancement decoder and a directional feature decoder; The dense feature sharing encoder enables users to obtain multi-scale refined road feature maps; the direction-guided feature stacking module is used to introduce road direction features and fully integrate them with road semantic features to obtain directional information fusion The feature map; the feature map after fusion of direction information is used for dual-branch decoding using the semantic feature enhancement decoder and the direction feature decoder respectively. 2. The multi-category road automatic extraction method combining direction and semantic features according to claim 1, characterized in that: in step 1, the data enhancement operations include random flipping, random rotation, random erasing and other operations, which are enhanced. the subsequent data. 3. The multi-category road automatic extraction method of combining direction and semantic features according to claim 1, characterized in that: in step 2, the dense feature sharing encoder consists of 1 initialization convolution layer and 2 dense connections. block and a maximum pooling layer; the densely connected block is composed of several intermediate layers, and the intermediate layer is composed of a normalization layer, a nonlinear activation layer, and a convolution layer. The input of each intermediate layer comes from all the previous The relationship between the output of the middle layer and the previous feature map is obtained by equation (1): X _N =H _N ([X ₀ ,X ₁ ,…,X _N-1 ]) (1) In the _{formula ,} [X _{0 ,} 4. The multi-category road automatic extraction method combining direction and semantic features according to claim 1, characterized in that: in step 2, the direction-guided feature stacking module consists of two dual-branch attention modules and a fusion module. Built from modules; The dual-branch attention module includes two network branches with the same structure, namely a semantic branch and a direction branch, which are used to learn road semantic information and direction information respectively; the two branches share the downsampling part of the entire module and the stride connection part, with a separate network propagation path in the upsampling part; The downsampling part includes three groups of max pooling layers and residual blocks set in series; the max pooling layer of each group uses a 2×2 kernel to reduce the redundant information of the feature map, and then uses the residual block to extract different Road information at scale; The input of the upsampling part is the high-dimensional road feature information finally obtained by the downsampling part, and then a series of residual modules and nearest neighbor interpolation upsampling operation feature maps are used to gradually restore the road features to the original size; The stride connection part includes three sets of parallel coordinate attention modules and residual blocks; the input of the coordinate attention module is a feature map with a channel number of C, a width of W, and a height of H, with length and height respectively. Average pooling is performed in two wide dimensions, and then the channel splicing operation is performed. The spliced feature map is passed through a convolution layer, a batch normalization layer and a sigmoid layer in order to achieve feature dimensionality reduction; then the feature map is re-separated. , the two feature maps with embedded direction-specific information are encoded into attention maps in two directions through a convolution layer and a sigmoid layer respectively. Each attention map captures the long-range dependence of the feature map along the horizontal or vertical direction. relationship; then the attention maps in both directions are applied to the input feature map through multiplication to emphasize the spatial feature representation of the road area; finally, through a residual block and a convolution layer, the feature map is finally restored to the original resolution Rate; The fusion module includes an intermediate supervision part and a branch fusion part; after the feature maps of the semantic branch and the direction branch are restored to the original resolution in the upsampling part, a rough road category prediction is obtained through a classification convolution layer in the intermediate supervision part Map or road direction prediction map, the loss value is calculated directly with the ground truth map of the corresponding scale to achieve intermediate supervision; the branch fusion part, based on the semantic branch and direction branch output, uses a convolution layer to restore the rough road category prediction map to Initial channel number, and preliminary fusion of this feature with the output feature through the convolutional layer; in the first dual-branch attention module, the direction feature will be fused with the semantic branch feature, ultimately realizing information flow sharing between tasks. 5. The multi-category road automatic extraction method combining direction and semantic features according to claim 1, characterized in that: in step 2, the semantic feature enhanced decoder includes a deconvolution-based decoder, a deep layer Supervision module and a category attention module; The deconvolution-based decoder gradually restores the feature map to the input size through a series of deconvolution and transition convolution; The deep supervision module uses the feature maps of different scales in the deconvolution-based decoder and the true values of the category labels of the corresponding scales to supervise and generate a rough classification map; The category attention module first passes the input feature map through a convolution layer with a convolution kernel of 1×1, and adjusts the number of channels to C'; then the feature map is subjected to matrix transposition and deformation operations to obtain a size of HW ×C' two-dimensional feature map; then perform matrix deformation on the input rough prediction map to obtain a category attention matrix P of size N×HW, and perform matrix multiplication of the category attention matrix P and the above two-dimensional feature map, Obtain the class center of N×C'; then transpose the class center into a two-dimensional matrix of C'×N and multiply it with the category attention matrix of N×HW. After matrix deformation, the size is C'×H×W. The feature map; finally, the feature map is passed through another convolution layer with a convolution kernel of 1×1, and the final output is a category optimization feature with a size of C×H×W; The category-optimized features that pass through the category attention module will be fused with the features in the deconvolution-based decoder, and finally the refined road category extraction results will be obtained through the classification layer. 6. The multi-category road automatic extraction method that combines direction and semantic features according to claim 1, characterized in that: in step 2, the direction feature decoder is a deconvolution-based decoder, which uses a series of inverse Convolution and transitional convolution gradually restore the feature map to the input size. 7. The multi-category road automatic extraction method combining direction and semantic features according to any one of claims 1-6, characterized in that: the multi-category road extraction network is a trained network; The training process includes the following sub-steps: Step 2.1: Create a multi-category road data set; The high-resolution remote sensing image data containing roads are randomly divided into training sets, verification sets, and test sets according to preset proportions. At the same time, the categories of road pixels in the corresponding areas are manually calibrated. The categories are railways, highways, roads, and bridges, and the road category labels are obtained. ; Then use the road direction calculation method and use the binarized road category labels to generate road direction labels; finally, perform data enhancement operations on the remote sensing images, road category labels, and road direction labels in the training set data set to complete the creation of the data set; Step 2.2: Input the training set data into the multi-category road extraction network and perform network training; During the training process, parameter update optimization is achieved by adjusting the learning rate using the SGD optimizer; given the road category labels and road direction labels in the training set, the joint loss function is used to train the network; after the training is completed, the optimal Network parameters. 8. The multi-category road automatic extraction method combining direction and semantic features according to claim 7, characterized in that: in step 2.1, the binarized road category labels are used to generate road direction labels, and the specific process is as follows: (1) Use the skeleton refinement algorithm to extract the road centerline and its key points, and arrange the key point coordinates from top to bottom and from left to right; (2) Connect the key points (p ₁ , p ₂ ,..., p _i ) in sequence, and calculate the unit direction vector formed by connecting adjacent key points Then convert the direction vector to the polar coordinate system to calculate its corresponding road direction angle value; (3) Afterwards, a certain pixel point M is connected to the nearest key point p _i to form a connection vector, and the vector in the unit direction is calculated. If the projection length is within the automatically calculated road directional width threshold, the pixel of the point pixel is assigned the same direction value as the road unit direction vector; (4) Traverse all pixels in the image and finally obtain the corresponding road direction label. 9. The multi-category road automatic extraction method of combining direction and semantic features according to claim 7, characterized in that: in step 2.2, the joint loss function consists of two parts: road direction loss and road semantic loss, wherein road direction loss Cross-entropy loss is used to calculate the loss value; for road semantic loss/> Calculated using SoftIOU loss based on class balance; where s represents the scale, its value is {(H,W),(H/2,W/2),(H/4,W/4)}, H is the height of the feature map, and W is the width of the feature map; and/> They are the predicted road category results and the real road category results respectively;/> is the number of valid samples of class i, E _all is the total number of all samples; Finally, the total loss function L _total is defined as follows: 10. A multi-category road automatic extraction system that combines direction and semantic features, which is characterized by: one or more processors; A storage device configured to store one or more programs, which when executed by the one or more processors, causes the one or more processors to implement any of claims 1 to 9 An automatic multi-category road extraction method that combines directional and semantic features.