CN114037891A - Method and device for building extraction from high-resolution remote sensing images based on U-shaped attention control network - Google Patents

Abstract

The invention discloses a high-resolution remote sensing image building extraction method based on a U-shaped attention control network, comprising: inputting remote sensing images to an encoder; Output to the converter; attention valves are strung in the skip connections between the convolutional blocks of the encoder and decoder; the converter extracts the abstract feature map and outputs it to the decoder; The step-by-step upsampling process generates a decoder feature map and outputs it to the segmentation unit; the segmentation unit adjusts the number of channels of the input decoder feature map to obtain the building segmentation result in the remote sensing image. In addition, the invention also discloses a high-resolution remote sensing image building extraction device based on a U-shaped attention control network. The U-shaped attention control network of the present invention achieves an effective improvement in the extraction quality of buildings from high-scoring remote sensing images in terms of prediction performance and result accuracy.

Description

Method and device for building extraction from high-resolution remote sensing images based on U-shaped attention control network

technical field

The invention relates to the technical field of remote sensing image extraction, in particular to a high-resolution remote sensing image building extraction method and device based on a U-shaped attention control network.

Background technique

As a real estate resource in urban and rural areas, buildings are of great significance in regional construction planning, regional population estimation, economic development assessment, topographic map making and updating, etc. With the development of satellite sensor technology, the imaging quality of remote sensing images has been continuously improved, and researchers can obtain batches of high-resolution remote sensing images more quickly. However, with the increasing volume of available remote sensing images, how to automatically, accurately and effectively extract building information from images has gradually become a difficult problem.

The amount of data in early remote sensing images was small, and the extraction of buildings in images mainly depended on manual recognition, visual interpretation, and vector feature extraction. However, in recent years, remote sensing image data has gradually presented the characteristics of mass, multi-scale, diversification, and big data, and it is almost impossible to complete batch image building extraction work only by manually interpreting images. This is due to the high work cost and low efficiency, and it is impossible to measure the quality of data extraction with a unified standard; the second is that a large number of professional personnel are required for interpretation and extraction, and long-term visual inspection will cause problems such as visual impairment of the interpretation personnel, resulting in high work efficiency. instability. Therefore, the automatic and intelligent extraction of buildings from remote sensing images through computer technology has become one of the important contents in the field of remote sensing image extraction research.

In the prior art, there are two main types of mainstream methods for extracting buildings from remote sensing images: one is to manually design and extract target features using mathematical and morphological knowledge, and the other is to automatically extract target features based on deep learning. Among them, the main principle of the method based on artificial design features is to use the texture, geometry, spectrum and context of buildings in remote sensing images to manually filter, design and construct effective features through professional knowledge to complete the extraction of buildings. Lin and Nevatia first used edge detection algorithms to detect the roofs, walls and shadows of buildings, and then extracted buildings. Jinku et al. process the image by first constructing a knowledge rule algorithm, then perform morphological repair and edge detection, and finally use three digital methods to regularize the shape features of buildings to obtain accurate extraction results of building outlines. Fang Xin et al. constructed screening conditions through the spatial relationship between shadows and buildings, so as to filter out areas suspected of buildings, and then used the graph cut algorithm to accurately extract the outline of buildings. Zhang et al. exploited the shape and spectral features of buildings in images to extract buildings by clustering homogeneous pixels with similar shape and contour information.

However, the inventor found through research that although the previous methods for extracting buildings from high-resolution remote sensing images have made some progress, such methods based on artificially designed features generally suffer from low extraction accuracy and complex processing procedures. And the shortcomings of insufficient use of image features. In addition, these methods also require various rules to manually pre-define features, which are laborious to achieve building extraction from large-scale remote sensing imagery.

With the improvement of computer computing performance in recent years, the concept and theory of deep learning, especially Convolutional Neural Network (CNN), have developed rapidly, and have been used in fields such as natural image classification, object detection and semantic segmentation. Get a wide range of applications. At present, common CNN models mainly include AlexNet, VGGNet, GoogLeNet and ResNet, etc. Compared with the traditional methods of extracting target features based on manual design, the CNN model can automatically extract the features of the input image and has strong feature representation capabilities. It has been paid attention by more and more researchers, and has made important progress in the field of target segmentation and recognition of remote sensing images. For example, Lv et al. used SEEDS-CNN and scale validity analysis to achieve high-precision classification of small-scale objects in remote sensing images; Li et al. solved the problem that CNN only stacks square images of different proportions and ignores the background information of segmentation objects when extracting features. Therefore, the problem of affecting the classification accuracy, a dual CNN model based on standard objects was proposed for image classification; Chen et al. applied multi-scale CNN and scale parameter estimation to achieve high accuracy classification of land cover; Zhang et al. used The OCNN model is fused to fuse and summarize information such as spectral patterns, geometric features, and object-level contextual features to accurately identify target objects in high-resolution images and improve the accuracy of object classification. However, only the CNN model cannot directly generate accurate building outlines, and the post-processing steps after image segmentation cannot be ignored. Therefore, in recent years, scholars have begun to try to introduce the Fully Convolutional Neural Networks (FCN) model into the model. In the field of building extraction from remote sensing images, "end-to-end" building contours can be extracted. Xie Yuehui et al. combined the texture features expressed by the local binary pattern and the scale features extracted by the Gaussian pyramid to construct the training samples of the model, used the SegNet network for sample training, and completed the rough building extraction through the SoftMax classifier. Liu Yifan et al. used the deep residual network Res-Une to extract the pixel-level features of the building, set a threshold to filter the noise in the predicted probability map, and preserved the integrity of the building to a large extent through post-processing.

SUMMARY OF THE INVENTION

Based on this, in order to solve the technical problems in the existing technology, a method for extracting buildings from high-resolution remote sensing images based on U-shaped attention control network is proposed, including:

Input remote sensing images to the encoder; the encoder utilizes global context information and local context information to extract features of different grid dimension levels and generate encoder feature maps; the encoder includes four concatenated convolution blocks, which are respectively The first convolution block, the second convolution block, the third convolution block, and the fourth convolution block;

The encoder is connected to the converter; the encoder outputs the encoder feature map generated by the extraction to the converter; the converter includes a fifth convolution block and an attention valve in four skip connections;

The converter is connected to a decoder; attention valves are inline in skip connections between convolutional blocks of the encoder and decoder; the converter extracts an abstract feature map and outputs it to the decoder ;

The decoder includes four convolution blocks in cascade, which are the sixth convolution block, the seventh convolution block, the eighth convolution block, and the ninth convolution block; wherein the first convolution block is skipped and connected to The ninth convolution block, the second convolution block is skip connected to the eighth convolution block, the third convolution block is skip connected to the seventh convolution block, and the fourth convolution block is skip connected to the sixth convolution block;

The decoder is connected to the segmentation unit; the decoder performs a step-by-step upsampling process on the abstract feature map input by the converter, generates a decoder feature map with the same size as the input image, and outputs it to the segmentation unit ; The segmentation unit adjusts the number of channels of the input feature map of the decoder, and obtains the building segmentation result in the remote sensing image.

In one embodiment, each convolution block in the encoder includes a convolution layer, a max pooling layer, a normalization layer, and an activation function unit; wherein, the max pooling in the convolution block of the encoder The layer constructs a new feature map by extracting the maximum value of the local area in the original feature map, and prevents overfitting by reducing the number of parameters;

The fifth convolution block includes a convolution layer, a maximum pooling layer, and a normalization layer; the fifth convolution block abstracts the encoder feature map input by the encoder to the highest level, and superimposes the number of channels of the feature map. Reduce the size of the feature map to extract the abstract feature map with the highest dimension.

In an embodiment, the decoder performs a step-by-step up-sampling process on the abstract feature map input by the converter, which specifically includes: the sixth convolution block performs an up-sampling process on the abstract feature map input by the fifth convolution block. Sampling and output to the seventh convolution block; the seventh convolution block performs up-sampling processing on the abstract feature map input by the sixth convolution block and outputs it to the eighth convolution block; the eighth convolution block convolutes the seventh convolution block The abstract feature map of the block input is up-sampled and output to the ninth convolution block;

The segmentation unit includes a tenth convolution block; the tenth convolution block includes a convolution layer and an activation function unit; the convolution layer of the tenth convolution block performs 1×1 convolution processing, and the tenth convolution block has a The activation function unit is a Sigmoid function; the decoder feature map adjusts the number of model channels to the number of categories through 1×1 convolution processing of the tenth convolution block, and obtains the building segmentation result in the remote sensing image through the Sigmoid function.

In one embodiment, the attention valve includes an attention layer, a valve control layer, a first activation function unit, a linear transformation unit, a second activation function unit, and a resampler;

In the valve control layer, a valve control signal is input to the attention valve, and the valve control signal includes a control vector; wherein, the valve control signal comes from the previous convolution block of the current convolution block;

Multiply the control vector and the control coefficient to obtain the first product;

In the attention layer, the feature map is input to the attention valve; wherein, the feature map input to the attention valve comes from the convolution block that is skip-connected to the current convolution block;

The pixel vector of the feature map of the input attention valve is multiplied by the attention coefficient to obtain the second product;

The first product and the second product are added together, and the first activation function unit is used for activation after bias processing to obtain the first activation result; The second activation function unit is activated to obtain the second activation result, and the second activation result is input into the resampler for adjustment to obtain the attention coefficient α; the feature map input by the attention layer is multiplied by the attention coefficient α, and the output is equal to the attention coefficient α. The feature maps input to the force layer have the same size multiplication result.

In an embodiment, the attention valve uses an additional attention mechanism to obtain an attention coefficient; the calculation formula of the additional attention mechanism is as follows:

为sigmoid函数；in,

is the sigmoid function;

b_ψ∈R；其中，W_x为控制系数，W_g为注意力系数，ψ为线性变换，b_g为第一偏置参数，b_ψ为第二偏置系数；其中，F_l表示第l层中特征图的数量，F_int表示输入层；The parameter group _Θatt of the attention valve includes:

b _ψ ∈ R; where W _x is the control coefficient, W _g is the attention coefficient, ψ is the linear transformation, b _g is the first bias parameter, and b _ψ is the second bias coefficient; among them, F _l represents the first The number of feature maps in the layer, F _int represents the input layer;

Among them, the first activation function performed by the first activation function unit of the attention valve is shown in the following formula:

其中，i表示空间维度，c表示通道维度；

Among them, i represents the spatial dimension, and c represents the channel dimension;

Wherein, the linear transformation performed by the linear transformation unit is to change the number of channels of the input vector through 1×1×1 channel convolution processing;

Wherein, the second activation function σ ₂ executed by the second activation function unit of the attention valve is a sigmoid function;

其中，注意力系数α∈[0，1]。The multiplication result of the input feature map and the attention coefficient α at the pixel level is the output of the attention valve, as shown in the following formula:

Among them, the attention coefficient α∈[0, 1].

In addition, in order to solve the technical problems in the prior art, a high-resolution remote sensing image building extraction device based on U-shaped attention control network is proposed, which includes an encoder, a converter, a decoder and a segmentation unit; the An encoder is connected to the converter, the converter is connected to the decoder, and the decoder is connected to the dividing unit;

Input remote sensing images to the encoder; the encoder utilizes global context information and local context information to extract features of different grid dimension levels and generate encoder feature maps; the encoder includes four concatenated convolution blocks, are respectively the first convolution block, the second convolution block, the third convolution block and the fourth convolution block; the encoder outputs the encoder feature map generated by extraction to the converter;

The decoder includes four convolution blocks in cascade, which are the sixth convolution block, the seventh convolution block, the eighth convolution block, and the ninth convolution block; wherein the first convolution block is skipped and connected to The ninth convolution block, the second convolution block is skip connected to the eighth convolution block, the third convolution block is skip connected to the seventh convolution block, and the fourth convolution block is skip connected to the sixth convolution block;

The converter includes a fifth convolution block and an attention valve in four skip connections; the converter connects the feature maps corresponding to the encoder and the decoder; Attention valves are serialized into skip connections between convolutional blocks of the decoder; the converter extracts abstract feature maps and outputs them to the decoder;

The decoder performs a step-by-step upsampling process on the abstract feature map input by the converter, generates a decoder feature map with the same size as the input image, and outputs it to the segmentation unit; the segmentation unit adjusts all the input images. The number of channels of the decoder feature map is described to obtain the building segmentation results in the remote sensing image.

In one embodiment, each convolution block in the encoder includes a convolution layer, a max pooling layer, a normalization layer, and an activation function unit; wherein, the max pooling in the convolution block of the encoder The layer constructs a new feature map by extracting the maximum value of the local area in the original feature map, and prevents overfitting by reducing the number of parameters;

The fifth convolution block includes a convolution layer, a maximum pooling layer, and a normalization layer; the fifth convolution block abstracts the encoder feature map input by the encoder to the highest level, and superimposes the number of channels of the feature map. Reduce the size of the feature map to extract the abstract feature map with the highest dimension.

In an embodiment, the decoder performs a step-by-step up-sampling process on the abstract feature map input by the converter, which specifically includes: the sixth convolution block performs an up-sampling process on the abstract feature map input by the fifth convolution block. Sampling and output to the seventh convolution block; the seventh convolution block performs up-sampling processing on the abstract feature map input by the sixth convolution block and outputs it to the eighth convolution block; the eighth convolution block convolutes the seventh convolution block The abstract feature map of the block input is up-sampled and output to the ninth convolution block;

The segmentation unit includes a tenth convolution block; the tenth convolution block includes a convolution layer and an activation function unit; the convolution layer of the tenth convolution block performs 1×1 convolution processing, and the tenth convolution block has a The activation function unit is a Sigmoid function; the decoder feature map adjusts the number of model channels to the number of categories through 1×1 convolution processing of the tenth convolution block, and obtains the building segmentation result in the remote sensing image through the Sigmoid function.

In one embodiment, the attention valve includes an attention layer, a valve control layer, a first activation function unit, a linear transformation unit, a second activation function unit, and a resampler;

In the valve control layer, a valve control signal is input to the attention valve, and the valve control signal includes a control vector; wherein, the valve control signal comes from the previous convolution block of the current convolution block;

Multiply the control vector and the control coefficient to obtain the first product;

In the attention layer, the feature map is input to the attention valve; wherein, the feature map input to the attention valve comes from the convolution block that is skip-connected to the current convolution block;

The pixel vector of the feature map of the input attention valve is multiplied by the attention coefficient to obtain the second product;

The first product and the second product are added together, and the first activation function unit is used for activation after bias processing to obtain the first activation result; The second activation function unit is activated to obtain the second activation result, and the second activation result is input into the resampler for adjustment to obtain the attention coefficient α; the feature map input by the attention layer is multiplied by the attention coefficient α, and the output is equal to the attention coefficient α. The feature maps input to the force layer have the same size multiplication result.

In an embodiment, the attention valve uses an additional attention mechanism to obtain an attention coefficient; the calculation formula of the additional attention mechanism is as follows:

为sigmoid函数；in,

is the sigmoid function;

b_ψ∈R；其中，W_x为控制系数，W_g为注意力系数，ψ为线性变换，b_g为第一偏置参数，b_ψ为第二偏置系数；其中，F_l表示第l层中特征图的数量，F_int表示输入层；The parameter group _Θatt of the attention valve includes:

b _ψ ∈ R; where W _x is the control coefficient, W _g is the attention coefficient, ψ is the linear transformation, b _g is the first bias parameter, and b _ψ is the second bias coefficient; among them, F _l represents the first The number of feature maps in the layer, F _int represents the input layer;

Among them, the first activation function performed by the first activation function unit of the attention valve is shown in the following formula:

其中，i表示空间维度，c表示通道维度；

Among them, i represents the spatial dimension, and c represents the channel dimension;

Wherein, the linear transformation performed by the linear transformation unit is to change the number of channels of the input vector through 1×1×1 channel convolution processing;

Wherein, the second activation function σ ₂ executed by the second activation function unit of the attention valve is a sigmoid function;

其中，注意力系数α∈[0，1]。The multiplication result of the input feature map and the attention coefficient α at the pixel level is the output of the attention valve, as shown in the following formula:

Among them, the attention coefficient α∈[0, 1].

Implementing the embodiment of the present invention will have the following beneficial effects:

In the present invention, the localization and segmentation of building objects are realized by integrating attention valve AGs in the convolutional neural network CNN, and there is no need to train multiple models and a large number of additional model parameters; Unlike localization models, attention valve AGs suppress feature responses in irrelevant background regions in multiple dimensions, without clipping regions of interest between networks.

The AGs-Unet model is a U-shaped attention control model built on the U-Net architecture; U-Nets are widely used for image segmentation tasks due to their good computational performance and efficient use of GPU memory; the attention mechanism can be used in image segmentation. In multi-scale feature extraction, the effective features of the target are highlighted and the redundant and invalid information is suppressed. The AGs-Unet model can combine the advantages of both. In the low-dimensional scale, it can capture the wide-area information of the context and extract the global coarse-scale features of the image; in the high-dimensional scale, the abstract and fine-level features can be extracted, and the position and boundary of the building in the feature map can be highlighted through the attention valve. The gridded, multi-scale extracted feature maps are connected to the decoder part through skip connections to fuse the dense building predictions at the coarse and fine levels.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

in:

1 is a schematic diagram of a high-resolution remote sensing image building extraction device based on a U-shaped attention control network in the present invention;

FIG. 2 is a schematic diagram of the structure of the attention valve in the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

High-resolution remote sensing images are optical images with many types of ground objects, complex ground backgrounds, complex data information, and high resolution. Therefore, the use of deep learning methods to process high-resolution remote sensing images also requires continuous development. In order to efficiently, quickly and accurately complete the task of building extraction from high-resolution remote sensing images, we use machine translation and image captioning tasks in the field of natural language processing to explore the importance of attention maps by interpreting the gradient of the output class scores relative to the input image. Attention Gates (AGs for short), the present invention discloses a grid-based attention gate AGs, and integrates the attention gate AGs into U-Net to form a U-shaped attention control network (AGs-Unet ).

The present invention proposes a high-resolution remote sensing image building extraction model based on a U-shaped attention control network, and compares it with the traditional FCN8s, SegNet, U-Net models with classical symmetric structural features and the collective channel attention mechanism Compared with the DANet model of the location attention mechanism, the WHU data set is used to perform an experimental comparison of the automatic extraction task of remote sensing image buildings in terms of prediction accuracy, parameter quantity, and training time.

As shown in Figure 1, the present invention discloses a method for extracting buildings from high-resolution remote sensing images based on U-shaped attention control network, including:

Input remote sensing images to the encoder; the encoder uses global context information and local context information to extract features (x ^l ) of different grid dimension levels and generate encoder feature maps; the encoder includes four cascaded The convolution blocks are the first convolution block, the second convolution block, the third convolution block, and the fourth convolution block;

In particular, each convolution block in the encoder, that is, the first convolution block, the second convolution block, the third convolution block, and the fourth convolution block all include a convolutional layer (Convolutional layer, referred to as Conv), Maximum pooling layer (Maxpool Layer, referred to as Maxpool), normalization layer (Batch Normalization, referred to as BN), activation function unit (RectifiedLinear Unit, referred to as ReLU); wherein, the maximum pooling layer in the convolution block of the encoder passes through Extract the maximum value of the local area in the original feature map to generate a new feature map, and prevent overfitting by reducing the number of parameters;

The encoder is connected to the converter; the encoder outputs the encoder feature map generated by the extraction to the converter; the converter includes a fifth convolution block and an attention valve in four skip connections;

The converter is connected to a decoder; an attention valve is serialized in a skip connection between the convolution blocks of the encoder and the decoder; the converter connects the phase between the encoder and the decoder. the corresponding feature map; the converter extracts the abstract feature map and outputs it to the decoder;

The decoder and the encoder have the same number of convolution blocks; the decoder includes four convolution blocks concatenated, which are the sixth convolution block, the seventh convolution block, the eighth convolution block, and the ninth convolution block. Convolution block; wherein, the first convolution block is connected to the ninth convolution block, the second convolution block is connected to the eighth convolution block, the third convolution block is connected to the seventh convolution block, and the fourth convolution block is connected to the fourth convolution block. The block skip is connected to the sixth convolution block;

The converter includes a fifth convolution block and an attention valve in four skip connections; the converter connects the feature maps corresponding to the encoder and the decoder; Attention valves are strung in the skip connections between the convolutional blocks of the decoder;

In particular, the fifth convolution block includes a convolution layer (Conv), a maximum pooling layer (Maxpool), and a normalization layer (BN); the fifth convolution block uses the encoder feature map input by the encoder Abstract to the highest level, superimpose the number of channels of the feature map and reduce the size of the feature map, thereby extracting the abstract feature map of the highest dimension; the converter outputs the extracted abstract feature map to the decoder;

In an embodiment, the fifth convolution block superimposes the number of channels of the encoder feature map input by the encoder to 1248, and reduces the size of the encoder feature map to 16×16;

The decoder is connected to the segmentation unit; the decoder performs a step-by-step upsampling process on the abstract feature map input by the converter, generates a decoder feature map with the same size as the input image, and outputs it to the segmentation unit ; the segmentation unit adjusts the input model channel number of the decoder feature map, and obtains the building segmentation result in the remote sensing image.

The decoder performs step-by-step up-sampling processing on the abstract feature map input by the converter, which specifically includes:

The sixth convolution block performs up-sampling processing on the abstract feature map input by the fifth convolution block and outputs it to the seventh convolution block; the seventh convolution block performs up-sampling processing on the abstract feature map input by the sixth convolution block and outputs output to the eighth convolution block; the eighth convolution block performs upsampling processing on the abstract feature map input by the seventh convolution block and outputs it to the ninth convolution block;

In particular, the segmentation unit includes a tenth convolution block; the tenth convolution block includes a convolution layer and an activation function unit; the convolution layer of the tenth convolution block performs 1×1 convolution processing, and the tenth convolution block The activation function unit is the Sigmoid function; the decoder feature map adjusts the number of model channels to the number of categories through the 1×1 convolution processing of the tenth convolution block, and obtains the building segmentation result in the remote sensing image through the Sigmoid function;

In one embodiment, the parameters of the U-shaped attention control network AGs-Unet are set as shown in Table 1 below, where Conv represents convolution, Maxpool represents maximum pooling, Up represents upsampling, and AGs represents attention valve, Up_conv means upsampling plus convolution;

Table 1

Figure 2 shows the attention valve, which includes:

Wherein, the attention valve includes an attention layer, a valve control layer, a first activation function unit, a linear transformation unit, a second activation function unit, and a resampler;

In the valve control layer, a valve control signal is input to the attention valve, and the valve control signal includes a control vector (g _i ); wherein, the valve control signal comes from the previous convolution block of the current convolution block;

Specifically, the convolution block of the previous stage of the sixth convolution block is the fifth convolution block, the convolution block of the previous stage of the seventh convolution block is the sixth convolution block, and the previous stage of the eighth convolution block The convolution block is the seventh convolution block, and the previous convolution block of the ninth convolution block is the sixth and eighth convolution block;

Multiply the control vector and the control coefficient to obtain the first product;

The control vector is applied to each pixel to determine the focus area; the control vector contains contextual information to remove low-level feature responses;

In the attention layer, the feature map is input to the attention valve; wherein, the feature map input to the attention valve comes from the convolution block that is skip-connected to the current convolution block;

Specifically, the first convolution block is skip connected to the ninth convolution block, and the feature map input to the ninth convolution block comes from the first convolution block; the second convolution block is skip connected to the eighth convolution block, and the eighth convolution block is input. The feature map of the convolution block comes from the second convolution block; the third convolution block is skip connected to the seventh convolution block, and the feature map input to the seventh convolution block comes from the third convolution block; the fourth convolution block is skip connected To the sixth convolution block, the feature map input to the sixth convolution block comes from the fourth convolution block;

与注意力系数相乘得到第二乘积；其中，像素向量

F_l表示第l层中特征图的数量；Pixel vector of the feature map of the input attention valve

Multiplied by the attention coefficient to get the second product; where the pixel vector

F _l represents the number of feature maps in the lth layer;

The first product and the second product are added together, and the first activation function unit is used for activation after bias processing to obtain the first activation result; The second activation function unit is activated to obtain the second activation result, and the second activation result is input into the resampler for adjustment to obtain the attention coefficient α; the feature map input by the attention layer is multiplied by the attention coefficient α, and the output is equal to the attention coefficient α. The feature map input by the force layer has the same size of the multiplication result;

其中，注意力系数α∈[0，1]；Specifically, the multiplication result of the input feature map and the attention coefficient α at the pixel level is the output of the attention valve, as shown in the following formula:

Among them, the attention coefficient α∈[0,1];

Specifically, the attention valve uses an additional attention mechanism to obtain an attention coefficient; the calculation formula of the additional attention mechanism is as follows:

为sigmoid函数；in,

is the sigmoid function;

b_ψ∈R；其中，W_x为控制系数，W_g为注意力系数，ψ为线性变换，b_g为第一偏置参数，b_ψ为第二偏置系数；其中，F_int表示输入层；The parameter group _Θatt of the attention valve includes:

b _ψ ∈ R; where W _x is the control coefficient, W _g is the attention coefficient, ψ is the linear transformation, b _g is the first bias parameter, and b _ψ is the second bias coefficient; where F _int represents the input layer ;

其中，i表示空间维度，c表示通道维度；Among them, the first activation function σ ₁ executed by the first activation function unit of the attention valve is shown in the following formula:

Among them, i represents the spatial dimension, and c represents the channel dimension;

Wherein, the linear transformation performed by the linear transformation unit is to change the number of channels of the input vector through 1×1×1 channel convolution processing;

Wherein, the second activation function σ ₂ executed by the second activation function unit of the attention valve is a sigmoid function;

Extracting buildings from remote sensing images belongs to a single-category semantic segmentation task, so an attention coefficient α can be set; the attention valve identifies prominent image areas and suppresses irrelevant feature responses, while maximizing retention and activation only related to buildings related neurons;

The realization of the attention valve includes: training control coefficient W _g ; training attention coefficient W _x ; P _s connecting the two parts and adjusting the number of output channels.

In addition, the invention also discloses a high-resolution remote sensing image building extraction device based on a U-shaped attention control network, comprising an encoder, a converter, a decoder and a segmentation unit; the encoder is connected to the converter , the converter is connected to the decoder, and the decoder is connected to the dividing unit;

Input remote sensing images to the encoder; the encoder uses global context information and local context information to extract features (x ^l ) of different grid dimension levels and generate encoder feature maps; the encoder includes four cascaded The convolution blocks are the first convolution block, the second convolution block, the third convolution block, and the fourth convolution block;

In particular, each convolution block in the encoder includes a convolution layer (Conv), a maximum pooling layer (Maxpool), a normalization layer (BN), and an activation function unit (ReLU); wherein, the encoder's The maximum pooling layer in the convolution block generates a new feature map by extracting the maximum value of the local area in the original feature map, and prevents overfitting by reducing the number of parameters;

The encoder outputs the encoder feature map generated by extraction to the converter;

The decoder and the encoder have the same number of convolution blocks; the decoder includes four convolution blocks concatenated, which are the sixth convolution block, the seventh convolution block, the eighth convolution block, and the ninth convolution block. block;

Among them, the first convolution block is skipped to connect to the ninth convolution block, the second convolution block is skipped to the eighth convolution block, the third convolution block is skipped to the seventh convolution block, and the fourth convolution block is skipped connected to the sixth convolution block;

The converter includes a fifth convolution block, attention valves (AGs) in four skip connections; the converter connects the corresponding feature maps in the encoder and the decoder; in the encoding attention valves are serialized in skip connections between the convolutional blocks of the decoder and the decoder; the converter extracts abstract feature maps and outputs them to the decoder;

In particular, the fifth convolution block includes a convolution layer (Conv), a maximum pooling layer (Maxpool), and a normalization layer (BN); the fifth convolution block uses the encoder feature map input by the encoder Abstract to the highest level, superimpose the number of channels of the feature map and reduce the size of the feature map, thereby extracting the abstract feature map of the highest dimension; the converter outputs the extracted abstract feature map to the decoder;

In an embodiment, the fifth convolution block superimposes the number of channels of the encoder feature map input by the encoder to 1248, and reduces the size of the encoder feature map to 16×16;

Among them, the attention valves stringed into the skip connection screen out low-dimensional feature maps that are conducive to extracting feature points of buildings, filtering and suppressing irrelevant features and nodes; four attention valves are used in four different networks from low to high. Extract effective features from all aspects and multiple dimensions in the grid dimension level;

The converter connects the corresponding feature maps in the encoder and the decoder, and solves the problem of the vanishing gradient of backpropagation;

The decoder performs a step-by-step upsampling process on the abstract feature map input by the converter, generates a decoder feature map with the same size as the input image, and outputs it to the segmentation unit;

In particular, the decoder performs a step-by-step upsampling process on the abstract feature map input by the converter, which specifically includes:

The sixth convolution block performs up-sampling processing on the abstract feature map input by the fifth convolution block and outputs it to the seventh convolution block; the seventh convolution block performs up-sampling processing on the abstract feature map input by the sixth convolution block and outputs output to the eighth convolution block; the eighth convolution block performs upsampling processing on the abstract feature map input by the seventh convolution block and outputs it to the ninth convolution block;

The segmentation unit adjusts the input channel number of the decoder feature map, and obtains the building segmentation result in the remote sensing image;

In particular, the segmentation unit includes a tenth convolution block; the tenth convolution block includes a convolution layer and an activation function unit;

The convolution layer of the tenth convolution block performs 1×1 convolution processing, and the activation function unit of the tenth convolution block is a sigmoid function; the decoder feature map is adjusted by the 1×1 convolution processing of the tenth convolution block. The number of model channels is the number of categories, and the building segmentation results in remote sensing images are obtained through the Sigmoid function;

Figure 2 shows the attention valve, which includes:

Wherein, the attention valve includes an attention layer, a valve control layer, a first activation function unit, a linear transformation unit, a second activation function unit, and a resampler;

In the valve control layer, a valve control signal is input to the attention valve, and the valve control signal includes a control vector (g _i ); wherein, the valve control signal comes from the previous convolution block of the current convolution block;

Specifically, the convolution block of the previous stage of the sixth convolution block is the fifth convolution block, the convolution block of the previous stage of the seventh convolution block is the sixth convolution block, and the previous stage of the eighth convolution block The convolution block is the seventh convolution block, and the previous convolution block of the ninth convolution block is the sixth and eighth convolution block;

Multiply the control vector and the control coefficient to obtain the first product;

The control vector is applied to each pixel to determine the focus area; the control vector contains contextual information to remove low-level feature responses;

In the attention layer, the feature map is input to the attention valve; wherein, the feature map input to the attention valve comes from the convolution block that is skip-connected to the current convolution block;

Specifically, the first convolution block is skip connected to the ninth convolution block, and the feature map input to the ninth convolution block is from the first convolution block; the second convolution block is skip connected to the eighth convolution block, and the eighth convolution block is input The feature map of the convolution block comes from the second convolution block; the third convolution block is skip connected to the seventh convolution block, and the feature map input to the seventh convolution block comes from the third convolution block; the fourth convolution block is skip connected To the sixth convolution block, the feature map input to the sixth convolution block comes from the fourth convolution block;

与注意力系数相乘得到第二乘积；其中，像素向量

F_l表示第l层中特征图的数量；Pixel vector of the feature map of the input attention valve

Multiplied by the attention coefficient to get the second product; where the pixel vector

F _l represents the number of feature maps in the lth layer;

The first product and the second product are added together, and the first activation function unit is used for activation after bias processing to obtain the first activation result; The second activation function unit is activated to obtain the second activation result, and the second activation result is input into the resampler for adjustment to obtain the attention coefficient α; the feature map input by the attention layer is multiplied by the attention coefficient α, and the output is equal to the attention coefficient α. The feature map input by the force layer has the same size of the multiplication result;

其中，注意力系数α∈[0，1]；Specifically, the multiplication result of the input feature map and the attention coefficient α at the pixel level is the output of the attention valve, as shown in the following formula:

Among them, the attention coefficient α∈[0,1];

Specifically, the attention valve uses an additional attention mechanism to obtain an attention coefficient; the calculation formula of the additional attention mechanism is as follows:

为sigmoid函数；in,

is the sigmoid function;

b_ψ∈R；其中，W_x为控制系数，W_g为注意力系数，ψ为线性变换，b_g为第一偏置参数，b_ψ为第二偏置系数；其中，F_int表示输入层；The parameter group _Θatt of the attention valve includes:

b _ψ ∈ R; where W _x is the control coefficient, W _g is the attention coefficient, ψ is the linear transformation, b _g is the first bias parameter, and b _ψ is the second bias coefficient; where F _int represents the input layer ;

其中，i表示空间维度，c表示通道维度；Among them, the first activation function σ ₁ executed by the first activation function unit of the attention valve is shown in the following formula:

Among them, i represents the spatial dimension, and c represents the channel dimension;

Wherein, the linear transformation performed by the linear transformation unit is to change the number of channels of the input vector through 1×1×1 channel convolution processing;

Wherein, the second activation function σ ₂ executed by the second activation function unit of the attention valve is a sigmoid function;

Extracting buildings from remote sensing images belongs to a single-category semantic segmentation task, so an attention coefficient α can be set; the attention valve identifies prominent image areas and suppresses irrelevant feature responses, while retaining and activating only buildings related to buildings to the greatest extent. related neurons;

The realization of the attention valve includes: training the control coefficient W _g ; training the attention coefficient W _x ; connecting the two parts and adjusting the number of output channels P _s ;

In one embodiment, the parameters for realizing the attention valve are shown in Table 2 below:

Table 2

In the image classification task, the softmax activation function is used to normalize the attention coefficient; however, using softmax multiple times will produce activation sparsity at the output, which will lead to the problem of gradient disappearance; therefore, using the sigmoid function in the attention valve makes the The parameters of the attention valve have better training convergence; in grid-based attention, the control signal is not a global single vector for all pixels, but a grid signal adapted to the spatial information of the image;

As shown in Figure 1, the valve control signal in each skip connection aggregates multi-dimensional information, which increases the grid resolution of the query signal and achieves better performance;

The invention adds an attention valve to the U-Net architecture, extracts the coarse-scale features in the multi-dimensional wide area obtained by skip connections, and eliminates the ambiguity of irrelevant information and noise responses. This process is performed in the connect operation so that only the relevant activations are called. In addition, the attentional valve selectively filters neurons that do not need to be activated in forward and backpropagation. Gradients starting from background regions are continuously weighted downwards in backpropagation, and the model parameters of the shallow layers can be continuously updated based on the spatial regions relevant to the building extraction task.

In the multi-dimensional attention valve, corresponding to a vector at each grid scale, in each attention valve, complementary information is extracted for fusion to define the output of the skip connection. To reduce the number of training parameters and computational complexity in the attention valve, a linear transformation (1×1×1 convolution) is performed without any spatial support, and the input feature map is mapped to the valve control through a downsampling process The resolution of the signal is then decoded by the corresponding linear transformation, and the feature map is mapped to the low-dimensional space for attention control operation.

The present invention utilizes deep supervision to make the intermediate feature maps have corresponding semantic distinction at each scale of the image; the depth supervision ensures that the attention valve units at different scales can process the input feature maps of a large range in time, and can also prevent Reconstruct dense predictions directly from a subset of skip connections.

The validity and accuracy of the model are verified and compared through experiments, and the overall accuracy (OA), precision (Precision) and intersection ratio (IoU) are used as indicators for model performance evaluation; OA represents the number of correctly classified pixels Ratio to the total number of pixels tested, Precision is the percentage of correctly classified positive pixels among all predicted positive pixels, and IoU describes segment-level accuracy;

The WHU building dataset is used for comparative experiments. The WHU building dataset is composed of aerial datasets and satellite datasets. Using the aerial image datasets among them, SegNet, FCN8s, DANet, U-Net and AGs-Unet are used to compare the WHU aviation datasets. The results of building extraction on the image dataset; compared with SegNet, FCN8s, DANet, U-Net and AGs-Unet have better smoothness of building edges on the verification data, and the buildings extracted by AGs-Unet are more accurate edge and less internal voids under the action of attention valve AGs;

Separately analyze the images in the test set, select the buildings in the images, and the SegNet model with the highest computational resource consumption is better than FCN8s and DANet to extract the integrity of the buildings. It has the best extraction effect to extract the more fluent and accurate edge lines and the more complete internal structure of the building;

Further analyze other images in the test set, and select two details of the entire building in the image: the convex part of the extension and the concave part of the concave part. The SegNet model prediction map does not extract the information of the convex part of the building, and it is in the concave local area. A large hole phenomenon is generated; both FCN8s and DANet fail to extract the convex local information of the building, and the FCN8s model is less missing for the concave local feature area, and relatively shows a better prediction effect; U-Net model extracts The building has convex local information, but there is a problem of incomplete target prediction in the concave local area; the AGs-Unett model can accurately extract the details of the convex and concave parts; according to the overall analysis of the validation set data, the SegNet and FCN8s models The integrity of the extracted buildings is not enough, and the edge fluency is not high; the AGs-Unet model extracts relatively smooth and accurate edge lines and relatively complete internal structures of buildings, which has the best extraction effect.

Comparing the accuracy results of overall accuracy, precision and average cross-union ratio, AGs-Unet achieves significantly the best results on all three evaluation metrics.

Implementing the embodiment of the present invention will have the following beneficial effects:

The AGs-Unet proposed by the present invention integrates the attention valve (AGs) into the U-Net, which solves the problem of extracting dense building objects in the semantic segmentation model of remote sensing images; the attention valve AGs is beneficial to eliminate the multi-level convolutional neural network The necessity of building localization module in (CNNs) reduces the number of parameters of the model and the time consumed by training, and improves the computational efficiency of the model; at the same time, attention valves (AGs) can be used for further expansion and integration into convolutional neural networks, Thus, tasks related to dense object prediction for image segmentation are efficiently accomplished.

The grid-based control gate proposed by the present invention makes the attention coefficient more specific to the local area, and proposes an implementation of the soft attention mechanism in the forward CNN model applied to the task of extracting buildings from remote sensing images. The proposed attention valve replaces the hard attention mechanism used in image classification and the object localization model in the image segmentation framework;

The present invention proposes an extension to the standard U-Net model to improve the sensitivity of the model to foreground pixels without the need for complex heuristic algorithms; through experimental comparison, it can be seen that compared with the standard U-Net model on the WHU data set , the accuracy of the AGs-Unet proposed in the present invention in extracting buildings from remote sensing images has been further improved.

The above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The recorded technical solutions are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. A high-resolution remote sensing image building extraction method based on a U-shaped attention control network is characterized by comprising the following steps:

inputting the remote sensing image to an encoder; the encoder extracts features of different grid dimension levels by using the global context information and the local context information and generates an encoder feature map; the encoder comprises four cascaded rolling blocks which are respectively a first rolling block, a second rolling block, a third rolling block and a fourth rolling block;

the encoder is connected to the converter; the encoder outputs the extracted and generated encoder characteristic diagram to the converter; the converter comprises a fifth volume block, attention valves in four jump connections;

the converter is connected to a decoder; (ii) cascading an attention valve in a jump connection between convolutional blocks of said encoder and decoder; the converter extracts an abstract feature map and outputs the abstract feature map to the decoder;

the decoder comprises four cascaded rolling blocks which are a sixth rolling block, a seventh rolling block, an eighth rolling block and a ninth rolling block respectively; the first convolution block is connected to the ninth convolution block in a jumping mode, the second convolution block is connected to the eighth convolution block in a jumping mode, the third convolution block is connected to the seventh convolution block in a jumping mode, and the fourth convolution block is connected to the sixth convolution block in a jumping mode;

the decoder is connected to a segmentation unit; the decoder performs up-sampling processing step by step on the abstract feature map input by the converter to generate a decoder feature map with the same size as the input image, and outputs the decoder feature map to the segmentation unit; and the segmentation unit adjusts the number of channels of the input decoder characteristic diagram to obtain a building segmentation result in the remote sensing image.

2. The method for extracting the high-resolution remote sensing image building based on the U-shaped attention control network as claimed in claim 1,

each convolution block in the encoder comprises a convolution layer, a maximum pooling layer, a normalization layer and an activation function unit; the maximum pooling layer in the rolling block of the encoder extracts the maximum value of a local area in the original characteristic graph to construct and generate a new characteristic graph, and overfitting is prevented by reducing the number of parameters;

the fifth convolution block comprises a convolution layer, a maximum pooling layer and a normalization layer; and abstracting the encoder characteristic diagram input by the encoder to the highest level by the fifth rolling block, overlapping the number of channels of the characteristic diagram and reducing the size of the characteristic diagram, thereby extracting the abstract characteristic diagram with the highest dimension.

3. The method for extracting the high-resolution remote sensing image building based on the U-shaped attention control network as claimed in claim 1,

wherein, the decoder performs progressive upsampling processing on the abstract feature map input by the converter, and specifically includes: the sixth convolution block performs up-sampling processing on the abstract feature map input by the fifth convolution block and outputs the abstract feature map to the seventh convolution block; the seventh convolution block performs up-sampling processing on the abstract feature map input by the sixth convolution block and outputs the abstract feature map to the eighth convolution block; the eighth convolution block performs up-sampling processing on the abstract feature map input by the seventh convolution block and outputs the abstract feature map to the ninth convolution block;

wherein the partition unit includes a tenth volume block; the tenth convolution block comprises a convolution layer and an activation function unit; the convolution layer of the tenth convolution block executes 1 × 1 convolution processing, and the active function unit of the tenth convolution block is a Sigmoid function; the decoder feature map adjusts the number of model channels to be the number of categories through 1 x 1 convolution processing of a tenth convolution block, and building segmentation results in remote sensing images are obtained through a Sigmoid function.

4. The method for extracting the high-resolution remote sensing image building based on the U-shaped attention control network as claimed in claim 1,

the attention valve comprises an attention layer, a valve control layer, a first activation function unit, a linear transformation unit, a second activation function unit and a resampler;

in a valve control layer, inputting a valve control signal to an attention valve, the valve control signal comprising a control vector; wherein the valve control signal is from a previous-stage volume block of a current volume block;

multiplying the control vector by the control coefficient to obtain a first product;

in the attention layer, inputting a characteristic diagram to an attention valve; wherein the feature map input to the attention valve is from a convolution block that is jumpingly connected to the current convolution block;

multiplying a pixel vector of the feature map input into the attention valve by the attention coefficient to obtain a second product;

adding the first product and the second product, performing offset processing, and activating by using a first activation function unit to obtain a first activation result; the first activation result is subjected to channel number adjustment and bias processing by the linear transformation unit, then is activated by the second activation function unit to obtain a second activation result, and the second activation result is input into the resampler to be adjusted to obtain an attention coefficient alpha; the feature map of the attention layer input is multiplied by the attention coefficient α, and a multiplication result having the same size as the feature map of the attention layer input is output.

5. The method for extracting the high-resolution remote sensing image building based on the U-shaped attention control network as claimed in claim 4,

the attention valve obtains an attention coefficient by using an additional attention mechanism; the calculation of the additional attention mechanism is as follows:

wherein,

is sigmoid function;

parameter set theta of attention valve_attThe method comprises the following steps:

b_ψe is R; wherein, W_xTo control the coefficient, W_gFor attention coefficients,. phi._gIs a first bias parameter, b_ψIs a second bias coefficient; wherein, F_lDenotes the number of feature maps in layer I, F_intRepresenting an input layer;

wherein the first activation function performed by the first activation function unit of the attention valve is as follows:

wherein i represents a spatial dimension and c represents a channel dimension;

wherein the linear transformation performed by the linear transformation unit changes the number of channels of the input vector by a 1 × 1 × 1 channel convolution process;

wherein the second activation function σ executed by the second activation function unit of the attention valve₂Is sigmoid function;

the result of multiplying the input feature map by the attention coefficient α at the pixel level is the output of the attention valve, as shown in the following equation:

wherein the attention coefficient alpha is ∈ [0, 1 ]]。

6. A high-resolution remote sensing image building extraction device based on a U-shaped attention control network is characterized by comprising an encoder, a converter, a decoder and a segmentation unit; the encoder is connected to the converter, the converter is connected to the decoder, and the decoder is connected to the dividing unit;

inputting a remote sensing image to the encoder; the encoder extracts features of different grid dimension levels by using the global context information and the local context information and generates an encoder feature map; the encoder comprises four cascaded rolling blocks which are respectively a first rolling block, a second rolling block, a third rolling block and a fourth rolling block; the encoder outputs the extracted and generated encoder characteristic diagram to the converter;

the decoder comprises four cascaded rolling blocks which are a sixth rolling block, a seventh rolling block, an eighth rolling block and a ninth rolling block respectively; the first convolution block is connected to the ninth convolution block in a jumping mode, the second convolution block is connected to the eighth convolution block in a jumping mode, the third convolution block is connected to the seventh convolution block in a jumping mode, and the fourth convolution block is connected to the sixth convolution block in a jumping mode;

the converter comprises a fifth volume block, attention valves in four jump connections; the converter connects the corresponding characteristic graphs of the encoder and the decoder; (ii) concatenating attention valves in a hopping connection between convolutional blocks of the encoder and the decoder; the converter extracts an abstract feature map and outputs the abstract feature map to the decoder;

the decoder performs up-sampling processing step by step on the abstract feature map input by the converter to generate a decoder feature map with the same size as the input image, and outputs the decoder feature map to the segmentation unit; and the segmentation unit adjusts the number of channels of the input decoder characteristic diagram to obtain a building segmentation result in the remote sensing image.

7. The building extraction device of high resolution remote sensing image based on U-shaped attention control network as claimed in claim 6,

each convolution block in the encoder comprises a convolution layer, a maximum pooling layer, a normalization layer and an activation function unit; the maximum pooling layer in the rolling block of the encoder extracts the maximum value of a local area in the original characteristic graph to construct and generate a new characteristic graph, and overfitting is prevented by reducing the number of parameters;

the fifth convolution block comprises a convolution layer, a maximum pooling layer and a normalization layer; and abstracting the encoder characteristic diagram input by the encoder to the highest level by the fifth rolling block, overlapping the number of channels of the characteristic diagram and reducing the size of the characteristic diagram, thereby extracting the abstract characteristic diagram with the highest dimension.

8. The building extraction device of high resolution remote sensing image based on U-shaped attention control network as claimed in claim 6,

wherein, the decoder performs progressive upsampling processing on the abstract feature map input by the converter, and specifically includes: the sixth convolution block performs up-sampling processing on the abstract feature map input by the fifth convolution block and outputs the abstract feature map to the seventh convolution block; the seventh convolution block performs up-sampling processing on the abstract feature map input by the sixth convolution block and outputs the abstract feature map to the eighth convolution block; the eighth convolution block performs up-sampling processing on the abstract feature map input by the seventh convolution block and outputs the abstract feature map to the ninth convolution block;

wherein the partition unit includes a tenth volume block; the tenth convolution block comprises a convolution layer and an activation function unit; the convolution layer of the tenth convolution block executes 1 × 1 convolution processing, and the active function unit of the tenth convolution block is a Sigmoid function; the decoder feature map adjusts the number of model channels to be the number of categories through 1 x 1 convolution processing of a tenth convolution block, and building segmentation results in remote sensing images are obtained through a Sigmoid function.

9. The building extraction device of high resolution remote sensing image based on U-shaped attention control network as claimed in claim 6,

the attention valve comprises an attention layer, a valve control layer, a first activation function unit, a linear transformation unit, a second activation function unit and a resampler;

in a valve control layer, inputting a valve control signal to an attention valve, the valve control signal comprising a control vector; wherein the valve control signal is from a previous-stage volume block of a current volume block;

multiplying the control vector by the control coefficient to obtain a first product;

in the attention layer, inputting a characteristic diagram to an attention valve; wherein the feature map input to the attention valve is from a convolution block that is jumpingly connected to the current convolution block;

multiplying a pixel vector of the feature map input into the attention valve by the attention coefficient to obtain a second product;

adding the first product and the second product, performing offset processing, and activating by using a first activation function unit to obtain a first activation result; the first activation result is subjected to channel number adjustment and bias processing by the linear transformation unit, then is activated by the second activation function unit to obtain a second activation result, and the second activation result is input into the resampler to be adjusted to obtain an attention coefficient alpha; the feature map of the attention layer input is multiplied by the attention coefficient α, and a multiplication result having the same size as the feature map of the attention layer input is output.

10. The building extraction device of high resolution remote sensing image based on U-shaped attention control network as claimed in claim 9,

the attention valve obtains an attention coefficient by using an additional attention mechanism; the calculation of the additional attention mechanism is as follows:

wherein,

is sigmoid function;

parameter set theta of attention valve_attThe method comprises the following steps:

b_ψe is R; wherein, W_xTo control the coefficient, W_gFor attention coefficients,. phi._gIs a first bias parameter, b_ψIs a second bias coefficient; wherein, F_lDenotes the number of feature maps in layer I, F_intRepresenting an input layer;

wherein the first activation function performed by the first activation function unit of the attention valve is as follows:

wherein i represents a spatial dimension and c represents a channel dimension;

wherein the linear transformation performed by the linear transformation unit changes the number of channels of the input vector by a 1 × 1 × 1 channel convolution process;

wherein the second activation function σ executed by the second activation function unit of the attention valve₂Is sigmoid function;

the result of multiplying the input feature map by the attention coefficient α at the pixel level is the output of the attention valve, as shown in the following equation:

wherein the attention coefficient alpha is ∈ [0, 1 ]]。

Images