CN116012364B - SAR image change detection method and device - Google Patents

Abstract

The embodiment of the invention discloses a SAR image change detection method and device. The method includes: acquiring target SAR images of two phases; performing feature extraction on the image through a feature extraction network; the feature extraction network includes: a first feature extraction module for feature extraction on the image to obtain a first feature map; a second feature The extraction module includes a channel attention module and a spatial attention module; the channel attention module is used to process the first feature map to determine the channel attention weight, and weight the first feature map based on the weight to obtain the second feature map; the spatial attention module The force module is used to process the second feature map to determine the spatial attention weight, weight the second feature map based on the weight to obtain the target feature map; perform difference analysis on the two target feature maps; determine the latter phase based on the target difference map The target image area that has changed relative to the previous phase. Based on this method, the change detection accuracy of SAR images can be improved.

Description

SAR image change detection method and device

Technical Field

The embodiments of the present invention relate to the field of computer technology, and in particular to a SAR image change detection method and device.

Background Art

Remote sensing image change detection extracts information about surface changes by calculating and analyzing the differences in images of the same geographic area at different times. It has been widely used in many fields, such as agricultural surveys, urban expansion, disaster prevention and early warning, and disaster assessment. Compared with common optical remote sensing images, the Synthetic Aperture Radar (SAR) system uses active microwave remote sensing pulse compression and synthetic aperture principles to improve the range resolution and azimuth resolution of imaging. It can not only obtain image data with a short revisit period and a wide coverage area all day and all weather, but also overcome the problems of optical images being easily disturbed by shadows, which makes sample preparation and information extraction difficult. It also has a strong recognition ability for water bodies and metals. The rich and stable information in SAR images with increasing resolution can provide a reliable data basis for research in target recognition and classification, and has great research value and development potential in change detection.

In recent years, in-depth research has been conducted on SAR image change detection methods at home and abroad. Based on structures such as wavelet convolutional neural networks, deep cascade networks, pyramid pooling convolutional neural networks, and multi-scale capsule networks, a series of change detection deep learning algorithms have been proposed, which can extract complex features at different levels from a large amount of data and greatly improve the speed and accuracy of image processing. However, the above methods are easily interfered by various factors in SAR images when dealing with change detection tasks, which affects the effect of information extraction.

The attention mechanism can perform weighted processing on feature maps, highlight the changing features of important types of objects in the image, suppress uninteresting targets and complex background factors, enhance noise robustness, and optimize the effect of information extraction. At present, a variety of change detection methods based on attention mechanisms have been proposed for optical images. For example, the deep supervised image fusion network IFN fuses the multi-level deep features of the original image with the image difference features to improve the boundary integrity and semantic consistency of the change feature map; the spatiotemporal attention neural network STANet is composed of a pyramid spatiotemporal attention module, and uses the self-attention mechanism to calculate weights for two pixels at different time positions to generate discriminative features, and improves the change detection performance by exploring the relationship between different spatiotemporal pixels.

However, deep learning change detection for SAR images based on the attention mechanism still has certain limitations in practical applications: First, the existing change detection model requires the analyzed image to have a high resolution and a stable and continuous revisit cycle in order to provide accurate change detection results. However, the spatial resolution and temporal resolution of most SAR images are difficult to meet the above requirements at the same time. Therefore, it is necessary to design a deep learning change detection model suitable for SAR images and capable of accurate detection. Secondly, due to the large differences between optical images and SAR images in imaging principles, image features, semantic information, etc., directly applying the deep learning change detection model based on the attention mechanism for optical images to SAR images will cause many problems. Therefore, most of the current research on deep learning change detection methods for SAR images remains at the theoretical level, or can only achieve simple recognition and analysis of changes in a very small range, and the actual prediction effect, robustness and generalization are very limited.

Summary of the invention

An object of embodiments of the present invention is to solve at least the above problems and/or disadvantages and to provide at least the advantages to be described below.

The embodiments of the present invention provide a SAR image change detection method and device, which can extract fine change information of small-scale targets in SAR images of different phases and improve the change detection accuracy of SAR images.

In a first aspect, a SAR image change detection method is provided, comprising:

Acquire target SAR images in two phases;

Performing feature extraction on the target SAR images of the two time phases through a feature extraction network to obtain target feature maps of the two time phases;

The feature extraction network includes:

A first feature extraction module is used to extract features from the target SAR image of each time phase to obtain a first feature map of each time phase;

The second feature extraction module includes a channel attention module and a spatial attention module; wherein the channel attention module is used to process the first feature map of each phase based on the attention mechanism, determine the channel attention weight of the first feature map of each phase, and perform weighted processing on the first feature map of each phase based on the channel attention weight to obtain the second feature map of each phase; the spatial attention module is used to process the second feature map of each phase based on the attention mechanism, determine the spatial attention weight of the second feature map of each phase, and perform weighted processing on the second feature map of each phase based on the spatial attention weight to obtain the target feature map of each phase;

Performing difference analysis on the target feature graphs of the two phases to generate a target difference graph;

According to the target difference map, a target image region in which the target SAR image of the latter phase of the target SAR images of the two phases changes relative to the target SAR image of the former phase is determined.

Optionally, the first feature extraction module includes a first convolutional layer, a plurality of residual modules and a second convolutional layer connected in sequence;

The first convolutional layer is used to perform dimensionality reduction processing on the target SAR image of each phase;

Each residual module is used to extract features from its own input features, and the input features of the first residual module are the output features of the first convolutional layer;

The second convolutional layer is used to perform dimensionality increase processing on the output features of the last residual module to obtain a first feature map with the same dimension as the target SAR image in each phase.

Optionally, the channel attention module includes a first maximum pooling layer, a first average pooling layer, a third convolutional layer, a first fusion layer, a first sigmoid activation function and a first output layer, wherein the first maximum pooling layer and the first average pooling layer are connected in parallel to the input end of the third convolutional layer, and the third convolutional layer, the first fusion layer, the first sigmoid activation function and the first output layer are connected in sequence;

The first maximum pooling layer and the first average pooling layer are used to perform maximum pooling processing and average pooling processing on the first feature map of each phase in the spatial dimension, respectively;

The third convolutional layer is used to perform convolution processing on the output features of the first maximum pooling layer and the first average pooling layer respectively to obtain two channel attention weight matrices;

The first fusion layer is used to add the two channel attention weight matrices to obtain a total channel attention weight matrix;

The first sigmoid activation function is used to activate the total channel attention weight matrix to obtain the channel attention weight;

The first output layer is used to perform weighted processing on the first feature map of each time phase based on the channel attention weight to obtain the second feature map of each time phase.

Optionally, the spatial attention module includes a second maximum pooling layer, a second average pooling layer, a second fusion layer, a fourth convolutional layer, a second sigmoid activation function and a second output layer, the second maximum pooling layer and the second average pooling layer are connected in parallel to the input end of the second fusion layer, and the second fusion layer, the fourth convolutional layer and the second output layer are connected in sequence;

The second maximum pooling layer and the second average pooling layer are used to perform maximum pooling processing and average pooling processing on the second feature map of each phase in the channel dimension respectively;

The second fusion layer is used to perform vector concatenation on output features of the second maximum pooling layer and the second average pooling layer;

The fourth convolutional layer is used to perform convolution processing on the output features obtained by vector concatenation to obtain a spatial attention weight matrix; the second sigmoid activation function is used to perform activation processing on the spatial attention weight matrix to obtain the spatial attention weight;

The second output layer is used to perform weighted processing on the second feature map of each phase based on the spatial attention weight to obtain a target feature map of each phase.

Optionally, performing difference analysis on the target feature maps of the two phases to generate a target difference map includes:

Calculating the target feature maps of the two phases based on a difference operator and a logarithmic ratio operator respectively to obtain a difference map based on the difference operator and a difference map based on the logarithmic ratio operator;

The difference map based on the difference operator and the difference map based on the logarithmic ratio operator are fused to generate the target difference map.

Optionally, the fusing the difference map based on the difference operator and the difference map based on the logarithmic ratio operator to generate the target difference map includes:

A weighted fusion process is performed on the difference map based on the difference operator and the difference map based on the logarithmic ratio operator to generate the target difference map.

Optionally, the method further comprises:

Acquire multiple sample SAR image groups, wherein each sample SAR image group includes sample SAR images of two phases and sample labels, wherein the sample labels are used to indicate an actual image region in which the sample SAR image of the latter phase of the sample SAR images of the two phases changes relative to the sample SAR image of the previous phase;

Performing feature extraction on sample SAR images of two time phases in each sample SAR image group through the feature extraction network to be trained, and obtaining sample feature maps of the two time phases;

Performing difference analysis on the sample feature maps of the two phases corresponding to each sample SAR image group to generate a sample difference map corresponding to each sample SAR image group;

According to the sample difference map corresponding to each sample SAR image group, determine the image area in which the sample SAR image of the latter phase in each sample SAR image group changes relative to the sample SAR image of the previous phase;

Determine loss information according to the image area of the sample SAR image of the latter phase that has changed relative to the sample SAR image of the previous phase in each sample SAR image group and the sample label;

The feature extraction network to be trained is trained according to the loss information.

Optionally, acquiring a plurality of sample SAR image groups includes:

Acquire original SAR images of multiple phases;

Performing data enhancement processing on the original SAR images of the multiple time phases to obtain sample SAR images of the multiple time phases;

A plurality of sample SAR image groups are constructed according to the sample SAR images of the plurality of time phases.

Optionally, determining the loss information according to the image area and sample label of the sample SAR image of the next phase in each sample SAR image group that has changed relative to the sample SAR image of the previous phase includes:

According to the image area and sample label of the sample SAR image of the latter phase in each sample SAR image group that has changed relative to the sample SAR image of the previous phase, the set similarity loss information and the cross entropy loss information are determined respectively;

Mixed loss information is determined according to the set similarity loss information and the cross entropy loss information.

In a second aspect, a SAR image change detection device is provided, comprising:

A target image acquisition module is used to acquire target SAR images in two time phases;

A target feature map extraction module is used to extract features from the target SAR images of the two time phases through a feature extraction network to obtain target feature maps of the two time phases;

The feature extraction network includes:

A first feature extraction module is used to extract features from the target SAR image of each time phase to obtain a first feature map of each time phase;

The second feature extraction module includes a channel attention module and a spatial attention module; wherein the channel attention module is used to process the first feature map of each phase based on the attention mechanism, determine the channel attention weight of the first feature map of each phase, and perform weighted processing on the first feature map of each phase based on the channel attention weight to obtain the second feature map of each phase; the spatial attention module is used to process the second feature map of each phase based on the attention mechanism, determine the spatial attention weight of the second feature map of each phase, and perform weighted processing on the second feature map of each phase based on the spatial attention weight to obtain the target feature map of each phase;

A target difference map generating module, used for performing difference analysis on the target feature maps of the two time phases to generate a target difference map;

The target image region determination module is used to determine, according to the target difference map, a target image region of the target SAR image of the latter phase of the two phases, where the target SAR image changes relative to the target SAR image of the previous phase.

According to a third aspect, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the described method.

In a fourth aspect, a storage medium is provided, on which a computer program is stored, characterized in that when the program is executed by a processor, the described method is implemented.

The embodiments of the present invention have at least the following beneficial effects:

The embodiment of the present invention provides a SAR image change detection method and device. In the method, firstly, target SAR images of two phases are acquired; then, feature extraction is performed on the target SAR images of the two phases through a feature extraction network to obtain target feature maps of the two phases, and the feature extraction network includes: a first feature extraction module, which is used to extract features from the target SAR image of each phase to obtain a first feature map of each phase; a second feature extraction module, which includes a channel attention module and a spatial attention module; wherein the channel attention module is used to process the first feature map of each phase based on an attention mechanism, determine the channel attention weight of the first feature map of each phase, and based on The channel attention weight performs weighted processing on the first feature map of each phase to obtain the second feature map of each phase; the spatial attention module is used to process the second feature map of each phase based on the attention mechanism, determine the spatial attention weight of the second feature map of each phase, and perform weighted processing on the second feature map of each phase based on the spatial attention weight to obtain the target feature map of each phase; then the target feature maps of the two phases are subjected to difference analysis to generate a target difference map; finally, according to the target difference map, the target image area of the target SAR image of the latter phase in the target SAR image of the two phases is determined to be changed relative to the target SAR image of the previous phase. Based on this method and device, it extracts the basic feature information in the target SAR image through the first feature extraction module, and then uses the channel attention module and the spatial attention module in the second feature module to further extract the deep, multi-dimensional and refined features of the target SAR image, thereby extracting the fine change information of small-scale targets in SAR images of different phases, and improving the change detection accuracy of SAR images.

Other advantages, objectives, and features of the embodiments of the present invention will be reflected in part through the following description, and in part will be understood by those skilled in the art through study and practice of the embodiments of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of a SAR image change detection method provided by an embodiment of the present invention.

FIG. 2 is a schematic diagram of a change detection model provided by an embodiment of the present invention.

FIG3 is a flow chart of a method for training a feature extraction network according to an embodiment of the present invention.

FIG. 4 is a flow chart of a SAR image change detection method provided by another embodiment of the present invention.

FIG. 5 is a flow chart of data set production provided by another embodiment of the present invention.

FIG6a is a target SAR image of a previous phase provided by another embodiment of the present invention; FIG6b is a target SAR image of a later phase provided by another embodiment of the present invention; FIG6c is a label image provided by another embodiment of the present invention; and FIG6d is a SAR image change detection result provided by another embodiment of the present invention.

FIG. 7 is a schematic diagram of the structure of a SAR image change detection device provided by an embodiment of the present invention.

FIG8 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

DETAILED DESCRIPTION

The embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings so that those skilled in the art can implement the embodiments according to the description.

FIG1 is a flow chart of a SAR image change detection method provided by an embodiment of the present invention, which is executed by a system with processing capabilities, a server device or a SAR image change detection apparatus. As shown in FIG1 , the method includes steps 110 to 140.

Step 110: Acquire target SAR images in two time phases.

Here, the time interval between the two target SAR images may be any time interval, such as 24 hours or 1 week. It should be understood that the two target SAR images used for change detection are SAR images for the same target area.

Step 120, extracting features from the target SAR images of the two phases through a feature extraction network to obtain target feature maps of the two phases; the feature extraction network includes: a first feature extraction module, used to extract features from the target SAR images of each phase to obtain a first feature map of each phase; a second feature extraction module, including a channel attention module and a spatial attention module; wherein the channel attention module is used to process the first feature map of each phase based on an attention mechanism, determine the channel attention weight of the first feature map of each phase, and perform weighted processing on the first feature map of each phase based on the channel attention weight to obtain a second feature map of each phase; the spatial attention module is used to process the second feature map of each phase based on an attention mechanism, determine the spatial attention weight of the second feature map of each phase, and perform weighted processing on the second feature map of each phase based on the spatial attention weight to obtain a target feature map of each phase.

In this step, first, the first feature extraction module extracts the basic feature information in the target SAR image, that is, the first feature map. Next, the channel attention module processes the first feature map based on the attention mechanism and calculates the channel attention weight. The channel attention weight can represent the importance of each channel in the first feature map, that is, a higher weight is given to the channel containing key information and a lower weight is given to the channel lacking key information. Therefore, weighted processing of the first feature map based on the channel attention weight can extract key information in the channel dimension, generate a second feature map, and suppress the extraction of non-key information in the channel dimension in the second feature map. Further, the spatial attention module processes the second feature map based on the attention mechanism and calculates the spatial attention weight. The spatial attention weight can represent the importance of each spatial position in the second feature map, that is, a higher weight is given to the spatial position containing key information and a lower weight is given to the spatial position lacking key information. Therefore, weighted processing of the second feature map based on the spatial attention weight can extract key information in the spatial dimension, generate a target feature map, and suppress the extraction of non-key information in the spatial dimension in the target feature map. Based on the channel attention module and the spatial attention module, deep, multi-dimensional and refined features of the target SAR image can be extracted, improving the feature representation capability of the second feature extraction module.

Since the target feature maps of the two time phases extracted by the above-mentioned special extraction model will contain the change characteristics of the key targets in the target SAR image, further difference analysis is performed based on the target feature maps of the two time phases to generate a target difference map, which can achieve accurate change detection of key targets in SAR images such as ships and buildings, thereby improving the change detection accuracy of SAR images.

Fig. 2 is a schematic diagram of a change detection model provided by an embodiment of the present invention. As shown in Fig. 2, the change detection model includes a feature extraction network and a difference generation and analysis module, wherein the feature extraction network includes a first feature extraction module and a second feature extraction module.

As shown in Figure 2, in some embodiments, the first feature extraction module includes a first convolutional layer, multiple residual modules and a second convolutional layer connected in sequence; the first convolutional layer is used to perform dimensionality reduction processing on the target SAR image of each phase; each residual module is used to extract features from its own input features, and the input features of the first residual module are the output features of the first convolutional layer; the second convolutional layer is used to perform dimensionality increase processing on the output features of the last residual module to obtain a first feature map with the same dimension as the target SAR image of each phase.

In a traditional residual network, after multiple residual modules, a global pooling layer and a fully connected layer are usually connected, which will result in a large number of parameters in the traditional residual network and a decrease in computational efficiency. Therefore, an embodiment of the present invention uses an improved residual network to implement the first feature extraction module. Specifically, the global pooling layer and the fully connected layer connected to multiple residual modules in the traditional residual structure are replaced with a convolutional layer, and the convolutional layer (i.e., the second convolutional layer) has the output features of the last residual module dimensionality-upgraded to output a first feature map with the same dimension as the target SAR image, thereby forming an improved residual structure. In this way, the number of parameters of the first feature extraction module can be reduced on the basis of ensuring the feature extraction capability of the first feature extraction module, thereby ensuring the overall training efficiency and stability of the feature extraction network.

Specifically, in the first feature extraction module, each residual module (Resblock) is used to extract features from its own input features. The input features of each residual module come from the output features of the previous module connected to it. The residual module further includes a plurality of residual structures with the same structure, and each residual structure includes a plurality of convolutional layers. The first convolutional layer is used to perform dimensionality reduction processing on the target SAR image so that the dimension of the output features of the first convolutional layer meets the dimensionality requirement of the first residual module for the input features. The second convolutional layer is used to perform dimensionality increase processing on the output features of the last residual module so that the output features of the second convolutional layer are restored to the dimension of the target SAR image. The embodiment of the present invention does not specifically limit the specific structure of the residual module and the residual structure.

In some examples, an improved ResNet-34 residual network is used to implement the first feature extraction module. Figure 2 specifically shows the improved ResNet-34 residual network. In the improved ResNet-34 residual network, it includes a first convolutional layer, 4 residual modules and a second convolutional layer, wherein the residual modules ResBlock1, ResBlock2, ResBlock3, and ResBlock4 contain 3, 4, 6, and 3 residual structures, respectively (Figure 2 shows that the residual structure in the first residual module consists of 2 3×3 convolutional layers), and the second convolutional layer is connected after the last residual module ResBlock4, and the output feature of the second convolutional layer is the first feature map.

As shown in Figure 2, in some embodiments, the channel attention module includes a first maximum pooling layer, a first average pooling layer, a third convolutional layer, a first fusion layer, a first sigmoid activation function and a first output layer, wherein the first maximum pooling layer and the first average pooling layer are connected in parallel to the input end of the third convolutional layer, and the third convolutional layer, the first fusion layer, the first sigmoid activation function and the first output layer are connected in sequence; the first maximum pooling layer and the first average pooling layer are respectively used to perform maximum pooling processing and average pooling processing on the first feature map of each phase in the spatial dimension; the third convolutional layer is used to convolve the output features of the first maximum pooling layer and the first average pooling layer respectively to obtain two channel attention weight matrices; the first fusion layer is used to add the two channel attention weight matrices to obtain a total channel attention weight matrix; the first sigmoid activation function is used to activate the total channel attention weight matrix to obtain the channel attention weight; the first output layer is used to weight the first feature map of each phase based on the channel attention weight to obtain the second feature map of each phase.

Specifically, the input feature of the channel attention module is the output feature of the first feature extraction module, that is, the first feature map. The processing process of the channel attention module on the first feature map is:

First, the first maximum pooling layer and the first average pooling layer are used to perform maximum pooling and average pooling on the first feature map in the spatial dimension, respectively, to obtain two output features. Maximum pooling and average pooling can realize compression processing of the first feature map to improve the computational efficiency of the feature extraction model; in addition, using two compression methods at the same time can realize the utilization of different information in the first feature map to improve the feature representation ability of the channel attention module. Given that the dimension of the first feature map is H×W×C, where H, W, and C represent the height, width, and number of channels of the first feature map, respectively, after maximum pooling and average pooling in the spatial dimension, two 1×1×C output features can be obtained.

In the traditional channel attention module, the fully connected layer is usually used to calculate the channel attention weight matrix. However, the large number of parameters in the fully connected layer will lead to unsatisfactory computational efficiency of the feature extraction model. Based on this, the embodiment of the present invention uses the third convolutional layer to convolve the output features of the first maximum pooling layer and the first average pooling layer to obtain two channel attention weight matrices, each of which is used to represent the importance of each channel in the corresponding output feature. Compared with the fully connected layer, the number of parameters of the convolutional layer is greatly reduced, so the computational efficiency of the model can be effectively improved. At the same time, the convolutional layer in the channel attention module provided in the embodiment of the present invention can meet the requirements of the feature extraction model and ensure the accurate change detection of SAR images. In some examples, the third convolutional layer is a one-dimensional convolutional layer.

Afterwards, the first fusion layer is used to add the two channel attention weight matrices to obtain the total channel attention weight matrix. Specifically, the addition of the two channel attention weight matrices is the addition of the elements in the two matrices.

Then use the first sigmoid activation function to activate the total channel attention weight matrix to obtain the channel attention weight.

Finally, the first output layer is used to weight the first feature map with the channel attention weight to obtain the second feature map. Specifically, the second feature map can be obtained by multiplying the channel attention weight with the first feature map.

Accordingly, the calculation formula of the channel attention module for processing the first feature map can be expressed as:

M _C (F)＝σ(f(AvgPool(F))+f(MaxPool(F)))

Where F represents the first feature map, AvgPool(F) and MaxPool(F) are the average pooling and maximum pooling of the spatial dimension of the first feature map, σ is the sigmoid activation function, f is the convolution process, _Mc (F) represents the channel attention weight, and F′ represents the second feature map.

As shown in Figure 2, in some embodiments, the spatial attention module includes a second maximum pooling layer, a second average pooling layer, a second fusion layer, a fourth convolutional layer, a second sigmoid activation function and a second output layer, the second maximum pooling layer and the second average pooling layer are connected in parallel to the input end of the second fusion layer, and the second fusion layer, the fourth convolutional layer and the second output layer are connected in sequence; the second maximum pooling layer and the second average pooling layer are respectively used to perform maximum pooling processing and average pooling processing in the channel dimension on the second feature map of each phase; the second fusion layer is used to vector splice the output features of the second maximum pooling layer and the second average pooling layer; the fourth convolutional layer is used to convolve the output features obtained by vector splicing to obtain a spatial attention weight matrix; the second sigmoid activation function is used to activate the spatial attention weight matrix to obtain the spatial attention weight; the second output layer is used to weight the second feature map of each phase based on the spatial attention weight to obtain the target feature map of each phase.

Specifically, the input features of the spatial attention module are the output features of the channel attention module, that is, the second feature map. The processing process of the spatial attention module on the second feature map is:

First, the second feature map is subjected to maximum pooling and average pooling in the channel dimension using the second maximum pooling layer and the second average pooling layer, respectively, to obtain two output features. Maximum pooling and average pooling can realize compression processing of the second feature map to improve the computational efficiency of the feature extraction model; in addition, the use of two compression methods at the same time can realize the utilization of different information in the second feature map to improve the feature representation ability of the spatial attention module. Given that the dimension of the second feature map is H×W×C, where H, W, and C represent the height, width, and number of channels of the second feature map, respectively, two H×W×1 output features can be obtained after maximum pooling and average pooling in the channel dimension.

Afterwards, the second fusion layer is used to vectorize the output features of the second maximum pooling layer and the second average pooling layer, thereby achieving the fusion of the two output features.

The fourth convolutional layer is then used to perform convolution processing on the output features after the vector splicing to obtain a spatial attention weight matrix, which is used to represent the importance of each spatial position in the output features after the vector splicing. In the spatial attention module provided in the embodiment of the present invention, the convolutional layer can meet the requirements of the feature extraction model and ensure the accurate change detection of the SAR image.

Then, the second sigmoid activation function is used to activate the spatial attention weight matrix to obtain the spatial attention weight.

Finally, the second output layer is used to weight the second feature map with the spatial attention weight to obtain the target feature map. Specifically, the target feature map can be obtained by multiplying the spatial attention weight with the second feature map. The target feature map is the output feature of the second feature extraction module.

Accordingly, the calculation formula of the spatial attention module for processing the second feature map can be expressed as:

M _S (F')=σ(f(AvgPool(F');MaxPool(F')))

Wherein, F′ represents the second feature map, AvgPool(F′) and MaxPool(F′) are the average pooling and maximum pooling of the channel dimension of the second feature map, σ is the sigmoid activation function, f is the convolution process, _Ms (F′) represents the spatial attention weight, and F″ represents the target feature map.

Step 130, performing difference analysis on the target feature graphs of the two phases to generate a target difference graph.

As shown in Figure 2, in some embodiments, in step 130, a difference analysis is performed on the target feature maps of the two time phases to generate a target difference map, including: calculating the target feature maps of the two time phases based on a difference operator and a logarithmic ratio operator (LR operator for short) to obtain a difference map based on the difference operator and a difference map based on the logarithmic ratio operator; fusing the difference map based on the difference operator and the difference map based on the logarithmic ratio operator to generate the target difference map.

Through the above method, noise interference can be overcome to the greatest extent, and the details and overall information in the two target feature maps can be better integrated, thereby improving the change detection accuracy of SAR images.

The difference maps of two operators can be fused by using an image fusion method, for example, pixel-level, feature-level or decision-level fusion is used according to the level to which the difference map belongs. Among them, the pixel-level fusion method has the advantages of being easy to implement and having low computational complexity. Among the pixel-level fusion methods, the weighted fusion method has low computational complexity and is easier to implement. Based on this, in some examples, the fusion processing of the difference map based on the difference operator and the difference map based on the logarithmic ratio operator to generate the target difference map includes: weighted fusion processing of the difference map based on the difference operator and the difference map based on the logarithmic ratio operator to generate the target difference map.

Specifically, the calculation formula for the generation process of the target difference map can be expressed as follows:

X _C =α(|X ₁ -X ₂ |)+(1-α)(|log(X ₂ +1)-log(X ₁ +1)|)

Wherein, _X1 and _X2 represent the target feature maps of two phases respectively; | _X1 - _X2 | represents the difference map generated based on the difference operator; |log( _X2 +1)-log( _X1 +1)| represents the difference map generated based on the logarithmic ratio operator; α is the weight coefficient, ranging from 0 to 1; _Xc represents the target difference map.

Step 140 , determining, according to the target difference map, a target image region in which the target SAR image of the latter phase of the two phases changes relative to the target SAR image of the former phase.

Here, the target image region where the target SAR image of the latter phase of the two determined target SAR images changes relative to the target SAR image of the previous phase is the change detection result, which can be represented by a binary result map. In the binary result map, the changed pixels can be represented by black, and the other unchanged pixels can be represented by white.

In this step, the pixel category can be set into two categories, namely, the changed category and the unchanged category. Then, a threshold is set for the target difference map, and the category of the pixel is judged according to the threshold. When a pixel in the target difference map exceeds the set threshold, it is determined that the pixel belongs to the changed category. Conversely, when a pixel in the target difference map does not exceed the set threshold, it is determined that the pixel belongs to the unchanged category. Finally, based on the judgment result, a binary result map representing the change information is generated.

In some embodiments, the Kilter & Illingworth (KI) threshold method can be used to establish a performance index function to perform histogram fitting on the conditional distribution of pixels of the changed class and the unchanged class, and the minimum value of the function is obtained as the optimal threshold. Other threshold analysis methods can also be used to calculate the threshold, and the pixels in the target difference map are classified based on the calculated threshold. The embodiments of the present invention do not specifically limit this.

Fig. 3 is a flow chart of a method for training a feature extraction network provided by an embodiment of the present invention. As shown in Fig. 3, in some embodiments, the method for training a feature extraction network includes steps 310 to 360.

Step 310, acquiring a plurality of sample SAR image groups, wherein each sample SAR image group includes sample SAR images of two phases and sample labels, wherein the sample labels are used to indicate an actual image region in which the sample SAR image of the latter phase of the two sample SAR images changes relative to the sample SAR image of the previous phase.

In practical applications, there are few large-scale public data sets that can be used for deep learning model training of SAR images, and it is difficult to accurately label sample data, resulting in serious model overfitting problems. Based on this, in some embodiments, the acquisition of multiple sample SAR image groups includes: acquiring original SAR images of multiple phases; performing data enhancement processing on the original SAR images of the multiple phases to obtain sample SAR images of the multiple phases; and constructing multiple sample SAR image groups based on the sample SAR images of the multiple phases.

By performing data enhancement processing on the original SAR images of multiple phases, the number and diversity of images in the sample data set can be expanded, and the stability and generalization ability of the SAR image deep learning model can be improved.

The data enhancement method may be rotation, scaling, adding noise and occlusion, etc., which is not specifically limited in the embodiment of the present invention.

Before data enhancement is performed on the original SAR images, the original SAR images can be preprocessed, including radiometric calibration, image adaptive filtering, and geocoding. At the same time, they can be processed into 8-bit images using a linear 2% stretching method, and images in the same area can be cropped to the same size and precisely aligned using the regional block adjustment tool. Furthermore, the preprocessed images can be cropped to a size of 256 pixels × 256 pixels, with no overlap between the cuts, and then data enhancement can be performed by rotation, scaling, adding noise, and occlusion.

Each sample SAR image group also includes a sample label. The color component synthesis tool of the image processing software can be used to roughly calculate the change difference of the original SAR image or the original SAR image after image preprocessing, and then combined with high-precision map data and prior knowledge, the change samples are manually marked to generate sample labels. Sample labels can be implemented using binary reference images.

Step 320: extract features from the sample SAR images of two time phases in each sample SAR image group through the feature extraction network to be trained, and obtain sample feature maps of the two time phases.

Step 330 , performing difference analysis on the sample feature maps of the two phases corresponding to each sample SAR image group, and generating a sample difference map corresponding to each sample SAR image group.

Step 340: Determine, based on the sample difference map corresponding to each sample SAR image group, the image region where the sample SAR image of the latter phase in each sample SAR image group has changed relative to the sample SAR image of the previous phase.

Step 350 , determining loss information according to the image region of the sample SAR image of the next phase in each sample SAR image group that has changed relative to the sample SAR image of the previous phase and the sample label.

In some embodiments, in step 350, determining the loss information according to the image area and sample label of the sample SAR image of the subsequent phase in each sample SAR image group that have changed relative to the sample SAR image of the previous phase includes: determining the set similarity loss information and the cross entropy loss information respectively according to the image area and sample label of the sample SAR image of the subsequent phase in each sample SAR image group that have changed relative to the sample SAR image of the previous phase; and determining the mixed loss information according to the set similarity loss and the cross entropy loss.

The set similarity loss function Dice Loss and the cross entropy loss function CE Loss have the characteristics of focusing on the global and examining micro information respectively. The embodiment of the present invention constructs a hybrid loss function based on the above two to calculate the hybrid loss information, which can focus on the global and examine micro information, thereby comprehensively and efficiently evaluating the training situation and solving the problem of poor learning effect caused by imbalance of positive and negative samples.

In some examples, the set similarity loss information and the cross entropy loss information may be weighted and summed to calculate the mixed loss information. By assigning different weights to the two loss information, the influence of global information or micro information on the feature extraction network may be adjusted. In addition, the set similarity loss information and the cross entropy loss information may be added to calculate the mixed loss information. This method is equivalent to assigning the same weight to the two loss information to balance the examination of global and micro information.

The calculation formula of the set similarity loss function Dice Loss is as follows:

The calculation formula of the cross entropy loss function CE Loss is as follows:

L _CE = _-RT log _RP

Among them, _LD represents the set similarity loss information; _LCE represents the cross entropy loss information; _RT represents the sample label, which has two values: 0 or 1, indicating no change and change respectively; _RP represents the change prediction result, with a value range of [0,1].

Then, in some examples, the calculation formula of the hybrid loss function L can be expressed as:

L＝ _LD + _LCE

Step 360: training the feature extraction network to be trained according to the loss information.

During the training process of the feature extraction network, multiple iterations of training are performed, and the parameters of the feature extraction network are adjusted in each iteration until the training end condition is reached. The training end condition may be the number of iterations or a set detection accuracy threshold. The embodiment of the present invention does not specifically limit this.

It should be understood that the process of extracting features from sample SAR images using the feature extraction network to be trained in step 320 is the same as the step performed in step 120 of extracting features from target SAR images using the trained feature extraction network. At the same time, step 330 performs difference analysis on the sample feature maps of the two phases corresponding to each sample SAR image group to generate a sample difference map corresponding to each sample SAR image group, which should also be consistent with the step performed in step 130 of performing difference analysis on the target feature maps of the two phases to generate a target difference map. Similarly, step 340 determines the image region of the sample SAR image of the latter phase in each sample SAR image group that has changed relative to the sample SAR image of the previous phase according to the sample difference map corresponding to each sample SAR image group, which should also be consistent with the step performed in step 140 of determining the target image region of the target SAR image of the latter phase in the target SAR image of the two phases that has changed relative to the target SAR image of the previous phase according to the target difference map.

In summary, an embodiment of the present invention provides a SAR image change detection method. In the method, firstly, target SAR images of two phases are acquired; then, feature extraction is performed on the target SAR images of the two phases through a feature extraction network to obtain target feature maps of the two phases, and the feature extraction network includes: a first feature extraction module, which is used to extract features from the target SAR images of each phase to obtain a first feature map of each phase; a second feature extraction module, including a channel attention module and a spatial attention module; wherein the channel attention module is used to process the first feature map of each phase based on the attention mechanism, determine the channel attention weight of the first feature map of each phase, and perform channel attention on each first feature map based on the channel attention weight. The first feature map of each phase is weighted to obtain the second feature map of each phase; the spatial attention module is used to process the second feature map of each phase based on the attention mechanism, determine the spatial attention weight of the second feature map of each phase, and weight the second feature map of each phase based on the spatial attention weight to obtain the target feature map of each phase; then the target feature maps of the two phases are analyzed for differences to generate a target difference map; finally, according to the target difference map, the target image area of the target SAR image of the latter phase in the target SAR image of the two phases is determined to change relative to the target SAR image of the previous phase. Based on this method, it extracts the basic feature information in the target SAR image through the first feature extraction module, and then uses the channel attention module and the spatial attention module in the second feature module to further extract the deep, multi-dimensional and refined features of the target SAR image, thereby extracting the fine change information of small-scale targets in SAR images of different phases, and improving the change detection accuracy of SAR images.

A specific implementation scenario is provided below to further illustrate the SAR image change detection method provided by an embodiment of the present invention.

Fig. 4 is a flow chart of a SAR image change detection method provided by an embodiment of the present invention. The SAR image change detection method provided by an embodiment of the present invention mainly includes three parts: data set preparation, model building and model training.

Step 410: Dataset Creation

FIG5 is a flow chart of data set preparation according to an embodiment of the present invention. As shown in FIG5 , step 410 of the embodiment of the present invention further includes steps 411 to 415 .

Step 411, obtaining the original SAR image

Download multiple 1m resolution original SAR images of the same area at different times. The processing objects mainly include ships, buildings and wasteland near the port.

Step 412: Image preprocessing

The original SAR images were subjected to radiometric calibration, image adaptive filtering and geocoding operations in sequence, and processed into 8-bit images using a linear 2% stretching method. Images in the same area were cropped to the same size and then precisely aligned using the regional block adjustment tool.

Step 413: Sample labeling

The color component synthesis tool of the image processing software is used to roughly calculate the change differences between images in different phases. The high-precision map data and prior knowledge are combined to manually and accurately mark the change samples to generate a binary reference image as the label image.

Step 414: Data trimming and enhancement

All images were cropped to 256 pixels × 256 pixels to ensure that there was no overlap between the slices. The number and diversity of images in the dataset were increased through data augmentation operations such as rotation, scaling, adding noise and occlusion. The stability and generalization ability of the deep learning model were improved through data augmentation processing. Finally, 95,040 image groups consisting of before and after phase images and label maps were obtained.

Step 415: Divide the data set

According to the comprehensive principle of proportion and image transformation type, 60,000 groups of all data are used as training sets, 20,000 groups are used as validation sets, and the remaining 12,040 groups are divided into test sets.

Step 420: Build a change detection model

Fig. 2 is a schematic diagram of a change detection model provided by an embodiment of the present invention. As shown in Fig. 2, the change detection model includes a feature extraction network and a difference generation and analysis module, wherein the feature extraction network includes a first feature extraction module and a second feature extraction module.

Specifically, the improved ResNet-34 residual network is used to implement the first feature extraction module. Figure 2 specifically shows the improved ResNet-34 residual network. In the improved ResNet-34 residual network, there are 2 convolutional layers Conv and 4 residual modules, among which the residual modules ResBlock1, ResBlock2, ResBlock3, and ResBlock4 contain 3, 4, 6, and 3 residual structures respectively (Figure 2 shows that the residual structure in the first residual module consists of 2 3×3 convolutional layers), and a convolutional layer is connected before the first residual module ResBlock1 and after the last residual module ResBlock4.

The target SAR images _I1 and _I2 of the two time phases are respectively input into the first feature extraction module for feature extraction, and the image currently input into the first feature extraction module is the input image. When the first feature extraction module performs feature extraction on the target SAR image, the input image is firstly subjected to dimensionality reduction processing through the first convolution layer so that the output feature dimension of the first convolution layer meets the dimensionality requirement of the first residual module for the input feature, and then each residual module is subjected to feature extraction on its own input feature, and finally the output feature of the last residual module is subjected to dimensionality increase processing through the second convolution layer so that the output feature of the second convolution layer is restored to the dimension of the target SAR image, and the output feature of the second convolution layer is the first feature map.

The second feature extraction module includes a channel attention module and a spatial attention module.

The channel attention module includes the first maximum pooling layer MaxPool, the first average pooling layer AvgPool, the third convolution layer Conv, the first fusion layer, the first sigmoid activation function and the first output layer, wherein the first maximum pooling layer and the first average pooling layer are connected in parallel to the input end of the third convolution layer, the third convolution layer, the first fusion layer, the first sigmoid activation function and the first output layer are connected in sequence, and the third convolution layer is a one-dimensional convolution layer. When the first feature map is processed by the channel attention module, the first maximum pooling layer and the first average pooling layer are first used to perform maximum pooling and average pooling processing on the first feature map in the spatial dimension, respectively, to obtain two output features. Given that the dimension of the first feature map is H×W×C, where H, W, and C represent the height, width, and number of channels of the first feature map, respectively, after the maximum pooling and average pooling processing in the spatial dimension, two 1×1×C output features can be obtained. Then, the first fusion layer is used to add the two channel attention weight matrices to obtain the total channel attention weight matrix. The first sigmoid activation function is then used to activate the total channel attention weight matrix to obtain the channel attention weight. Finally, the first output layer is used to multiply the channel attention weight with the first feature map to obtain the second feature map.

The calculation formula of the channel attention module for processing the first feature map can be expressed as:

M _C (F)＝σ(f(AvgPool(F))+f(MaxPool(F)))

Where F represents the first feature map, AvgPool(F) and MaxPool(F) are the average pooling and maximum pooling of the spatial dimension of the first feature map, σ is the sigmoid activation function, f is the convolution process, _Mc (F) represents the channel attention weight, and F′ represents the second feature map.

The spatial attention module includes a second maximum pooling layer, a second average pooling layer, a second fusion layer, a fourth convolutional layer, a second sigmoid activation function, and a second output layer. The second maximum pooling layer and the second average pooling layer are connected in parallel to the input end of the second fusion layer. The second fusion layer, the fourth convolutional layer, and the second output layer are connected in sequence. The processing process of the spatial attention module on the second feature map is as follows: first, the second maximum pooling layer and the second average pooling layer are used to perform maximum pooling and average pooling on the second feature map in the channel dimension, respectively, to obtain two output features. Given that the dimension of the second feature map is H×W×C, where H, W, and C represent the height, width, and number of channels of the second feature map, respectively, after the maximum pooling and average pooling of the channel dimension, two H×W×1 output features can be obtained. Then, the second fusion layer is used to vectorize the output features of the second maximum pooling layer and the second average pooling layer. The fourth convolutional layer is then used to convolve the output features after vectorization to obtain the spatial attention weight matrix. Then, the second sigmoid activation function is used to activate the spatial attention weight matrix to obtain the spatial attention weight. Finally, the spatial attention weight is multiplied by the second feature map through the second output layer to obtain the target feature map as the output feature of the second feature extraction module.

The calculation formula of the spatial attention module for processing the second feature map can be expressed as:

M _S (F')=σ(f(AvgPool(F');MaxPool(F')))

Wherein, F′ represents the second feature map, AvgPool(F′) and MaxPool(F′) are the average pooling and maximum pooling of the channel dimension of the second feature map, σ is the sigmoid activation function, f is the convolution process, _Ms (F′) represents the spatial attention weight, and F″ represents the target feature map.

The embodiment of the present invention adopts a hybrid loss function that combines the set similarity loss function Dice Loss and the cross entropy loss function CELoss.

The calculation formula of the set similarity loss function Dice Loss is as follows:

The calculation formula of the cross entropy loss function CE Loss is as follows:

L _CE = _-RT log R _P

Among them, _LD represents the set similarity loss information; _LCE represents the cross entropy loss information; _RT represents the sample label, which has two values: 0 or 1, indicating no change and change respectively; _RP represents the change prediction result, with a value range of [0,1].

The calculation formula of the mixed loss function L can be expressed as:

L＝ _LD + _LCE

Next, the target feature maps _X1 and _X2 of the two phases are input to the difference generation and analysis module for subsequent processing. First, the target feature maps of the two phases are calculated based on the difference operator and the logarithmic ratio operator (LR operator for short), respectively, to obtain the difference map based on the difference operator and the difference map based on the logarithmic ratio operator, then the difference map based on the difference operator and the difference map based on the logarithmic ratio operator are fused to generate the target difference map, and then the threshold analysis is performed on the target difference map, and finally the target image area where the target SAR image of the latter phase changes relative to the target SAR image of the previous phase is determined, and the change detection result is output.

The calculation formula of the target difference map generation process can be expressed as follows:

X _C =α(X ₁ -X ₂ |)+(1-α)(log(X ₂ +1)-log(X ₁ +1))

XC=α(|X1-X ₂ |)+(1-α)(|log(X ₂ +1)-log(X ₁ +1)|)

Wherein, _X1 and _X2 represent the target feature maps of two phases respectively; | _X1 - _X2 | represents the difference map generated based on the difference operator; |log( _X2 +1)-log( _X1 +1)| represents the difference map generated based on the logarithmic ratio operator; α is the weight coefficient, ranging from 0 to 1; _Xc represents the target difference map.

Step 430, model training and testing

The model was iteratively trained on NVIDIA M4000 using the deep learning framework Pytorch, and the convolution weights, biases and other parameters were updated using the AdaMax optimizer. The initial learning rate was set to 0.001, and the attenuation factors β1=0.9 and β2=0.999.

The performance of the trained model is tested using a test data set. The performance of the model built by the embodiment of the present invention is compared with the performance of the existing model to verify the effectiveness of the method provided by the embodiment of the present invention. The evaluation indicators include precision (Precision, P), recall (Recall, R), comprehensive evaluation index F1 and overall accuracy (OverallAccuracy, OA), and the formula is as follows:

Among them, TP, TN, FP, and FN are four types of judgments in the classic confusion matrix; TP represents the number of changed pixels predicted correctly; TN represents the number of unchanged pixels predicted correctly; FP represents the number of pixels predicted as changed; and FN represents the number of pixels predicted as unchanged.

In the change detection task, a higher precision indicates that fewer errors are generated in the prediction results, a larger recall indicates that more positive samples are detected, and F1 and overall accuracy measure the prediction results as a whole. The larger the value of the indicator, the better the prediction effect of the model. The indicator evaluation results are shown in Table 1. As can be seen from Table 1, compared with other methods, the comprehensive performance of the method provided by the present invention is significantly improved.

Table 1 Index evaluation results

Model Name Accuracy Recall R F1 Overall accuracy OA STANet 0.934 0.846 0.887 0.990 DSAMNet 0.928 0.869 0.898 0.987 SNUNet 0.950 0.906 0.927 0.983 Embodiments of the present invention 0.988 0.968 0.978 0.994

FIG6a is a target SAR image of the previous phase provided by an embodiment of the present invention; FIG6b is a target SAR image of the next phase provided by an embodiment of the present invention; FIG6c is a label map provided by an embodiment of the present invention; and FIG6d is a SAR image change detection result provided by an embodiment of the present invention. As shown in FIG6a to FIG6d, the method provided by the embodiment of the present invention can detect fine change information in SAR images, has strong detection capability, extracts accurate and complete information, suppresses false alarms and missed detections, and the prediction results are close to the manually labeled reference samples.

In summary, the SAR image change detection method provided by the embodiment of the present invention adopts a residual network to implement the first feature extraction module, and the second feature extraction module is composed of a channel attention module and a spatial attention module to build a change detection model. In the comparative experiment of a large-scale data set containing different scenes, the best prediction results and higher accuracy are achieved, proving that it has the ability to efficiently utilize the rich information and complex features in high-resolution SAR images and improve the convergence and optimization efficiency of the model, and can achieve accurate extraction of dynamic information of targets such as ships and buildings, and can improve the representation of ground feature and change detection effect. The SAR image change detection method provided by the embodiment of the present invention can not only overcome the shortcomings and defects of traditional methods and improve the efficiency of the algorithm, but also provide a reliable reference for applications in fields such as urban development and geological disaster monitoring, so as to more accurately and targetedly solve practical problems in multiple fields such as urban development and geological disaster monitoring.

FIG7 is a schematic diagram of the structure of a SAR image change detection device provided by an embodiment of the present invention. As shown in FIG7 , the SAR image change detection device 700 includes: a target image acquisition module 710, which is used to acquire target SAR images of two phases; a target feature map extraction module 720, which is used to extract features of the target SAR images of the two phases through a feature extraction network to obtain target feature maps of the two phases; the feature extraction network includes: a first feature extraction module, which is used to extract features of the target SAR images of each phase to obtain the first feature map of each phase; a second feature extraction module, including a channel attention module and a spatial attention module; wherein the channel attention module is used to process the first feature map of each phase based on the attention mechanism, determine the channel attention weight of the first feature map of each phase, and based on the channel attention mechanism, obtain the channel attention weight of the first feature map of each phase. The attention weight is used to weight the first feature map of each phase to obtain the second feature map of each phase; the spatial attention module is used to process the second feature map of each phase based on the attention mechanism, determine the spatial attention weight of the second feature map of each phase, and weight the second feature map of each phase based on the spatial attention weight to obtain the target feature map of each phase; the target difference map generation module 730 is used to perform difference analysis on the target feature maps of the two phases to generate a target difference map; the target image area determination module 740 is used to determine the target image area of the target SAR image of the latter phase in the target SAR image of the two phases relative to the target SAR image of the previous phase according to the target difference map.

In some embodiments, the first feature extraction module includes a first convolutional layer, multiple residual modules and a second convolutional layer connected in sequence; the first convolutional layer is used to perform dimensionality reduction processing on the target SAR image of each phase; each residual module is used to extract features from its own input features, and the input features of the first residual module are the output features of the first convolutional layer; the second convolutional layer is used to perform dimensionality increase processing on the output features of the last residual module to obtain a first feature map with the same dimension as the target SAR image of each phase.

In some embodiments, the channel attention module includes a first maximum pooling layer, a first average pooling layer, a third convolutional layer, a first fusion layer, a first sigmoid activation function and a first output layer, wherein the first maximum pooling layer and the first average pooling layer are connected in parallel to the input end of the third convolutional layer, and the third convolutional layer, the first fusion layer, the first sigmoid activation function and the first output layer are connected in sequence;

The first maximum pooling layer and the first average pooling layer are used to perform maximum pooling processing and average pooling processing on the first feature map of each phase in the spatial dimension, respectively;

The third convolutional layer is used to perform convolution processing on the output features of the first maximum pooling layer and the first average pooling layer respectively to obtain two channel attention weight matrices;

The first fusion layer is used to add the two channel attention weight matrices to obtain a total channel attention weight matrix;

The first sigmoid activation function is used to activate the total channel attention weight matrix to obtain the channel attention weight;

The first output layer is used to perform weighted processing on the first feature map of each time phase based on the channel attention weight to obtain the second feature map of each time phase.

In some embodiments, the spatial attention module includes a second maximum pooling layer, a second average pooling layer, a second fusion layer, a fourth convolutional layer, a second sigmoid activation function, and a second output layer, the second maximum pooling layer and the second average pooling layer are connected in parallel to the input end of the second fusion layer, and the second fusion layer, the fourth convolutional layer, and the second output layer are connected in sequence;

The second maximum pooling layer and the second average pooling layer are used to perform maximum pooling processing and average pooling processing on the second feature map of each phase in the channel dimension respectively;

The second fusion layer is used to perform vector concatenation on output features of the second maximum pooling layer and the second average pooling layer;

The fourth convolutional layer is used to perform convolution processing on the output features obtained by vector concatenation to obtain a spatial attention weight matrix;

The second sigmoid activation function is used to activate the spatial attention weight matrix to obtain the spatial attention weight;

The second output layer is used to perform weighted processing on the second feature map of each phase based on the spatial attention weight to obtain a target feature map of each phase.

In some embodiments, the target difference map generation module includes:

A difference map generation submodule, used to calculate the target feature maps of the two phases based on a difference operator and a logarithmic ratio operator, respectively, to obtain a difference map based on a difference operator and a difference map based on a logarithmic ratio operator;

The difference map fusion submodule is used to fuse the difference map based on the difference operator and the difference map based on the logarithmic ratio operator to generate the target difference map.

In some embodiments, the difference map fusion submodule is specifically used to:

A weighted fusion process is performed on the difference map based on the difference operator and the difference map based on the logarithmic ratio operator to generate the target difference map.

In some embodiments, the apparatus further comprises:

A sample image group acquisition module, used to acquire multiple sample SAR image groups, wherein each sample SAR image group includes sample SAR images of two time phases and sample labels, wherein the sample labels are used to indicate an actual image region in which the sample SAR image of the latter time phase of the sample SAR images of the two time phases changes relative to the sample SAR image of the previous time phase;

A sample feature map extraction module is used to extract features from sample SAR images of two time phases in each sample SAR image group through a feature extraction network to be trained, so as to obtain sample feature maps of two time phases;

A sample difference map generating module is used to perform difference analysis on the sample feature maps of the two phases corresponding to each sample SAR image group to generate a sample difference map corresponding to each sample SAR image group;

An image region determination module, for determining, according to a sample difference map corresponding to each sample SAR image group, an image region in which a sample SAR image of a later phase in each sample SAR image group has changed relative to a sample SAR image of a previous phase;

A loss information determination module, used to determine loss information according to an image area and a sample label of a sample SAR image of a later phase in each sample SAR image group that has changed relative to a sample SAR image of a previous phase;

A training module is used to train the feature extraction network to be trained according to the loss information.

In some embodiments, the sample image group acquisition module includes:

The original image acquisition submodule is used to acquire original SAR images of multiple phases;

A data enhancement submodule is used to perform data enhancement processing on the original SAR images of the multiple time phases to obtain sample SAR images of the multiple time phases;

The sample image group construction submodule is used to construct a plurality of sample SAR image groups according to the sample SAR images of the plurality of time phases.

In some embodiments, the loss information determination module is specifically used to:

According to the image area and sample label of the sample SAR image of the latter phase in each sample SAR image group that has changed relative to the sample SAR image of the previous phase, the set similarity loss information and the cross entropy loss information are determined respectively;

Mixed loss information is determined according to the set similarity loss information and the cross entropy loss information.

Fig. 8 is an electronic device according to an embodiment of the present invention. As shown in Fig. 8, the electronic device 800 includes: at least one processor 810, and a memory 820 connected to the at least one processor 810 for communication, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform a method.

Specifically, the above-mentioned memory 820 and processor 810 are connected together via a bus 830, and can be a general memory and processor, which are not specifically limited here. When the processor 810 runs the computer program stored in the memory 820, it can perform the various operations and functions described in combination with Figures 1 to 7 in the embodiments of the present invention.

In an embodiment of the present invention, the electronic device 800 may include but is not limited to: a personal computer, a server computer, a workstation, a desktop computer, a laptop computer, a notebook computer, a mobile computing device, a smart phone, a tablet computer, a personal digital assistant (PDA), a handheld device, a messaging device, a wearable computing device, and the like.

The embodiment of the present invention further provides a storage medium on which a computer program is stored, and when the program is executed by a processor, a method is implemented. For specific implementation, please refer to the method embodiment, which will not be repeated here. Specifically, a system or device equipped with a storage medium can be provided, on which a software program code that implements the functions of any of the above embodiments is stored, and the computer or processor of the system or device reads and executes the instructions stored in the storage medium. The program code read from the storage medium itself can implement the functions of any of the above embodiments, so the machine-readable code and the storage medium storing the machine-readable code constitute a part of the present invention.

Storage media include, but are not limited to, floppy disks, hard disks, magneto-optical disks, optical disks, magnetic tapes, non-volatile memory cards, and ROMs. Program codes may also be downloaded from a server computer or a cloud via a communication network.

It should be noted that, in the above-mentioned processes and system structures, not all steps and modules are necessary, and some steps and units can be ignored according to actual needs. The execution order of each step is not fixed and can be determined as needed. The device structure described in the above-mentioned embodiments can be a physical structure or a logical structure. A module or unit may be implemented by the same physical entity, a module or unit may be implemented by multiple physical entities respectively, and a module or unit may also be implemented by multiple components in multiple independent devices.

Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and the embodiments. It can be fully applied to various fields suitable for the embodiments of the present invention. For those familiar with the art, additional modifications can be easily realized. Therefore, without departing from the general concept defined by the claims and equivalent scopes, the embodiments of the present invention are not limited to the specific details and the legends shown and described here.

Claims (7)

1. A SAR image change detection method, comprising:

acquiring target SAR images of two time phases;

extracting features of the target SAR images of the two time phases through a feature extraction network to obtain target feature images of the two time phases;

the feature extraction network includes:

the first feature extraction module is used for carrying out feature extraction on the target SAR image of each time phase to obtain a first feature map of each time phase;

the second feature extraction module comprises a channel attention module and a space attention module; the channel attention module is used for processing the first feature map of each time phase based on an attention mechanism, determining the channel attention weight of the first feature map of each time phase, and weighting the first feature map of each time phase based on the channel attention weight to obtain the second feature map of each time phase; the spatial attention module is used for processing the second feature map of each time phase based on an attention mechanism, determining the spatial attention weight of the second feature map of each time phase, and carrying out weighting processing on the second feature map of each time phase based on the spatial attention weight to obtain a target feature map of each time phase;

Performing differential analysis on the target feature graphs of the two time phases to generate a target differential graph;

according to the target difference map, determining a target image area in which a target SAR image of a later time phase in the target SAR images of the two time phases changes relative to a target SAR image of a previous time phase;

the first feature extraction module comprises a first convolution layer, a plurality of residual error modules and a second convolution layer which are sequentially connected;

the first convolution layer is used for performing dimension reduction processing on the target SAR image of each time phase;

each residual error module is used for extracting the characteristics of the input characteristics of the residual error module, and the input characteristics of the first residual error module are the output characteristics of the first convolution layer;

the second convolution layer is used for carrying out dimension-lifting processing on the output characteristics of the last residual error module to obtain a first characteristic diagram with the same dimension as the target SAR image of each time phase;

performing differential analysis on the target feature graphs of the two time phases to generate a target differential graph, wherein the differential analysis comprises the following steps:

calculating the target feature graphs of the two time phases based on a difference operator and a logarithmic ratio operator respectively to obtain a difference graph based on the difference operator and a difference graph based on the logarithmic ratio operator;

Carrying out fusion processing on the difference graph based on the difference operator and the difference graph based on the logarithmic ratio operator to generate the target difference graph;

the fusing processing is carried out on the difference graph based on the difference operator and the difference graph based on the logarithmic ratio operator, and the target difference graph is generated, which comprises the following steps:

and carrying out weighted fusion processing on the difference graph based on the difference operator and the difference graph based on the logarithmic ratio operator to generate the target difference graph.

2. The SAR image change detection method of claim 1, wherein the channel attention module comprises a first max-pooling layer, a first average pooling layer, a third convolution layer, a first fusion layer, a first sigmoid activation function, and a first output layer, wherein the first max-pooling layer and the first average pooling layer are connected in parallel with an input end of the third convolution layer, and the third convolution layer, the first fusion layer, the first sigmoid activation function, and the first output layer are sequentially connected;

the first maximum pooling layer and the first average pooling layer are respectively used for carrying out maximum pooling processing and average pooling processing of space dimension on the first feature map of each time phase;

the third convolution layer is used for respectively carrying out convolution processing on the output characteristics of the first maximum pooling layer and the first average pooling layer so as to obtain two channel attention weight matrixes;

The first fusion layer is used for adding the attention weight matrixes of the two channels to obtain a total channel attention weight matrix;

the first sigmoid activation function is used for activating the total channel attention weight matrix to obtain channel attention weights;

the first output layer is used for carrying out weighting processing on the first characteristic map of each time phase based on the channel attention weight to obtain a second characteristic map of each time phase.

3. The SAR image change detection method of claim 1, wherein the spatial attention module comprises a second maximum pooling layer, a second averaging pooling layer, a second fusion layer, a fourth convolution layer, a second sigmoid activation function, and a second output layer, wherein the second maximum pooling layer and the second averaging pooling layer are connected in parallel with an input end of the second fusion layer, and the second fusion layer, the fourth convolution layer, and the second output layer are sequentially connected;

the second maximum pooling layer and the second average pooling layer are respectively used for carrying out maximum pooling treatment and average pooling treatment on the channel dimension of the second feature map of each time phase;

the second fusion layer is used for vector splicing of the output features of the second maximum pooling layer and the second average pooling layer;

The fourth convolution layer is used for carrying out convolution processing on the output characteristics obtained by vector splicing so as to obtain a space attention weight matrix;

the second sigmoid activation function is used for activating the spatial attention weight matrix to obtain spatial attention weight;

and the second output layer is used for carrying out weighting processing on the second characteristic map of each time phase based on the spatial attention weight to obtain a target characteristic map of each time phase.

4. The SAR image change detection method of claim 1, wherein the method comprises:

acquiring a plurality of sample SAR image groups, wherein each sample SAR image group comprises two time-phase sample SAR images and a sample label, and the sample label is used for indicating an actual image area in which a sample SAR image of a later time phase in the two time-phase sample SAR images is changed relative to a sample SAR image of a previous time phase;

carrying out feature extraction on the sample SAR images of two time phases in each sample SAR image group through a feature extraction network to be trained to obtain sample feature images of the two time phases;

performing differential analysis on the sample feature images of the two phases corresponding to each sample SAR image group to generate a sample differential image corresponding to each sample SAR image group;

According to the sample difference map corresponding to each sample SAR image group, determining an image area in which the sample SAR image of the next time phase in each sample SAR image group changes relative to the sample SAR image of the previous time phase;

determining loss information according to an image area and a sample label of a sample SAR image of a next time phase in each sample SAR image group, wherein the image area changes relative to the sample SAR image of a previous time phase;

and training the feature extraction network to be trained according to the loss information.

5. The SAR image change detection method of claim 4, wherein the acquiring a plurality of sample SAR image sets comprises:

acquiring original SAR images of a plurality of time phases;

performing data enhancement processing on the original SAR images of the multiple time phases to obtain sample SAR images of the multiple time phases;

and constructing a plurality of sample SAR image groups according to the sample SAR images of the plurality of phases.

6. The SAR image change detection method of claim 4, wherein the determining the loss information based on the image area and the sample label in which the sample SAR image of the subsequent phase in each sample SAR image group changes with respect to the sample SAR image of the previous phase comprises:

Respectively determining aggregate similarity loss information and cross entropy loss information according to an image area and a sample label of a sample SAR image of a later time phase in each sample SAR image group relative to a sample SAR image of a previous time phase;

and determining mixed loss information according to the aggregate similarity loss information and the cross entropy loss information.

7. A SAR image change detection apparatus, comprising:

the target image acquisition module is used for acquiring target SAR images of two time phases;

the target feature map extraction module is used for carrying out feature extraction on the target SAR images of the two time phases through a feature extraction network to obtain target feature maps of the two time phases;

the feature extraction network includes:

the first feature extraction module is used for carrying out feature extraction on the target SAR image of each time phase to obtain a first feature map of each time phase;

the second feature extraction module comprises a channel attention module and a space attention module; the channel attention module is used for processing the first feature map of each time phase based on an attention mechanism, determining the channel attention weight of the first feature map of each time phase, and weighting the first feature map of each time phase based on the channel attention weight to obtain the second feature map of each time phase; the spatial attention module is used for processing the second feature map of each time phase based on an attention mechanism, determining the spatial attention weight of the second feature map of each time phase, and carrying out weighting processing on the second feature map of each time phase based on the spatial attention weight to obtain a target feature map of each time phase;

The target difference map generation module is used for carrying out difference analysis on the target feature maps of the two time phases to generate a target difference map;

the target image area determining module is used for determining a target image area of which the target SAR image of the next time phase in the target SAR images of the two time phases is changed relative to the target SAR image of the previous time phase according to the target difference image;

the first feature extraction module comprises a first convolution layer, a plurality of residual error modules and a second convolution layer which are sequentially connected; the first convolution layer is used for performing dimension reduction processing on the target SAR image of each time phase; each residual error module is used for extracting the characteristics of the input characteristics of the residual error module, and the input characteristics of the first residual error module are the output characteristics of the first convolution layer; the second convolution layer is used for carrying out dimension-lifting processing on the output characteristics of the last residual error module to obtain a first characteristic diagram with the same dimension as the target SAR image of each time phase;

the target difference graph generation module comprises:

the difference map generation sub-module is used for calculating the target feature maps of the two time phases based on a difference operator and a logarithmic comparison operator respectively to obtain a difference map based on the difference operator and a difference map based on the logarithmic comparison operator;

The difference map fusion sub-module is used for carrying out fusion processing on the difference map based on the difference operator and the difference map based on the logarithmic ratio operator to generate the target difference map;

the difference map fusion submodule is specifically configured to:

and carrying out weighted fusion processing on the difference graph based on the difference operator and the difference graph based on the logarithmic ratio operator to generate the target difference graph.