CN117437530A - Synthetic aperture sonar twin matching identification method and system for small targets of interest - Google Patents

Abstract

The invention provides a synthetic aperture sonar interesting small target twin matching identification method and a system. The method comprises the following steps: processing the received echo data of the synthetic aperture sonar to obtain a real-time synthetic aperture sonar image, and performing coarse recognition by adopting a target detection method to obtain a small target image of interest; inputting the small target image of interest into a pre-established and trained twin matching model, and matching with different reference image groups one by one to obtain the similarity with each reference image group; determining a recognition result according to the sorting result of the similarity, and realizing refined recognition; the twin matching model adopts a Siam FC network structure, and the backbone network is an improved MobileNet V3 network. The invention provides an effective solution for the task of autonomous underwater small target identification of the SAS image under the condition of rare samples.

Description

Synthetic aperture sonar twin matching identification method and system for small targets of interest

Technical field

The invention relates to the field of underwater acoustic signal processing, and in particular to a synthetic aperture sonar twin matching identification method and system for small targets of interest.

Background technique

Synthetic Aperture Sonar (SAS) is a high-resolution underwater imaging sonar. Its basic principle is to use the movement of a small aperture matrix to form a virtual large aperture, thereby obtaining high resolution in the azimuth direction. Compared with ordinary side scan sonar, the most significant advantage of SAS is its higher azimuth resolution, and the theoretical resolution has nothing to do with the target distance and the acoustic frequency band used. The synthetic aperture sonar image target detection task plays an important role in the autonomous navigation and search of underwater unmanned platforms. Object detection can locate the location of small targets of interest, but is limited by the small number of samples and cannot achieve further precise identification.

Target matching is an important means of target recognition. Generally, target matching recognition can be divided into feature-based matching and convolutional neural network-based matching. Feature-based matching recognition is performed by extracting the features of the target in the template and the features in the image to be recognized for matching calculation. However, this type of method is not suitable for small underwater targets of interest. Because the scale of small targets in SAS images is small, this also leads to the lack of texture information in the targets, and accordingly it is difficult to extract effective feature descriptors. In addition, targets that are too small are easily affected by environmental noise, and feature points with sparse detection characteristics are very sensitive to noise, which makes it difficult for the feature point method to accurately describe small underwater targets of interest.

In recent years, due to the development of convolutional neural networks (CNN), some discriminative target matching algorithms based on CNN network models have received widespread attention from researchers due to their excellent matching performance. Some scholars have proposed target matching methods based on CNN networks. This method uses CNN to extract deep features of candidate samples and target templates, and achieves accurate matching between deep features by learning an accurate and robust similarity measure function. In recent years, Siamese convolutional neural networks have great advantages in learning similarity metric functions. Siam FC proposed by Bertinetto et al. has excellent performance in feature matching, but the real-time performance of the algorithm needs to be improved. Howard et al. proposed a lightweight convolutional neural network MobileNet V1. MobileNet V1 replaces standard convolution with depthwise separable convolution (DSC) to reduce the parameters and calculation amount of the model. Sandler et al. proposed MobileNet V2, an improved version of MobileNetV1. MobileNet V2 introduces a shortcut connections structure based on depth-separable convolution, and designs a new feature extraction module IRB (inverted residualblock). The new module adjusts the original "compression" and then "expansion" to "expansion" and then "compression". At the same time, in order to reduce the loss and damage of the activation function when converting high-dimensional information to low-dimensional information, the final convolution layer is The activation layer is changed from non-linear to linear. Howard et al. proposed an improved version of MobileNet V3 and feature extraction module IRB ⁺ based on MobileNet V1 and MobileNet V2. IRB ⁺ introduced the SE (squeeze and excitation) attention mechanism. SE first performs a squeeze operation on the features obtained by convolution to obtain the global features, then performs an excitation operation on the global features to obtain the weights of different features, and finally multiplies the features of the corresponding channels to obtain the final features. In essence, the SE component makes selections in the feature dimension, which can pay more attention to the features with the most information and suppress those unimportant features. IRB ⁺ has better feature extraction capabilities while maintaining a low computational load. However, IRB ⁺ capturing all channel dependencies is inefficient and unnecessary.

In summary, there is an urgent need for an identification method suitable for small targets of interest underwater to improve the accuracy and efficiency of refined identification of small targets under conditions of sparse samples.

Contents of the invention

The purpose of the present invention is to overcome the shortcomings of the existing technology and propose a synthetic aperture sonar twin matching identification method and system for small targets of interest.

In order to achieve the above objectives, the present invention proposes a synthetic aperture sonar twin matching identification method for small targets of interest, which method includes:

Process the received echo data of synthetic aperture sonar to obtain real-time synthetic aperture sonar images, and use the target detection method to perform rough identification to obtain images of small targets of interest;

Input the small target image of interest into the pre-established and trained twin matching model, match it with different reference image groups one by one, and obtain the similarity with each reference image group;

Determine the recognition result based on the ranking results of similarity to achieve refined recognition;

The twin matching model is an improved Siam FC network structure, and the backbone network is an improved MobileNet V3 network.

Preferably, the target detection method is used for rough identification to obtain images of small targets of interest; specifically including:

The target detection method is used for rough recognition, and the detection results are saved as fixed-size images to obtain small target images of interest.

Preferably, the improved MobileNet V3 network uses an EIRB module to replace the IRB ⁺ module in the original MobileNet V3; the EIRB module adopts an inverse residual network structure, including two branch networks, in which the upper branch network maintains the input characteristics. D remains unchanged, the lower branch network is used to extract and select features of small underwater targets of interest, and is added to the output features of the upper branch network; the lower branch network includes: expansion layer, channel Optional components and compression layers, where

The expansion layer is used to expand the input feature channels; the convolution kernel size is 1×1, and the number of convolution kernels is K times the input feature channels;

The channel selectable component is used to select channels containing important information by learning weights;

The compression layer is used to compress the feature channel into a quantity consistent with the input feature.

Preferably, the processing process of the EIRB module includes:

The input features D∈Φ ^H×H×M enter the two branch networks respectively, where H×H is the size of the input feature, M is the number of channels of the input feature, and Φ is the feature size;

For the lower branch network, the feature D _ex of the input feature D after passing through the expansion layer F _ex is:

D _ex =F _ex (D)

The output feature D _se after D _ex enters the channel selectable component is:

D _se =s·D _ex

s＝f _h (f ₃ (P _g (D _ex )))

In the formula, D _se ∈Φ ^H×H×(K×M) , s is the selection coefficient of the channel, s∈Φ ^1×(K×M) ; P _g is the global pooling function, Φ ^{1×(K×M )} is the output feature dimension, f ₃ is a one-dimensional convolution layer with a convolution kernel size of 3, f _h is the hard swish activation function,

The compression layer performs channel compression on D _se to obtain the channel-compressed feature D', and the output feature dimension is Φ ^H×H×M :

D′＝F _sq (D _se )

Add the input feature D that passes through the upper branch network and the output D' of the lower branch network to get the output feature for:

In the formula,

Preferably, the twin matching model also includes a similarity result calculation module for using cosine distance as a similarity measure function to calculate the difference between the output feature x of the small target image of interest to be identified and the output feature y of each reference image group. Average similarity sim(x,y):

Among them, N is the number of reference images in the reference image group, and y _is the output feature of the i-th reference image.

Preferably, the method also includes a training step of the twin matching model, specifically including:

Create a training set;

The training set data is sequentially input into the improved Siam FC network for model training. When the training requirements are met, the trained twin matching model is obtained.

On the other hand, the present invention proposes a synthetic aperture sonar twin matching and identification system for small targets of interest, which system includes:

The rough identification module is used to process the received echo data of synthetic aperture sonar to obtain real-time synthetic aperture sonar images, and uses the target detection method to perform rough identification to obtain images of small targets of interest;

The fine recognition module is used to input small target images of interest into the pre-established and trained twin matching model, and match them with different reference image groups one by one to achieve fine recognition and obtain the similarity of each reference image group; and

The result output module is used to determine the recognition result based on the ranking results of similarity;

The twin matching model is an improved Siam FC network structure, and the backbone network is an improved MobileNet V3 network.

Preferably, the twin matching model is deployed on an edge computing platform.

Compared with the existing technology, the advantages of the present invention are:

1. The present invention combines the target detection results of synthetic aperture sonar images with the feature matching results, and proposes a method for identifying small underwater targets of interest, which solves the problem of existing methods under the condition of sparse samples in a two-level coarse-fine and fine-grained manner. This method solves the problem of low identification accuracy of small targets of interest; this method first uses a target detection model to detect small targets of interest, and then uses the twin network to identify small targets of interest, and provides underwater small targets for SAS images under the condition of sparse samples. The autonomous target recognition task provides an effective means to solve it;

2. The improved MobileNet V3 network introduces the ECA attention mechanism. The core idea of ECA is to focus on the relationship between channels, rather than focusing on the relationship between different positions like traditional self-attention mechanisms (such as Transformer). , so the performance of the network can be significantly improved; in addition, the ECA attention mechanism uses a simple 1x1 convolution kernel to calculate the channel attention weight, so the additional parameters introduced are very limited;

3. The improved Siam FC network structure matches the image to be recognized with a set of reference images and calculates the average similarity, which greatly improves the reliability of the matching results, and the reference image group can be added and deleted on-site, improving the network rapid deployment capabilities.

Description of the drawings

Figure 1 is a synthetic aperture sonar underwater small target identification method and system implementation framework provided by the present invention;

Figure 2 is a schematic structural diagram of the improved feature extraction module;

Figure 3 is a schematic diagram of the structure of the twin matching model.

Detailed ways

In order to solve the above technical problems, the present invention proposes a method and system implementation for identifying small targets of interest in synthetic aperture sonar images, achieving refined identification of small targets of interest underwater; the method is a two-level coarse and fine method. The underwater target recognition method improves the recognition accuracy of underwater small targets of interest under the condition of sparse samples through the target detection model and twin matching network.

In order to achieve the above purpose, the present invention provides a method and system implementation for identifying small targets of interest in synthetic aperture sonar images, which is characterized in that the model includes: a small target of interest detection module, a data set production module, and a model training module. and feature matching module;

The data set production module is used to annotate and produce target classification data sets;

The model training module is used for parameter initialization, training and testing of the twin network;

The feature matching module is used for matching small target images of interest and target templates, and for refined identification of small targets of interest in real time.

The echo extraction module further includes: a synthetic aperture sonar image sub-module, a target detection sub-module and a small target of interest sub-module to be identified;

The synthetic aperture sonar image sub-module is used to process the received array element data to obtain real-time synthetic aperture sonar images;

The target detection sub-module is used to detect small targets of interest in synthetic aperture sonar images;

The small target of interest to be identified sub-module is used to save the detection results of the target detection sub-module as a fixed-size image;

Optionally, the data set production module sub-module further includes: data collection sub-module, data annotation sub-module and target classification data set production sub-module;

The data collection sub-module collects target images;

The data annotation sub-module annotates the collected images in combination with task requirements;

The target detection data set production sub-module randomly divides the data into a training set and a test set according to the standard target classification data set format.

Optionally, it is characterized in that the model training sub-module further includes: a parameter setting module, a twin network module and a model testing module.

The parameter setting sub-module is used to complete the initialization of parameters required for model training;

The twin network sub-module is used to achieve matching and identification of small targets of interest;

The model testing module sub-module is used to monitor the model training status in real time.

Optionally, the feature matching sub-module further includes: a target template sub-module, a twin network sub-module and a result output sub-module;

The target template submodule is used to manage and store underwater small target templates of interest;

The twin network sub-module is used to match the small target image of interest to be identified and the target template;

The result output sub-module is used to display and output the image similarity of the small target of interest to be identified.

The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and examples.

Example 1

The invention includes a point synthetic aperture sonar image preprocessing module, a data set production module, a model training module and a platform deployment module. The first step is to collect the target image, label the sample and generate an image classification data set; the second step is to initialize the training parameters, train the twin network, and evaluate the quality of the training results; the third step is to collect the detected water. Small targets of interest are matched with the target template to achieve refined recognition. The overall flow chart is shown in Figure 1, and the specific steps are as follows:

Step 1. Creation of target classification data set

Step 1-1. Collect target image samples;

Step 1-2, classify the image samples;

Steps 1-3: According to the format of the target classification data set, use the principle of random division to divide the annotated images into training sample sets and test sample sets.

Step 2. Model training

Step 2-1. Build the environment required for the training platform on the deep learning server, including open source software Anaconda, Pytorch, and Torchvision, etc., and set the model training initialization parameters, including batchsize, epoch, validation_epochs, etc.;

Step 2-2. Build an improved feature extraction module (EIRB, Efficient Inverted ResidualBlock), as shown in Figure 2. The EIRB module adopts an inverse residual network structure, that is, the channel is first "expanded" and then "compressed", and is composed of an expansion layer, a channel selectable component, and a compression layer. The expansion layer is responsible for the expansion of the input feature channel; the channel The selectable component selects channels containing important information by learning weights; the compression layer is responsible for compressing the feature channels into a number consistent with the input features.

For an arbitrary input feature D∈Φ ^H×H×M , where H×H is the size of the input feature and M is the number of channels of the input feature. The input feature D enters the two branch networks of the EIRB module: the lower branch is responsible for extracting and selecting features of small underwater targets of interest; the upper branch keeps the input feature D unchanged, and finally combines it with the output features of the top branch network Add up. For the lower branch network, the input feature D first passes through the expansion layer, and the mathematical expression of its output feature is:

D _ex ＝F _ex (D),D∈Φ ^H×H×M (1)

In the formula, D is the original input feature; D _ex is the feature after the expansion layer. The convolution kernel size of the expansion layer is 1×1, and the number of convolution kernels is K times the input feature channels, that is, K×M.

Then, the output feature D _ex is sent to the ECA channel selection component, and the mathematical expression of its output feature is:

D _se =s·D _ex (2)

s＝f _h (f ₃ (P _g (D _ex ))) (3)

In the formula, D _se is the channel feature after channel selection; s is the selection coefficient of the channel, s∈Φ ^1×(K×M) ; P _g () is the global pooling function, and the output feature dimension is Φ ^{1×(K ×M)} ; f ₃ is a one-dimensional convolution layer with a convolution kernel size of 3, and the output feature dimension is Φ ^1×(K×M) ; f _h is a hard swish activation function,

Next, channel compression is performed on D _se , and the mathematical expression is:

D'＝F _sq (D _se ),D _se ∈Φ ^H×H×(K×M) (4)

In the formula, D' is the feature after channel compression, and the output feature dimension is Φ ^H×H×M .

Through the above calculation, the mathematical expression of the output characteristics of the EIRB module can finally be obtained:

In the formula, is the output feature of the EIRB module,/> The feature map size is H×H, and the number of channels is M.

Step 2-3 build an improved MobileNet V3 network. The improved MobileNet V3 network is shown in Table 1, in which the input channel, intermediate channel, output channel, step size and activation function of each convolutional layer (RE represents the ReLU activation function, HS represents the H-Swich activation function) are the same as the original MobileNet V3 network Remain consistent; the difference is that the improved MobileNet V3 uses EIRB to replace the IRB ⁺ module in the original MobileNet V3.

Table 1 Improved MobileNet V3 network structure

Steps 2-4: Monitor the training process and test results of the twin network in real time, and stop training when the evaluation indicators meet the requirements.

Step 3. Refined identification of small targets of interest

Step 3-1. Build the operating environment required for target detection and twin matching on the edge computing platform.

Step 3-2. Target detection model deployment. Deploy the trained target detection model to the platform, and set the input data format and output data format.

Step 3-3. Twin matching network construction and deployment. Build a twin network structure as shown in Figure 3, and the backbone network is the improved MobileNet V3. If the template and the image to be matched come from the same category of positive sample pairs, the features extracted by the CNN will be very similar. On the contrary, if the template and the image to be matched come from negative sample pairs or samples of different categories, there will be a significant gap in the features extracted by the CNN. The twin convolutional neural network uses this method to improve the generalization ability, allowing it to achieve accurate and robust matching with only a small number of samples.

Step 3-4, similarity calculation and result output. When calculating similarity, cosine distance is used as the similarity measure function. The specific calculation formula is as follows:

where x is the output feature of the image to be recognized on the improved MobileNet V3 network; y _i is the output feature of the i-th reference image on the improved MobileNet V3 network; sim(x,y) is the group of the image to be recognized and the reference image the average similarity.

The following is a further explanation of the technical effects of the present invention in combination with simulation experiments:

The experimental hardware platform is i7-8750H, the memory is 32GB (16GB*2), the GPU is 2070s (8G), and the software environment is win10, python3.7, Torch1.3.1 and Torchvision0.4.2, etc. The size of the input image is 224 pixels * 224 pixels. In order to ensure the fairness of the algorithm, the reference image is a set of images, that is, the same type of target contains multiple images, and a total of three types of reference targets are set, namely non-target, possible target and suspected target. The experimental results are shown in Figure 1.

Table 2 Performance comparison of target detection models

non-target suspicious target suspected target Suspected target (to be identified) 0.8267 0.8879 0.9303 Suspicious target (to be identified) 0.8655 0.9192 0.8852 Non-target (to be identified) 0.8589 0.8412 0.8348

From the experimental results in Table 2, it can be found that the improved twin network in this paper can effectively make further judgments on the types of small targets of interest. For the suspected target (to be identified), the highest similarity is the reference suspected target group (0.9303), and the lowest similarity is the reference non-target group (0.8267); for the suspicious target (to be identified), the highest similarity For the reference suspicious target group (0.9192), the lowest similarity is the reference non-target group (0.8655); for non-targets (to be identified), the highest similarity is the reference non-target group (0.8589), and the lowest similarity is Reference suspected target group (0.8348). The three types of targets to be identified have the greatest similarity with similar targets.

Example 2

Embodiment 2 of the present invention proposes a synthetic aperture sonar twin matching and identification system for small targets of interest, which is implemented based on the method of Embodiment 1. The system includes:

The rough identification module is used to process the received echo data of synthetic aperture sonar to obtain real-time synthetic aperture sonar images, and use the target detection method to perform rough identification to obtain images of small targets of interest;

The fine recognition module is used to input the small target image of interest into the pre-established and trained twin matching model, and match it with different reference image groups one by one to achieve fine recognition and obtain the similarity with each reference image group;

The result output module is used to determine the recognition result based on the ranking results of similarity;

The twin matching model adopts an improved Siam FC network structure, and the backbone network is an improved MobileNetV3 network.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the embodiments, those of ordinary skill in the art will understand that modifications or equivalent substitutions may be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and they shall all be covered by the scope of the present invention. within the scope of the claims.

Claims (8)

1. A synthetic aperture sonar small target twinning matching identification method, the method comprising:

processing the received echo data of the synthetic aperture sonar to obtain a real-time synthetic aperture sonar image, and performing coarse recognition by adopting a target detection method to obtain a small target image of interest;

inputting the small target image of interest into a pre-established and trained twin matching model, and matching with different reference image groups one by one to obtain the similarity with each reference image group;

determining a recognition result according to the sorting result of the similarity, and realizing refined recognition;

the twin matching model is an improved Siam FC network structure, and the backbone network is an improved MobileNet V3 network.

2. The synthetic aperture sonar small target twinning matching identification method of claim 1, wherein the coarse identification is performed by using a target detection method to obtain a small target image of interest; the method specifically comprises the following steps:

and (3) performing coarse recognition by adopting a target detection method, and storing a detection result as an image with a fixed size to obtain a small target image of interest.

3. The method for identifying twin matching of small target of interest by using synthetic aperture sonar as recited in claim 3, wherein the improved MobileNet V3 network replaces IRB in original MobileNet V3 by EIRB module ⁺ A module; the EIRB module adopts an inverse residual error network structure and comprises two branch networks, wherein an upper branch network keeps an input characteristic D unchanged, and a lower branch network is used for extracting and selecting characteristics of a small underwater object of interest and adding the characteristics with output characteristics of the upper branch network; the lower leg network includes: an expansion layer, a channel selectable component, and a compression layer, wherein,

the expansion layer is used for expanding the input characteristic channel; the size of the convolution kernel is 1 multiplied by 1, and the number of the convolution kernels is K times of the number of the input characteristic channels;

the channel selectable component is used for selecting a channel containing important information through learning weights;

the compression layer is used for compressing the characteristic channels into the quantity consistent with the input characteristics.

4. A synthetic aperture sonar small target twinning match identification method as defined in claim 3, wherein the processing of the EIRB module comprises:

input features D ε Φ ^H×H×M Respectively entering two branch networks, wherein H multiplied by H is the size of an input feature, M is the number of channels of the input feature, and phi is the feature size;

for the lower branch network, the input feature D passes through the expansion layer F _ex Post feature D _ex The method comprises the following steps:

D _ex ＝F _ex (D)

D _ex output feature D after entering channel selectable component _se The method comprises the following steps:

D _se ＝s·D _ex

s＝f _h (f ₃ (P _g (D _ex )))

wherein D is _se ∈Φ ^H×H×(K×M) S is the selection coefficient of the channel, s ε Φ ^1×(K×M) ；P _g For global pooling function, Φ ^1×(K×M) To output feature dimensions, f ₃ One-dimensional convolution layer with convolution kernel size 3, f _h For the hard swish to activate the function,

from compressed layer pair D _se Channel compression is carried out to obtain a characteristic D' after channel compression, and the output characteristic dimension is phi ^H×H×M ：

D′＝F _sq (D _se )

Adding the input characteristic D passing through the upper branch network and the D' output by the lower branch network to obtain an output characteristicThe method comprises the following steps:

in the method, in the process of the invention,

5. a synthetic aperture sonar small target-of-interest twin matching recognition method according to claim 3, wherein the twin matching model further comprises a similarity result calculation module for calculating an average similarity sim (x, y) of the output feature x of the small target image of interest to be recognized and the output feature y of each reference image group using the cosine distance as a similarity metric function:

wherein N is the number of reference images of the reference image group, y _i Output characteristics of the i-th reference image.

6. The method for identifying twin matching of small objects of interest by using synthetic aperture sonar according to claim 1, wherein the method further comprises a training step of a twin matching model, and specifically comprises:

building a training set;

and sequentially inputting the training set data into an improved Siam FC network to perform model training, and obtaining a trained twin matching model when the training requirement is met.

7. A synthetic aperture sonar small target twinning match recognition system, the system comprising:

the coarse recognition module is used for processing the received echo data of the synthetic aperture sonar to obtain a real-time synthetic aperture sonar image, and performing coarse recognition by adopting a target detection method to obtain a small target image of interest;

the fine recognition module is used for inputting the small target image of interest into a pre-established and trained twin matching model, and matching the small target image with different reference image groups one by one to realize fine recognition and obtain the similarity with each reference image group; and

the result output module is used for determining a recognition result according to the sorting result of the similarity;

the twin matching model is an improved Siam FC network structure, and the backbone network is an improved MobileNet V3 network.

8. The synthetic aperture sonar small target twinning match recognition system of claim 8, wherein the twinning match model is deployed on an edge computing platform.