CN116129207B - A Method for Image Data Processing with Multi-Scale Channel Attention - Google Patents

Abstract

The invention discloses an image data processing method with multi-scale channel attention. By extracting the global features and local features in the input data, the convolutional neural network pays more attention to the overall information and local detail features of the input data, thereby alleviating Target aggregation and target occlusion problems in complex scenes.

Description

A Method for Image Data Processing with Multi-Scale Channel Attention

technical field

The invention relates to the field of computer vision, in particular to an image data processing method of multi-scale channel attention.

Background technique

The channel attention mechanism can significantly improve the expressiveness and generalization ability of the model, and has a low computational cost, and is easy to be integrated into the existing convolutional neural network structure. Due to these advantages, the channel attention mechanism has been widely used in deep learning applications such as image classification, object detection, and semantic segmentation.

The essence of the channel attention mechanism is to weight and average the features of different channels, so as to obtain more abundant, stable and reliable feature expressions.

The existing channel attention includes SE, ECA, CA, etc. These channel attention only focus on the detailed information in a certain local feature or the semantic information in the global feature, but do not pay attention to the detailed information and semantic information at the same time, resulting in insufficient richness. Feature representation of the channel dimension.

Contents of the invention

The purpose of the present invention is to provide a multi-scale channel attention image data processing method.

The problem to be solved by the present invention is:

A multi-scale channel attention image data processing method is proposed to extract the global features and local features in the input data, so that the convolutional neural network pays more attention to the overall information and local detail features of the input data, thereby alleviating the occurrence of complex scenes. The problem of target aggregation and target occlusion.

A technical scheme adopted by an image data processing method of multi-scale channel attention is as follows:

A Method for Image Data Processing with Multi-Scale Channel Attention

S21: Digitize the input data (original image or feature map), convert the extracted features into digits, store them in a tensor matrix, and speed up the convergence of the convolutional neural network after normalization;

S22: Use the method of combining the global channel attention mechanism and the local channel attention mechanism to perform feature extraction and feature fusion on the input data;

S22.1: In the global channel attention mechanism, global average pooling, one-dimensional convolutional layer with adaptive selection of convolution kernel size and Sigmoid activation function are used. Global channel attention can be achieved through global average pooling of feature maps and Element-by-element transformation, adaptively adjust the weights of different channels, so that the model can focus on more important features, improve the classification performance and robustness of the model, and the calculation formula of the global average is:

in Indicates the global average pooling result, /> is the input image, its size is W×H×C, W, H and C represent the width, height and channel of the input image respectively, and i and j represent the pixel position on the width and height respectively;

The calculation formula for adaptive selection is:

Where k represents the convolution kernel size of one-dimensional convolution, C represents the number of channels, Indicates that k can only take odd numbers, /> and b are used to change the ratio between C and k, in the present invention /> and b take 2 and 1 respectively;

The Sigmoid activation function is also known as the S-type growth curve, and the calculation formula is:

where x is the input;

S22.2: In the local channel attention mechanism, a multi-layer perceptron MLP implemented by two-dimensional convolution is used to extract local features. The MLP architecture is two two-dimensional convolutions with a convolution kernel size of 1 and an intermediate The ReLU function activation of the input data only changes the number of channels after two-dimensional convolution. The number of output channels of the first convolution operation is one-sixteenth of the number of input channels, and the number of output channels of the second convolution operation Consistent with the number of embedding position channels, local channel attention can help the model better capture the local information in the input features. The ReLU function keeps only positive elements and discards all negative elements by setting the corresponding activity value to 0;

S22.3: Fuse the output of global attention and local attention, and use the Sigmoid function to activate the data to obtain the final attention weight, and then multiply the activated data with the input data pixel by pixel.

S22.4: Compress through the Sigmoid function, which compresses any input to a value in the interval (0, 1) according to its range to ensure normalization;

S22.5: Multiply the input data and the activated data pixel by pixel to complete the weighting operation of different positions of the input data, so as to pay more attention to global features and local features.

Further, the above-mentioned input data only changes the number of channels after the two-dimensional convolution in the above-mentioned step S24, and in the entire MLP architecture, the channel of the input data is estimated in a way of shrinking first and then expanding, The shrinkage coefficient is r, the feature scale after shrinkage is H×W×C/r, and the ReLU activation function is used, and the feature scale after expansion is H×W×C.

Further, in the above steps S23 and S24, the global features and local features in the input data are extracted through the global channel attention mechanism and the local channel attention mechanism respectively, and the output of the global attention and local attention in the step S26 is The fusion operation is to perform feature fusion on different features, so that the convolutional neural network pays more attention to the overall information of the input data and local detail features, thereby alleviating the problem of target aggregation and target occlusion in complex scenes.

Beneficial effects of the present invention: the problems of low detection accuracy and high missed detection rate caused by the large number of small target detection and serious occlusion in complex scenes can be further alleviated by the image data processing method of multi-scale channel attention , the image data processing method of multi-scale channel attention extracts the global features and local features in the data and performs feature fusion on different features, so that the convolutional neural network pays more attention to the overall information of the input data and local detail features, thereby alleviating Target aggregation and target occlusion problems in complex scenes.

Description of drawings

Fig. 1 is a schematic diagram of an image data processing method of multi-scale channel attention in the present invention;

Fig. 2 is a linear schematic diagram of ReLU function modification in the present invention;

Fig. 3 is a schematic diagram of normalization of sigmoid function data in the present invention.

Implementation

The present invention will be further clearly and completely described below in conjunction with the accompanying drawings, but the protection scope of the present invention is not limited thereto.

Example

As shown in Figures 1 to 3, an image data processing method for multi-scale channel attention includes the following steps:

S21: Digitize the input data (original image or feature map), convert the extracted features into digits, store them in a tensor matrix, and speed up the convergence of the convolutional neural network after normalization;

S22: Use the method of combining the global channel attention mechanism and the local channel attention mechanism, as shown in Figure 1, to perform feature extraction and feature fusion on the input data;

S22.1: In the global channel attention mechanism, use global average pooling, one-dimensional convolution layer with adaptive selection of convolution kernel size and Sigmoid activation function, as shown in the left column of Figure 1, the global channel attention can be passed to The global average pooling and element-by-element transformation of the feature map adaptively adjust the weights of different channels so that the model can focus on more important features and improve the classification performance and robustness of the model. The calculation formula of the global average pooling is:

Where y represents the global average pooling result, X is the input image, its size is W×H×C, W, H and C represent the width, height and channel of the input image respectively, and i and j represent the pixels on the width and height respectively point position;

The calculation formula for adaptive selection is:

Where k represents the convolution kernel size of one-dimensional convolution, C represents the number of channels, Indicates that k can only take odd numbers, where and b are used to change the ratio between C and k, in this embodiment /> and b take 2 and 1 respectively;

The Sigmoid activation function is also called the S-type growth curve, as shown in Figure 3, the calculation formula is:

where x is the input;

S22.2: In the local channel attention mechanism, a multi-layer perceptron MLP implemented by two-dimensional convolution is used to extract local features. The MLP architecture is two two-dimensional convolutions with a convolution kernel size of 1 and an intermediate The activation of the ReLU function, the activation of the ReLU function makes the output of some neurons 0, which reduces the interdependence of parameters and alleviates the occurrence of over-fitting problems. The input data only changes the number of channels after two-dimensional convolution. The number of output channels of the first convolution operation is one-sixteenth of the number of input channels, the number of output channels of the second convolution operation is consistent with the number of embedding position channels, and local channel attention can help the model better capture the input The local information in the feature, as shown in the right column of Figure 1, where the ReLU function sets the corresponding activity value to 0, as shown in Figure 2, only retaining positive elements and discarding all negative elements;

S22.3: Fuse the output of global attention and local attention, and use the Sigmoid function to activate the data to obtain the final attention weight, and then multiply the activated data with the input data pixel by pixel.

S22.4: Compress through the Sigmoid function, which compresses any input to a certain value in the interval (0, 1) according to its range to ensure normalization, as shown in Figure 1;

S22.5: Multiply the input data and the activated data pixel by pixel to complete the weighting operation of different positions of the input data, so as to pay more attention to the global features and local features, as shown in Figure 1.

The above-mentioned input data only changes the number of channels after the two-dimensional convolution in the above-mentioned step S24, and in the entire MLP architecture, the channels of the input data are first contracted and then expanded to estimate the attention between channels, where the contraction The coefficient is r, the feature scale after shrinkage is H×W×C/r, and the ReLU activation function is used, and the feature scale after expansion is H×W×C.

In the above steps S23 and S24, the global features and local features in the input data are extracted through the global channel attention mechanism and the local channel attention mechanism respectively, and in the step S26, the fusion operation is performed on the output of the global attention and the local attention That is, feature fusion is performed on different features, so that the convolutional neural network pays more attention to the overall information of the input data and local detail features, thereby alleviating the problems of target aggregation and target occlusion in complex scenes.

The embodiments of the present invention disclose preferred embodiments, but are not limited thereto. Those skilled in the art can easily comprehend the spirit of the present invention based on the above-mentioned embodiments, and make different extensions and changes. But as long as it does not deviate from the spirit of the present invention, it is within the protection scope of the present invention.

Claims (2)

1. an image data processing method of multi-scale channel attention, is characterized in that, comprises the following steps: S21: Digitize the input data, that is, the original image or feature map, convert the extracted features into digits, store them in a tensor matrix, and speed up the convergence of the convolutional neural network after normalization; S22: Use the method of combining the global channel attention mechanism and the local channel attention mechanism to perform feature extraction and feature fusion on the input data; S22.1: Use global average pooling, a one-dimensional convolutional layer with adaptive selection of the convolution kernel size, and a Sigmoid activation function in the global channel attention mechanism, wherein the calculation formula of the global average pooling process is: , where /> Indicates the global average pooling result, /> is the input image, its size is W×H×C, W, H and C represent the width, height and channel of the input image respectively, and i and j represent the pixel position on the width and height respectively; The calculation formula for adaptive selection is: , where k represents the convolution kernel size of one-dimensional convolution, C represents the number of channels, /> Indicates that k can only take odd numbers, /> and b are used to vary the ratio between C and k; The Sigmoid activation function is also known as the S-type growth curve, and the calculation formula is: , where x is the input; S22.2: In the local channel attention mechanism, a multi-layer perceptron MLP implemented by two-dimensional convolution is used to extract local features. The MLP architecture is two two-dimensional convolutions with a convolution kernel size of 1 and an intermediate The ReLU function activation of the input data only changes the number of channels after two-dimensional convolution. The number of output channels of the first convolution operation is one-sixteenth of the number of input channels, and the number of output channels of the second convolution operation Consistent with the number of embedding position channels, the ReLU function keeps only positive elements and discards all negative elements by setting the corresponding activity value to 0; S22.3: Fuse the output of global attention and local attention, and use the Sigmoid function to activate the data to obtain the final attention weight, and then multiply the activated data with the input data pixel by pixel; S22.4: Compress through the Sigmoid function, which compresses any input to a value in the interval (0, 1) according to its range to ensure normalization; S22.5: Multiply the input data and the activated data pixel by pixel to complete the weighting operation of different positions of the input data, so as to pay more attention to global features and local features. 2. the image data processing method of a kind of multi-scale channel attention according to claim 1, is characterized in that, After the input data passes through the two-dimensional convolution in the step S24, only the number of channels is changed, and in the entire MLP architecture, the channels of the input data are first contracted and then expanded to estimate the attention between channels, wherein The shrinkage coefficient of is r, the feature scale after shrinkage is H×W×C/r, and the ReLU activation function is used, and the feature scale after expansion is H×W×C.