CN116306886A - Channel Attention-Based Convolutional Neural Network Pruning Approach for Text Detection in Natural Scenes - Google Patents

Abstract

The invention provides a convolutional neural network pruning method based on channel attention for natural scene text detection, which uses a structured pruning method to entirely remove unimportant convolutional kernels and can realize parameter compression and calculation acceleration on any equipment. In order to ensure the accuracy of target detection, the importance of each convolution channel in the network is evaluated by designing a channel attention scoring module, and only channels with importance scores higher than a preset threshold value are reserved to participate in calculation in training. Because the amplitude of the output characteristic corresponding to the channel to be cut is restrained in the output characteristic spectrum of the scoring module, namely, the redundant parameter is cut in the training process, only the reserved parameter participates in the calculation of the next layer network, and the calculated amount is reduced. The invention avoids the manual design of evaluation indexes; the channel attention scoring module has the characteristics of plug and play and has strong adaptability; non-important parameters in the training process are suppressed, and the time required for fine-tuning the pruning network is greatly reduced.

Description

Channel Attention-based Convolutional Neural Network Clipping for Text Detection in Natural Scenes stick method

technical field

The invention relates to computer vision deep learning technology, in particular to a channel attention (Channel Attention)-based Convolutional Neural Networks (CNN) pruning technology for natural scene text detection.

Background technique

Convolutional neural network is an algorithm model widely used in the field of computer vision. The emergence of CNN has effectively improved the accuracy of text detection tasks in natural scenes. Huge network architecture design can usually significantly improve the accuracy, but it also brings higher memory usage and computing power consumption. This characteristic makes it difficult for CNNs to be deployed in resource-constrained environments such as mobile phones and smart glasses. Using model pruning technology to reduce CNN model parameters and calculation load is of great significance for the deployment of CNN algorithms on embedded devices.

Model pruning algorithms usually pre-train a parameter-dense model to achieve high accuracy, then prune unimportant weights to eliminate redundancy, and finally fine-tune the pruned model to restore performance. The core of the pruning algorithm is to design an effective evaluation system and indicators to evaluate the importance of network parameters and guide the deletion of redundant parameters. Common parameter importance evaluation methods include weight magnitude evaluation, such as L1 norm, KL divergence evaluation, simulated annealing, importance sampling and other direct evaluation methods. A commonly used indirect evaluation method is to use the weight of the gradient of the loss function to construct a Taylor series approximation, so that the value of the loss function changes little after a certain weight is zero, indicating that the weight is not important. In addition, it is also an effective evaluation method to evaluate the output feature map corresponding to the channel instead of analyzing the importance of the convolutional channel, for example, the matrix average rank of the output feature spectrum is used as the pruning basis.

Early pruning methods delete a single parameter from the model to obtain a sparse structure to achieve parameter compression, that is, such unstructured pruning methods usually set unimportant parameters to zero, and the number of parameters in the model has not decreased. Want to obtain The actual inference acceleration effect requires the support of special hardware devices, and it is difficult to achieve acceleration effects on existing hardware devices. Some structured pruning methods manually design the importance evaluation method, the evaluation index is usually relatively single, and the importance expression ability is insufficient; for different networks and different tasks, it may be necessary to redesign effective evaluation methods. In addition, the three-step pruning method of pre-training-pruning-fine-tuning generally has a long training time. After the model is pruned, the target detection accuracy will be greatly reduced. It takes time to re-fine-tune the pruned model to recover precision.

Contents of the invention

The problem to be solved by the present invention is to provide a method suitable for natural scene text detection tasks, which can greatly save the time required for pruning and fine-tuning, and speed up the establishment of a convolutional neural network for natural scene text detection.

The technical scheme that the present invention adopts for solving the above-mentioned technical problems is, the channel attention-based convolutional neural network pruning method for natural scene text detection comprises the following steps:

Step 1: Pre-train a parameter-dense convolutional neural network for natural scene text detection;

Step 2: Add a channel attention scoring module in the convolutional neural network;

Step 3: Use the natural scene text detection training set to optimize the parameters of the channel attention scoring module until the predetermined number of iterations is reached;

Step 4: Use the natural scene text detection test set to obtain the importance score of each channel in the channel attention scoring module;

Step 5: Statistically score the channel importance scores under all samples in the test set, and calculate the mean value of the importance scores;

Step 6: Determine the threshold;

Step 7: The importance of the convolution kernel in the convolutional neural network is measured by the mean value of the importance score, and the convolution kernel corresponding to the channel with the mean value of the importance score in the convolutional neural network is less than the threshold value is pruned;

Step 8: Use the pruned convolutional neural network as a natural scene text detection model;

Step 9: Fine-tune the natural scene text detection model using the natural scene text detection training set.

Specifically, the channel attention scoring module includes a global average pooling layer, a global maximum pooling layer, a convolutional layer, an activation function module, and a scaling module. The working steps include:

1) The global average pooling layer receives the input image feature spectrum F, and outputs the feature average value Avg in each channel to the convolutional layer; the global maximum pooling layer receives the input image feature spectrum F, and outputs the feature maximum value Max in each channel to Convolution layer;

2) The convolutional layer sums the Avg and Max of each channel and then re-encodes to obtain the attention score S, and outputs the attention score S of each channel to the activation function;

3) The activation function remaps the attention score S of each channel to obtain the importance score S' of each channel;

4) The scaling module uses the importance score S' of each channel to weight the image feature spectrum F, and uses the weighted result as the output feature F'=F+F×S' of the channel attention scoring module.

The present invention uses a structured pruning method to completely eliminate unimportant convolution kernels, and can realize parameter compression and calculation acceleration on any device. In order to ensure the accuracy of text target detection, a channel attention scoring module is designed to evaluate the importance of each convolution channel in the network, and only channels with an importance score higher than a predetermined threshold are reserved to participate in the calculation during training. Since the output feature spectrum of the scoring module suppresses the amplitude of the output feature corresponding to the channel to be trimmed, that is, the redundant parameters have been trimmed during the training process, only the retained parameters will participate in the calculation of the next layer of the network, Reduce the amount of calculation.

The beneficial effect of the present invention is that structural pruning is carried out with the convolution kernel as the smallest unit, and the pruned model can obtain the effect of memory compression and reasoning acceleration on the existing embedded device; use the channel attention mechanism to let the network learn Which output features are not important, avoiding the manual design of evaluation indicators; the channel attention scoring module has a plug-and-play feature, which is suitable for pruning of any convolutional neural network; non-important parameters have been suppressed during the training process, greatly reducing The time required to fine-tune the pruned network.

Description of drawings

Figure 1: Structural diagram of channel attention scoring module.

Figure 2: Schematic diagram of the network structure of the DBNet pruning experiment example.

Detailed ways

First, the channel attention scoring module used in the embodiment is described. The specific structural design is shown in Figure 1, including the global average pooling layer Avg Pooling, the global maximum pooling layer Max Pooling, convolutional layer, tanh activation function module and scaling Module Scale; the convolutional layer uses two layers of convolutional layers Conv with a size of 1×1 connected by the activation function ReLU.

The workflow steps of the channel attention scoring module are as follows:

Input the image feature spectrum F with the size of H×W×C into the global average pooling layer Avg Pooling to obtain the feature average vector Avg in the channel with the size of 1×1×C;

Input the image feature spectrum F with the size of H×W×C into the global maximum pooling layer Max Pooling to obtain the feature maximum value vector Max in the channel with the size of 1×1×C;

Use two convolutional layers of size 1×1 to re-encode (Avg+Max) to obtain the attention score S;

The tanh activation function module uses the tanh activation function and two hyperparameters α, β to remap the attention score S, and adjusts the scoring range to (-α+β,α+β), and obtains the score S'=α×tanh(S )+β;

The scaling module Scale uses S' to scale the parameters of each channel of the input feature spectrum to obtain the output feature spectrum F'=F+F×S' of the channel attention scoring module.

The hyperparameters α and β in the above process are usually 0.5 and -0.5, and the scoring interval is limited to (-1,0). If the score of a certain convolution kernel is -1, the output feature of the channel is F'=F-F=0, and the importance score can reflect the pruning intention of the channel attention scoring module.

The embodiment uses the PyTorch framework to perform DBNet algorithm pruning on the TITAN X server. DBNet is a text box detection algorithm, and the specific structure is shown in the dashed box in Figure 2. The network extracts four feature spectra of different scales from the network skeleton backbone, and forms a shared feature spectrum feature after fusion. The prediction head uses the parameters in the feature to predict the position information of the text box in the input image. In the embodiment, a channel attention scoring module (CA-module) is added after the feature, and the module outputs a pruned feature spectrum feature', and the prediction head uses the pruned feature' to locate the text box during the training process. The specific steps of the DBNet pruning experiment are as follows:

Step 1: Pre-train a high-precision DBNet network model;

Step 2: Add a channel attention scoring module after the feature in the model;

Step 3: Freeze the original network parameters and optimize the parameters of the channel attention scoring module until the predetermined number of iterations is reached;

Step 4: Use the test set to obtain the importance scores of each channel under all test samples, and calculate the mean value of the importance scores;

Step 5: Sort the mean value of the importance score from small to large, calculate the product of the pruning rate and the number of channels, round it down and record it as N, and the importance score of the Nth ranking is the pruning threshold;

Step 6: Taking the convolution kernel as the unit, pruning all the convolution kernels of the channel whose scoring average is less than the threshold;

Step 7: Copy the reserved parameters to form the DBNet model after pruning;

Step 8: Fine-tune the pruned DBNet model to restore accuracy, and form a model that is finally used for text box detection Pre-Head.

In terms of network configuration and training details, the hyperparameter α of the channel attention scoring module is set to 0.5, and β is set to -0.5; the optimization algorithm for training the network uses the stochastic gradient descent method, the batch data size is 16, and the initial learning rate is 0.001. Exponential decay strategy, the decay rate is 0.998, the decay step size is 500 iterations, and the total number of iterations is 20K; the pruning rate is 0.5.

Claims (4)

1. the convolutional neural network pruning method based on channel attention for natural scene text detection, is characterized in that, comprises the following steps: Step 1: Pre-train a parameter-dense convolutional neural network for natural scene text detection; Step 2: Add a channel attention scoring module in the convolutional neural network; Step 3: Use the natural scene text detection training set to optimize the parameters of the channel attention scoring module until the predetermined number of iterations is reached; Step 4: Use the natural scene text detection test set to obtain the importance score of each channel in the channel attention scoring module; Step 5: Calculate the channel importance scores under all samples in the test set, and calculate the mean value of the importance scores; Step 6: Determine the threshold; Step 7: Prune the convolution kernel corresponding to the channel whose importance mean score in the convolutional neural network is less than the threshold; Step 8: Use the pruned convolutional neural network as a natural scene text detection model; Step 9: Fine-tune the natural scene text detection model using the natural scene text detection training set. 2. method as claimed in claim 1, is characterized in that, channel attention scoring module comprises global average pooling layer, global maximum pooling layer, convolutional layer, activation function module and scaling module, and its work step comprises: 1) The global average pooling layer receives the input image feature spectrum F, and outputs the feature average value Avg in each channel to the convolutional layer; the global maximum pooling layer receives the input image feature spectrum F, and outputs the feature maximum value Max in each channel to Convolution layer; 2) The convolutional layer sums the Avg and Max of each channel and then re-encodes to obtain the attention score S, and outputs the attention score S of each channel to the activation function; 3) The activation function remaps the attention score S of each channel to obtain the importance score S' of each channel; 4) The scaling module uses the importance score S' of each channel to weight the image feature spectrum F, and uses the weighted result as the output feature F'=F+F×S' of the channel attention scoring module. 3. The method according to claim 1, wherein the calculation method of the mean value of the importance score is: output all sample images of the natural scene text detection test set to the convolutional neural network respectively, and count the importance of the channel attention module under different samples. scores, and then calculate their average. 4. The method according to claim 2, characterized in that, in step 3), the attention score S is remapped using the tanh activation function and two hyperparameters α, β, and the range of the scoring interval is adjusted to (-α+β, α +β), so as to obtain the importance score S': S'=α×tanh(S)+β.

Images