CN116266387A - YOLOV4 image recognition algorithm and system based on reparameterized residual structure and coordinate attention mechanism - Google Patents

Abstract

The invention discloses a YOLOV4 image recognition algorithm and system based on a reparameterized residual structure and a coordinate attention mechanism. Among them, the algorithm includes the following steps: obtain the image set to be recognized; use the Mosaic data enhancement method to process the input training image set, and use the K-means++ clustering algorithm to calculate the initial aiming frame of the training image set; The YOLOV4 model of the difference structure and coordinate attention mechanism, the training image set and the verification image set are used for model training to generate a feature extraction model; the image set to be recognized is input into the model to obtain the image recognition result. The algorithm uses the proposed YOLOV4 model based on the reparameterized residual structure and coordinate attention mechanism to perform target recognition on images, which can realize the classification and positioning of targets in complex backgrounds, and the model is robust, which can effectively improve target recognition accuracy and speed.

Description

Image Recognition of YOLOV4 Based on Reparameterized Residual Structure and Coordinate Attention Mechanism different algorithms and systems

technical field

The invention relates to the fields of image processing and pattern recognition, in particular to a YOLOV4 image recognition algorithm and system based on a reparameterized residual structure and a coordinate attention mechanism.

Background technique

Target recognition plays an important role in the field of image processing, which is to locate and classify targets at the same time. Traditional target recognition algorithms include V-J (Viola-Jones) detection algorithm, Histogram of Oriented Gradient (HOG) detection algorithm and Deformable Parts Model (DPM) algorithm. The V-J detection algorithm is mainly used for face detection. The main principle is to search for haar features by sliding the window on the input image. The HOG detection algorithm constructs the corresponding feature table by extracting the gradient, and constructs a histogram for each grid of the image. DPM is the most successful traditional detection model before the rise of deep learning algorithms. However, in the case of complex backgrounds and multiple detection targets, traditional algorithms do not have advantages in speed and accuracy. Therefore, in recent years, the hot research direction of researchers is the target detection algorithm based on deep learning. Compared with traditional learning algorithms, deep learning algorithms have faster recognition speed and more stable recognition results. The deep learning algorithm of target recognition is divided into one-stage algorithm and two-stage algorithm. Typical representatives of one-stage algorithm include YOLO series and SSD. Typical representatives of the two-stage algorithm include R-CNN, Fast R-CNN and Faster R-CNN, etc. The target recognition speed of the second-stage algorithm is slow and the accuracy is high, and the target recognition speed of the first-stage algorithm is fast and the accuracy is low. However, in the past two years, the typical representative YOLO series of the first-stage algorithm has achieved great progress in the accuracy and speed of target recognition. Very well balanced effect.

The following is a brief introduction to the series of algorithms:

In 2016, Joseph Redmon and others proposed the first generation model of the YOLO series, YOLOV1. The specific principle of the model is to divide the input picture into n×n grids, and each grid predicts x candidate boxes and object categories. , the detection speed of this model is very fast, and it can process 45 images per second, but the detection accuracy is poor. In 2017, the second-generation model YOLOV2 added an average pooling layer and a BN layer to the backbone network on the basis of the first-generation model to make the model converge faster, and introduced a frame-seeing mechanism that does not directly predict coordinate values , the location of the target can be obtained relatively accurately through the offset and confidence of the coordinates. In 2018, the model YOLOV3 was obtained by improving the second-generation model YOLOV2. The model selected three aiming frames of different sizes to achieve accurate detection of targets of different sizes, and selected multi-label classification in the classification layer. , you can judge whether it is true or not for each type, so as to achieve higher precision. In 2020, the fourth-generation model YOLOV4 was launched. Its backbone network chooses CSPDarknet53, adding the SPP module and the feature fusion structure of FPN+PAN. Such improvements not only give the model an advantage in speed, but also in terms of detection accuracy. Other models also have significant advantages. But the model still needs to be improved to achieve better predictions.

Contents of the invention

The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

For this reason, the first purpose of the present invention is to propose a YOLOV4 image recognition algorithm based on a reparameterized residual structure and coordinate attention mechanism, which is suitable for target recognition in complex backgrounds, and has a significant speed increase in the reasoning stage. promote.

Another object of the present invention is to propose a YOLOV4 image recognition system based on a reparameterized residual structure and a coordinate attention mechanism.

In order to achieve the above two purposes, the present invention proposes a YOLOV4 image recognition algorithm based on a reparameterized residual structure and a coordinate attention mechanism in the first aspect, which includes the following steps: input the image set to be recognized; Perform data enhancement on the input training image set, and use the K-means++ clustering algorithm to calculate the initial aiming frame of the training image set; obtain the YOLOV4 model of the reparameterized residual structure and coordinate attention mechanism, which is based on YOLOV4. Adding spatial information in the X and Y directions in the feature extraction stage can improve the accuracy of the model. In the extraction stage of complex features, a re-parameterized residual structure is added to improve the model reasoning speed; according to the re-parameter The YOLOV4 model of the residual structure and the coordinate attention mechanism and the training image set and the verification image set are used for model training to generate the YOLOV4 recognition model of the reparameterized residual structure and the coordinate attention mechanism; according to the image to be identified The image recognition results are obtained through the YOLOV4 recognition model of the reparameterized residual structure and coordinate attention mechanism.

The YOLOV4 image recognition algorithm based on the reparameterized residual structure and coordinate attention mechanism of the embodiment of the present invention can obtain the YOLOV4 image recognition algorithm based on the reparameterized residual structure and coordinate attention through the deep learning residual network theory and the model training algorithm based on deep learning. The YOLOV4 image recognition model of the force mechanism, which is not limited by the complexity of the image background to be recognized, has better robustness and faster reasoning speed, which can effectively improve the accuracy and speed of target recognition.

In addition, the YOLOV4 image recognition algorithm based on the reparameterized residual structure and coordinate attention mechanism according to the above-mentioned embodiments of the present invention may also have the following additional technical features:

In the first step, in one embodiment of the present invention, the YOLOV4 model includes an input terminal, a backbone network, a bottleneck network and an output terminal. The input end will perform data enhancement on the training image set, and calculate the initial aiming frame of the training image set according to the K-means++ clustering algorithm. The backbone network includes Darknet53 network, Mish activation function and Leakyrelu activation function. The bottleneck network includes the SPP module and the feature fusion structure of FPN+PAN. The output includes the CIOU_Loss loss function and the CIOU_nms prediction box screening method.

In the second step, in one embodiment of the present invention, the Rer used in the training includes the first bottleneck module 1 (Bottleneck1) and the first 1×1 convolutional layer to perform the Add operation, and pass the Mish activation function , and perform the Add operation through the first bottleneck module 2 (Bottleneck2) and the second 1×1 convolutional layer, and finally through the Mish activation function. Among them, the specific operation of Add is to superimpose the feature map information under the condition that the dimension is unchanged, so as to increase the information describing the image features. Bottleneck1 includes the first 3×3 convolutional layer and the first 1×1 convolutional layer for Add operations. Bottleneck2 includes the first 3×3 convolutional layer, the first 1×1 convolutional layer and Identity for Add operation. The model reparameterizes the residual structure before inference, converts Bottleneck1 and Bottleneck2 into 3×3 convolutional layers, and finally converts Rer into two concatenated 3×3 convolutional layers, and the converted The one-way structure can greatly speed up inference. The specific reparameterized fusion process includes fusing each convolutional layer with the BN layer, and the convolutional layer can be expressed as:

Conv(x)=W(x)+b

Where x represents the input vector, Conv represents the convolution operation, W represents the weight vector, and b represents the bias.

The BN layer can be expressed as:

Where x represents the input vector, BN represents the batch normalization operation, mean represents the average value of the input vector, var represents the variance of the input vector, and β and γ represent learnable parameters.

The result of the convolutional layer is brought into the BN layer to obtain the fusion result, which can be expressed as:

Where x represents the input vector, Conv represents the convolution operation, BN represents the batch normalization operation, β, γ represents the learnable parameters, W represents the weight vector, b represents the bias, mean represents the average value of the input vector, and var represents the input vector Variance. The 1×1 convolutional layer in the residual structure is expanded into a 3×3 convolutional layer, and its value is placed in the center of the 3×3 convolutional layer, and the remaining positions are filled with 0. The fusion operation is a reparameterization process, which can greatly increase the speed of model inference.

In the third step, in one embodiment of the present invention, the coordinate attention mechanism includes average pooling in the two spatial directions of X and Y, which can be expressed as:

Where x is the specified input, d represents the number of channels, and the average pooling kernel of (H, 1), (1, W) is used to encode each channel in the horizontal direction and vertical direction respectively, and i represents each channel in the height A feature point, j represents each feature point on the width, and z represents the output after average pooling in the X and Y directions. The coordinate attention mechanism enables the model to better recognize targets in complex backgrounds, and aggregates the position information and spatial information in the two spatial directions of the input feature map to obtain a more robust model.

The fourth step, in one embodiment of the present invention, the input end uses the K-means++ clustering algorithm to set the initial aiming frame of the training image set, and the algorithm sets the input image set X={x ₁ , x ₂ ,. .., x _n } and the number of clusters k, randomly select a sample point from the image set as the initial clustering center c ₁ , for each sample point x _i in the image set, calculate the distance between the two The shortest distance D(x). Points with larger values of D(x) have a higher probability of being selected as new cluster centers. Repeat the above steps to obtain k cluster centers.

The fifth step, in one embodiment of the present invention, the Leaky relu activation function and the Mish activation function are selected as the activation function of the benchmark network, and the Dropblock regularization method is used to alleviate the over-fitting phenomenon in the model training. The specific operation is in Randomly discard blocky feature points in the feature map.

The sixth step, in one embodiment of the present invention, the bottleneck network part of the model uses the SPP structure, which simultaneously performs the first 1×1 maximum pooling layer and the first 5×5 maximum pooling layer on the input feature map. The pooling layer and the first 9×9 maximum pooling layer perform concat operations on the feature maps after pooling with three different maximum pooling kernels and the original feature maps to obtain feature map views in different ranges. At the same time, the FPN+PAN structure is selected in the feature fusion part. The specific operation of the FPN structure is to upsample the feature map twice to obtain a feature map with richer semantic information. The specific operation of the PAN structure is to downsample the feature map twice. Sampling to obtain a feature map with richer positional information, and a feature map with extremely rich positional and semantic information through the feature fusion structure of FPN+PAN.

The seventh step, in one embodiment of the present invention, the output terminal includes a CIOU_Loss loss function, which can be expressed as:

Among them, IOU represents the intersection and union ratio between the real frame and the predicted frame, which is a standard to measure the accuracy of target detection, b ^p represents the center point of the predicted frame, b ^gt represents the center point of the real frame, ρ ² (b ^p ,b ^gt ) Represents the Euclidean distance between the predicted frame and the center point of the real frame, c represents the diagonal distance between the real frame and the minimum circumscribed rectangle of the predicted frame, v can represent the impact of the two types of frames on the loss function in terms of aspect ratio, w ^gt Represents the width of the real frame, h ^gt represents the height of the real frame, w ^p represents the width of the predicted frame, h ^p represents the height of the predicted frame, and finally selects the improved NMS of CIOU to screen many candidate frames, CIOU considers the candidate frame The effect of the aspect ratio on the result, so the screening result is more accurate.

Description of drawings

The above and/or additional aspects and advantages of the present invention will become apparent and easy to understand from the following description of the embodiments in conjunction with the accompanying drawings, wherein:

FIG. 1 is a schematic flow diagram of a YOLOV4 image recognition algorithm based on a reparameterized residual structure and a coordinate attention mechanism provided by an embodiment of the present invention;

2 is a schematic diagram of the model structure of the YOLOV4 image recognition algorithm based on a reparameterized residual structure and a coordinate attention mechanism according to an embodiment of the present invention and a specific structural explanation of each module in the model;

FIG. 3 is a schematic diagram of reparameterization of the residual network structure in a YOLOV4 image recognition algorithm based on a reparameterized residual structure and coordinate attention mechanism provided by an embodiment of the present invention.

Detailed ways

Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

The following describes the image recognition algorithm of YOLOV4 based on the reparameterized residual structure and coordinate attention mechanism according to the embodiments of the present invention with reference to the accompanying drawings. YOLOV4 Image Recognition Algorithm with Difference Structure and Coordinate Attention Mechanism.

Fig. 1 is a flow chart of the YOLOV4 image recognition algorithm based on reparameterized residual structure and coordinate attention mechanism according to an embodiment of the present invention.

As shown in Figure 1, the YOLOV4 image recognition algorithm based on reparameterized residual structure and coordinate attention mechanism includes the following steps:

In step S101, an image set to be recognized is input.

An image set with a complex background environment and many targets to be detected is selected, and the image set is used to evaluate the target recognition effect of the model.

In step S102, data enhancement is performed on the training image set, and K-means++ clustering algorithm is used to calculate the initial aiming frame of the training image set.

It can be understood that the training image set is processed by cropping, scaling, saturation transformation, etc., and the Mosaic data enhancement operation is used. The specific principle is: extract a batch (batch) in the training set, and randomly extract four pictures from it Cutting, splicing into a new picture, repeating this operation batch size (batch size) times, and finally inputting batch size Mosaic processed pictures into the model, this operation can effectively improve the training speed of the network. Using the K-means++ clustering algorithm to obtain the initial aiming frame of the training image set, the algorithm takes the input image set X={x ₁ , x ₂ ,...,x _n } and the number of clusters k, from the image set Randomly select a sample point as the initial cluster center c ₁ , and for each sample point _xi in the image set, calculate the shortest distance D(x) between the two. Points with larger values of D(x) have a higher probability of being selected as new cluster centers. Repeat the above steps to obtain k cluster centers.

In step S103, the YOLOV4 model of the reparameterized residual structure and coordinate attention mechanism as shown in Figure 2 is obtained, wherein the YOLOV4 model of the reparameterized residual structure and coordinate attention mechanism is based on YOLOV4, and the The residual network structure is improved to a reparameterized residual network structure, and a coordinate attention mechanism is added to the model to perform feature aggregation in the X and Y spatial directions.

It can be understood that, as shown in Figure 2, the size of the input image is 608*608*3, the model includes the first CBM module, the first coordinate attention mechanism, the first Rer1 module, the first Rer2 module, The first Rer8 module, the second Rer8 module, the first CBL*4 module, the first CBL*3 module, the first SPP module, the second CBL*3 module, the first CBL module, the obtained After the feature map is subjected to the upconvolution operation and the feature map processed by the second Rer8 module is subjected to the Concat operation after the second CBL module, the first CBL*5 module and the third CBL module are subjected to the upconvolution operation, After the feature map processed by the first Rer8 module passes through the fourth CBL module, Concat operation is performed, the second CBL*5 module, the fifth CBL module, and the first Conv module get the first predicted feature The size of the graph is 76*76*255. Pass the feature map processed by the second CBL*5 module through the sixth CBL module, and perform Concat operation with the feature map processed by the first CBL*5 module. Three CBL*5 modules, the seventh CBL module, and the second Conv module get the second predicted feature map size of 38*38*255, and pass the feature map processed by the third CBL*5 module through the second Eight CBL modules, and perform Concat operation on the feature map obtained after processing with the second CBL*3 module, the fourth CBL*5 module, the eighth CBL module, and the third Conv module to get the third The predicted feature map size is 19*19*255. The CBM module includes the first convolutional layer, the first BN layer and the first Mish activation function. The CBL module includes the first convolutional layer, the first BN layer and the first Leakyrelu activation function. The SPP module is to input the feature map at the same time. The first Max pool module with a maximum pooling kernel of 1×1, the first Max pool module with a maximum pooling kernel of 5×5, and the first maximum pooling kernel of 9× 9's Maxpool module, and Concat operation with the unprocessed feature map, the first CBL module. The coordinate attention mechanism is to simultaneously perform the first XAVGpool module and the first YAVG pool module on the input feature map, and then perform the Concat operation. The first Conv module, the first BN layer, and the first Leaky relu activation function will be The previous step outputs the second Conv module, the second Leaky relu activation function, and the third Conv module, the third Leaky relu activation function, the feature map processed by the second Leaky relu activation function and the third The feature map processed by a Leaky relu activation function is added to the unprocessed feature map. The RerX module is composed of X groups of modules shown in Figure 2 in series. The Rer module used in the training includes the first Bottleneck1 and the first 1×1 convolution layer to perform the Add operation, pass the Mish activation function, and pass the first A Bottleneck2 performs an Add operation with the second 1×1 convolutional layer, and finally passes the Mish activation function. Bottleneck1 includes the first 3×3 convolutional layer, and the first 1×1 convolutional layer performs the Add operation. Bottleneck2 includes the first 3×3 convolutional layer, the first 1×1 convolutional layer and Identity for the Add operation. The model reparameterizes the residual structure before inference, converts Bottleneck1 and Bottleneck2 into 3×3 convolutional layers respectively, and finally converts the Rer module into two concatenated 3×3 convolutional layers, after conversion The single-channel structure of the method can greatly speed up the reasoning speed. The specific fusion part includes the fusion of each convolutional layer and the BN layer as shown in Figure 3. The convolutional layer can be expressed as:

Conv(x)=W(x)+b

Where x represents the input vector, Conv represents the convolution operation, W represents the weight vector, and b represents the bias.

The BN layer can be expressed as:

Where x represents the input vector, BN represents the batch normalization operation, mean represents the average value of the input vector, var represents the variance of the input vector, and β and γ represent learnable parameters.

The result of the convolutional layer is brought into the BN layer to obtain the fusion result, which can be expressed as:

Where x represents the input vector, Conv represents the convolution operation, BN represents the batch normalization operation, β, γ represents the learnable parameters, W represents the weight vector, b represents the bias, mean represents the average value of the input vector, and var represents the input vector deviation. Bottleneck1 has two branches as shown in A1 in Figure 3, including the first 1×1 convolutional layer and the first 3×3 convolutional layer, expanding the first 1×1 convolutional layer to 3 ×3 convolutional layer, put its value in the middle of the 3×3 convolutional layer, and fill the rest with 0. The filling result is shown in A2 in Figure 3. Finally, the two 3×3 convolutional layers are combined to form a 3×3 convolutional layer, as shown in A3 in Figure 3. Bottleneck2 is shown in A4 in Figure 3, where Identity does not change the input value, so it is converted into a 3×3 convolutional layer, and the 1×1 convolutional layer in the residual structure is expanded to A 3×3 convolutional layer, put its value in the center of the 3×3 convolutional layer, and fill the rest of the positions with 0. After the process is completed, as shown in A5 in Figure 3, the residual structure includes three Each branch becomes a 3×3 convolutional layer, and the weights and biases of the above three branches are superimposed to obtain a new 3×3 convolutional layer, as shown in A6 in Figure 3. The Rer module is composed of two structures similar to Bottleneck1 in series. After the same fusion process of Bottleneck1, the first 3×3 convolutional layer is obtained, after the first Mish activation function, the second 3×3 convolutional layer , and finally through the second Mish activation function, the fusion operation is a reparameterization process, which can greatly increase the model reasoning speed.

Figure 2 is a structure diagram of the YOLOV4 model based on the reparameterized residual structure and the coordinate attention mechanism. The coordinate attention mechanism is added after the first CBL, and the coordinate attention mechanism includes X and Y in the two spatial directions. Average pooling can be expressed as:

Where x is the specified input, d represents the number of channels, and the average pooling kernel of (H, 1), (1, W) is used to encode each channel in the horizontal direction and vertical direction respectively, and i represents each channel in the height A feature point, j represents each feature point on the width, and z represents the output after average pooling in the X and Y directions. The coordinate attention mechanism enables the model to better recognize targets in complex backgrounds, aggregate the position information and spatial information in the two spatial directions of the input feature map, and obtain a more robust model. The bottleneck network part in Figure 2 is the feature fusion structure of FPN+PAN. The SPP module increases the receptive field of the network, fuses features of different scales together, and finally outputs three prediction maps of different scales. As shown in Figure 2, the output of the network passes a CBL and a Conv for feature map prediction. The output includes the CIOU_Loss loss function, which can be expressed as:

Among them, IOU represents the intersection and union ratio between the real frame and the predicted frame, which is a standard to measure the accuracy of target detection, b ^p represents the center point of the predicted frame, b ^gt represents the center point of the real frame, ρ ² (b ^p ,b ^gt ) Indicates the Euclidean distance between the predicted frame and the center point of the real frame, and c represents the diagonal distance between the real frame and the smallest circumscribed rectangle of the predicted frame. v can represent the influence of two types of frames on the loss function in terms of aspect ratio, w ^gt represents the width of the real frame, h ^gt represents the height of the real frame, w ^p represents the width of the predicted frame, h ^p represents the height of the predicted frame, and finally The NMS improved by CIOU is also used to screen many candidate boxes. CIOU takes into account the influence of the aspect ratio of the candidate box on the results, so the screening results are more accurate. The specific operation process is as follows:

In the first step, the one with the highest confidence among the many candidate frames is used as a sample, and the CIOU of other candidate frames and samples is calculated, which can be expressed as;

Among them, IOU represents the intersection and union ratio between the real frame and the predicted frame, which is a standard to measure the accuracy of target detection, b ^p represents the center point of the predicted frame, b ^gt represents the center point of the real frame, ρ ² (b ^p ,b ^gt ) Indicates the Euclidean distance between the predicted frame and the center point of the real frame, and c represents the diagonal distance between the real frame and the smallest circumscribed rectangle of the predicted frame. v can represent the influence of two types of boxes on the loss function in terms of aspect ratio.

In the second step, when the calculated CIOU value is greater than the set threshold, the candidate box is removed;

Repeat the above steps to filter a large number of predicted repeated frames to obtain accurate prediction results. CIOU takes into account the overlapping area between frames, the distance between center points and the aspect ratio of frames, so it can be more accurate. prediction results.

In step S104, model training is performed according to the YOLOV4 model of the reparameterized residual structure and the coordinate attention mechanism, the training image set, and the verification image set to generate a recognition model.

It can be understood that, firstly, the LabelImg tool is used to make the image set, and the initial aiming frame of the training image set is calculated according to the K-means++ clustering algorithm. The Adam optimizer is used in the training process, which comprehensively considers the first-order moment estimation and the second-order moment estimation of the gradient. The specific operation steps of the Adam optimizer are:

The first step is to set the learning rate lr, the smoothing constants β ₁ and β ₂ are used to smooth m and v respectively, and the initial value of the learnable parameters is set to θ ₀ , m ₀ =0, v ₀ =0, t=0;

In the second step, on the premise of not stopping the training, the training times are updated to t=t+1;

The third step is to calculate the gradient g _t ;

In the fourth step, the cumulative gradient can be expressed as:

m _t ＝β ₂ *v _t-1 +(1-β ₂ )*(g _t ) ²

In the fifth step, the deviation correction m can be expressed as:

In the sixth step, the deviation correction v can be expressed as:

In the seventh step, the update parameters can be expressed as:

Where ε is a small constant to avoid the case where the denominator is 0.

In order to avoid the occurrence of over-fitting phenomenon, the DropBlock regularization method and class label smoothing are adopted. The specific operation of the DropBlock regularization method is: randomly extract the block feature points in the feature map and discard them. The specific operation of class label smoothing is to adjust the upper limit of the model prediction target to a value less than 1.0, which reduces the memory of the model to the prediction results to a certain extent, so that the model will not be overconfident.

In step S105, the image set to be recognized is input into the YOLOV4 recognition model of re-parameterized residual structure and coordinate attention mechanism, and the image recognition result is obtained.

It can be understood that after the training, the weight file will be generated, the generated weight file will be called, and the test will be performed according to the YOLOV4 model of the reparameterized residual structure and the coordinate attention mechanism. The training model can quickly and accurately target the target under the complex background identification.

It should be noted that the foregoing explanations for the embodiment of the YOLOV4 image recognition algorithm based on the reparameterized residual structure and the coordinate attention mechanism are also applicable to the YOLOV4 based on the reparameterized residual structure and the coordinate attention mechanism of this embodiment The image recognition system will not be repeated here.

According to the YOLOV4 image recognition algorithm and system of the re-parameterized residual structure and coordinate attention mechanism of the embodiment of the present invention, the end-to-end recognition task can be realized, and the image can be fully automatically recognized without being affected by the background of the image to be recognized. The limitation of complexity, strong applicability, good model performance and robustness make the target recognition not only fast, but also high precision.

All the steps included in the algorithm of the embodiment of the present invention can be completed by the hardware related to the program instruction, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the algorithm embodiment.

In the description of this specification, descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific examples", or "some examples" mean that specific features described in connection with the embodiment or example , structure, material or characteristic is included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the described specific features, structures, materials or characteristics may be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples and features of different embodiments or examples described in this specification without conflicting with each other.

In addition, the terms "first" and "second" are used for descriptive purposes only, and cannot be interpreted as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include at least one of these features. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

Any process or algorithmic description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing a custom logical function or process step , and the scope of preferred embodiments of the invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including substantially concurrently or in reverse order depending on the functions involved, which shall It is understood by those skilled in the art to which the embodiments of the present invention pertain.

The logic and/or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium, For use with an instruction execution system, system, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, system, or device and execute instructions), or in conjunction with such an instruction execution system, system or equipment used. For the purposes of this specification, a "computer-readable medium" may be any system that can contain, store, communicate, propagate or transmit a program for use in or in conjunction with an instruction execution system, system or device. More specific examples (non-exhaustive list) of computer-readable media include the following: electrical connection with one or more wires (electronic system), portable computer disk case (magnetic system), random access memory (RAM), Read Only Memory (ROM), Erasable and Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Systems, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program can be printed, as it may be possible, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or other suitable processing if necessary. The program is processed electronically and stored in computer memory.

It should be understood that various parts of the present invention can be realized by hardware, software, firmware or their combination. In the above described embodiments, various steps or algorithms may be implemented by software or firmware stored in memory and executed by a suitable instruction execution system.

Those of ordinary skill in the art can understand that all or part of the steps carried by the algorithm of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. During execution, one or a combination of the steps of the algorithm embodiment is included.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing module, each unit may exist separately physically, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. If the integrated modules are implemented in the form of software function modules and sold or used as independent products, they can also be stored in a computer-readable storage medium.

The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, and the like. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention, those skilled in the art can make the above-mentioned The embodiments are subject to changes, modifications, substitutions and variations.

Claims (13)

1. A YOLOV4 image recognition algorithm based on reparameterized residual structure and coordinate attention mechanism, characterized in that, comprising: Input the image set to be recognized; Perform data enhancement on the training image set, and use the K-means++ clustering algorithm to calculate the initial aiming frame of the image set; Obtain the YOLOV4 model of reparameterized residual structure and coordinate attention mechanism; According to the YOLOV4 model of the reparameterized residual structure and coordinate attention mechanism, the training image set and the verification image set are used for model training to generate a recognition model; Input the image set to be recognized into the YOLOV4 recognition model with re-parameterized residual structure and coordinate attention mechanism to obtain the image recognition result. 2. the YOLOV4 image recognition algorithm based on reparameterization residual structure and coordinate attention mechanism according to claim 1, is characterized in that, described YOLOV4 model comprises input end, backbone network, bottleneck network, output end four parts . At the input end, data enhancement is performed on the training image set, and the K-means++ clustering algorithm is used to set the initial aiming frame of the training image set. The backbone network uses the Darknet53 network, which can extract the features of the image set. Bottleneck networks include Feature Pyramid Networks (FPN) and Pyramid Attention Networks (PAN), which can extract complex features of image sets. The output end includes a convolution module, and finally predicts the location and category of the target. 3. the YOLOV4 image recognition algorithm based on reparametric residual structure and coordinate attention mechanism according to claim 1, is characterized in that, in the network, the reparametric residual structure (Reparametric residual structure, Rer) is in The residual structure with branches is used in model training, and the single-pass convolution module is obtained after reparameterization of the residual structure with branches. The above-mentioned single-pass convolution module is used in the inference process. The Rer used in the training includes the first bottleneck module 1 (Bottleneck1) and the first 1×1 convolutional layer to perform the Add operation, through the Mish activation function, and through the first bottleneck module 2 (Bottleneck2) and the second Two 1×1 convolutional layers perform the Add operation, and finally pass the Mish activation function. The specific principle of the Add operation is to superimpose the feature map information under the condition that the dimension of the feature map remains unchanged, so as to increase the information describing the image features. Bottleneck1 includes the first 3×3 convolutional layer and the first 1×1 convolutional layer for the Add operation. Bottleneck2 includes the first 3×3 convolutional layer, the first 1×1 convolutional layer and the original input (Identity) for the Add operation. The model reparameterizes the residual structure before inference, converts Bottleneck1 and Bottleneck2 into 3×3 convolutional layers respectively, and finally converts Rer into two concatenated 3×3 convolutional layers, and the converted The one-way structure can greatly speed up inference. 4. the YOLOV4 image recognition algorithm based on reparameterization residual structure and coordinate attention mechanism according to claim 1, is characterized in that, the coordinate attention mechanism of described addition can be to the feature of X, Y two spatial directions The graph is aggregated, which includes the average pooling in the two spatial directions of X and Y, which can be expressed as:

Where x is the specified input, d represents the number of channels, and the average pooling kernel of (H, 1), (1, W) is used to encode each channel in the horizontal direction and vertical direction respectively, and i represents each channel in the height A feature point, j represents each feature point on the width, z represents the output of the average pooling in the X and Y directions, the coordinate attention mechanism can extract the position accuracy information in one spatial direction and the other spatial direction remote dependencies. 5. the YOLOV4 image recognition algorithm based on reparameterized residual structure and coordinate attention mechanism according to claim 2, is characterized in that, the input end of described YOLOV4 model comprises Mosaic data enhancement, at first any four pictures are carried out Basic processing such as cropping, shrinking, and transparency transformation, and finally splicing the processed four images into a new image, this operation can not only speed up the inference speed of the model, but also perform data enhancement on the training image set. The Dropblock regularization method is used to alleviate the overfitting phenomenon in model training. The specific operation is to randomly discard block feature points in the feature map. 6. the YOLOV4 image recognition algorithm based on reparameterization residual structure and coordinate attention mechanism according to claim 2, is characterized in that, the backbone network of described YOLOV4 model selects Darknet53, and the activation function in the benchmark network selects Mish activation function and the Leaky relu activation function. 7. the YOLOV4 image recognition algorithm based on reparameterized residual structure and coordinate attention mechanism according to claim 2, is characterized in that, the bottleneck network of described YOLOV4 model, selects pyramid pooling layer (Spatial PyramidPooling Layer, SPP ), which includes performing the first 1×1 maximum pooling layer, the first 5×5 maximum pooling layer, the first 9×9 maximum pooling layer on the input feature map and three different maximum pooling layers The feature map after core pooling and the original feature map perform Concat operation. The specific principle of Concat operation is that the number of channels describing image features increases, but the information of each channel does not increase to obtain features in different ranges. Figure field of view. The model also uses the structure of FPN+PAN. FPN is a feature map that is upsampled twice to obtain a feature map with richer semantic information. PAN is a feature map that is downsampled twice to obtain a feature map with richer positional information. Finally, the two feature maps with rich semantic information and location information are superimposed to obtain an output that can fully express the feature map information. 8. the YOLOV4 image recognition algorithm based on reparameterization residual structure and coordinate attention mechanism according to claim 2, is characterized in that, the output end of described YOLOV4 model comprises complete IOU loss function (Complete-IOU_Loss, CIOU_Loss) , which can be expressed as:

Among them, IOU represents the intersection ratio between the real frame and the predicted frame, which is a standard to measure the accuracy of target detection, b ^p represents the center point of the predicted frame, b ^gt represents the center point of the real frame, ρ ² (b ^p ,b ^gt ) Represents the Euclidean distance between the predicted frame and the center point of the real frame, c represents the diagonal distance between the real frame and the minimum circumscribed rectangle of the predicted frame, v represents the influence of the two types of frames on the loss function in terms of aspect ratio, w ^gt represents The width of the real frame, h ^gt represents the height of the real frame, w ^p represents the width of the predicted frame, and h ^p represents the height of the predicted frame. Finally, the improved NMS of the complete IOU (Complete-IOU, CIOU) is selected to carry out many candidate frames. filter. 9. the YOLOV4 image recognition algorithm based on reparameterization residual structure and coordinate attention mechanism according to claim 3, is characterized in that, the specific process of the single-way structure when the residual structure during training is improved into reasoning For, Identity is equivalent to a 1×1 convolutional layer, and a 1×1 convolutional layer is equivalent to a 3×3 convolutional layer that puts the value in the center of the convolutional layer and fills the remaining positions with 0 . 10. A YOLOV4 target recognition system based on reparameterized residual structure and coordinate attention mechanism, characterized in that, comprising: The input module is used to input the image set to be recognized, and the data format is VOC format; The data enhancement and clustering module is used to perform data enhancement on the input training image set, and calculate the initial aiming frame of the training image set according to the K-means++ clustering algorithm; The obtaining module is used to obtain the YOLOV4 target recognition model based on the reparameterized residual structure and coordinate attention mechanism; The training module is used for carrying out model training according to the YOLOV4 target recognition model based on the reparameterized residual structure and coordinate attention mechanism and the training image set and the verification image set to generate a recognition model; The recognition module is used to obtain an image recognition result through the YOLOV4 target recognition model based on the reparameterized residual structure and coordinate attention mechanism according to the image set to be recognized. 11. the YOLOV4 target recognition algorithm system based on reparameterization structure and coordinate attention mechanism according to claim 10, it is characterized in that, in the network, the reparameterization residual structure adopts branched Residual structure, reparameterize the residual structure with branches to obtain a single-pass convolution module, and use the above-mentioned single-pass convolution module in the inference process. 12. The YOLOV4 target recognition algorithm system based on re-parameterized structure and coordinate attention mechanism according to claim 10, wherein the coordinate attention mechanism added can be used for the feature maps of X and Y two spatial directions Aggregation is carried out, which includes the average pooling in the two spatial directions of X and Y, which can be expressed as:

Where x is the specified input, d represents the number of channels, and the average pooling kernel of (H, 1), (1, W) is used to encode each channel in the horizontal direction and vertical direction respectively, and i represents each channel in the height A feature point, j represents each feature point on the width, and z represents the output after average pooling in the X and Y directions. The coordinate attention mechanism can extract location accuracy information in one spatial direction and long-range dependencies in another spatial direction. 13. The YOLOV4 target recognition algorithm system based on reparameterized structure and coordinate attention mechanism according to claim 10, wherein the output terminal includes a CIOU_Loss loss function, which can be expressed as:

Among them, IOU represents the intersection ratio between the real frame and the predicted frame, which is a standard to measure the accuracy of target detection, b ^p represents the center point of the predicted frame, b ^gt represents the center point of the real frame, ρ ² (b ^p ,b ^gt ) Represents the Euclidean distance between the predicted frame and the center point of the real frame, c represents the diagonal distance between the real frame and the minimum circumscribed rectangle of the predicted frame, v can represent the impact of the two types of frames on the loss function in terms of aspect ratio, w ^gt Represents the width of the real frame, h ^gt represents the height of the real frame, w ^p represents the width of the predicted frame, h ^p represents the height of the predicted frame, and finally selects CIOU's improved NMS to screen many candidate frames.

Images