CN116912796A - Novel dynamic cascade YOLOv 8-based automatic driving target identification method and device - Google Patents

Abstract

The invention discloses a method and device for automatic driving target recognition based on new dynamic cascade YOLOv8. The original images of traffic vehicles obtained in advance are pre-processed and divided into training sets and test sets; constructing a new dynamic cascade YOLOv8-based automatic driving target recognition method and device. Self-driving target recognition network; the self-driving target recognition network replaces the entire Backbone backbone network in the YOLOv8 network with a new dynamic cascade backbone network; replaces the detection head in the last part of the YOLOv8 network with the new cross-scale shared convolution weight ShareSepHead Detection head; use improved PolyLoss as the loss function of the automatic driving target recognition network; use the training set to train the automatic driving target recognition network type; input the test set into the trained automatic driving target recognition network, and conduct the test on the automatic driving target recognition network Evaluate. The invention can improve the accuracy and speed of target recognition in automatic driving and provide guarantee for the safety of automatic driving.

Description

An automatic driving target recognition method based on new dynamic cascade YOLOv8 and device

Technical field

The invention belongs to the application of deep learning in the field of computer vision, and specifically relates to an automatic driving target recognition method and device based on a new dynamic cascade YOLOv8.

Background technique

As one of the core issues of computer vision, target detection aims to find the category and location of specific targets in images. It has been widely used in various fields, such as autonomous driving, remote sensing images, video surveillance, and medical detection.

The development of YOLO has been continuously updated since 2016, and it has reached v8. In 2016, the single-stage (One-Stage) target detection method represented by YOLOv1 appeared. Looking at the development history of single-stage target detection methods, we can find that from the first single-stage target detection method YOLOv1 to YOLOv8 in 2023, the YOLO series of target detection methods have developed with the development of single-stage target detection and have become One- A typical representative of the Stage method.

Although YOLOv8 can detect objects quickly when processing simple images, when faced with complex scenes, such as traffic jams in reality, when there are a large number of vehicles and pedestrians, more time is needed for detection. The real-time nature of autonomous driving is crucial for decision-making, and the processing speed still needs to be improved. There is also an improvement in accuracy. Autonomous driving requires highly accurate target detection results to ensure correct responses to various traffic conditions. Although YOLOv8 may perform better in some scenarios, the detection accuracy still needs to be improved in some complex traffic situations. The backbone backbone network of the existing technology YOLOv8 is very fast when processing simple images, but it takes more time when encountering complex images with many targets; the detection head model of the existing YOLOv8 contains more parameters, resulting in higher computational complexity. high. In autonomous driving systems, computing resources are limited, so more efficient model design is needed to ensure target detection in embedded or resource-limited environments.

Contents of the invention

Purpose of the invention: The present invention proposes an automatic driving target recognition method and device based on a new dynamic cascade YOLOv8, which can perform accurate target detection in automatic driving.

Technical solution: The present invention proposes an automatic driving target recognition method based on a new dynamic cascade YOLOv8, which specifically includes the following steps:

(1) Preprocess the original images of traffic vehicles obtained in advance and divide them into training sets and test sets;

(2) Construct an automatic driving target recognition network based on the new dynamic cascade YOLOv8; the automatic driving target recognition network replaces the entire Backbone backbone network in the YOLOv8 network with a new dynamic cascade backbone network; replaces the detection head in the last part of the YOLOv8 network Replaced with the new ShareSepHead detection head that shares convolution weights across scales;

(3) Use improved PolyLoss as the loss function of the autonomous driving target recognition network;

(4) Use the training set to train the automatic driving target recognition network type;

(5) Input the test set into the trained automatic driving target recognition network and evaluate the automatic driving target recognition network.

Further, the new dynamic cascade backbone network described in step (2) has two cascade backbone networks, and a dynamic router is inserted between the two backbone networks to automatically select the best route for each image to be detected; The image will extract the first-level multi-scale features through the first backbone network, and send the multi-scale features to the dynamic router to judge the difficulty of the image; the features will be mapped to the difficulty score through two linear mapping layers; if the judgment is If it is a "simple" image, the first-level multi-scale features will be sent to the head part of YOLOv8; if it is judged to be a "difficult" image, the image to be detected and its first-level multi-scale features will be sent to the second backbone network. The second-level multi-scale features are extracted and sent to the head part of YOLOv8.

Further, the implementation process of the new dynamic cascade backbone network described in step (2) is as follows:

For the input image x, first extract its multi-scale feature F1, and the first backbone B1 is:

In the formula, L is the number of stages, that is, the number of multi-scale features; then, the router R will use these multi-scale features F1 to predict the difficulty score φ∈(0,1) of the image as:

If the router classifies the input image as a "simple" image, then the following neck head D1 will output the detection result y as:

If the router classifies the input image as a "complex" image, the multi-scale features will need to be further enhanced by a second backbone, embedding the multi-scale features F1 into H through a composite connection module G:

Among them, G is the DHLC that implements CBNet; the input image x is sent to the second backbone, and the features of the second backbone are enhanced by sequentially summing the corresponding elements of the embedded H at each stage, recorded as:

Test results, the second head and neck D2 decoding is:

y=D ₂ (F ₁ ).

Further, the ShareSepHead detection head in step (2) shares convolution weights between different layers and independently calculates the statistics of BN; the ShareSepHead includes a first convolution layer and a first depth-separable convolution layer connected in sequence. layer, the second depth separable convolutional layer, the second convolutional layer and the BN normalization layer.

Further, the first convolution layer is a 3x3 convolution layer, changing the number of channels of the input feature map from x to c2*2; the first depth-separable convolution layer first applies convolution to each input channel respectively. product operation, and then combine the features between channels; the second depth-separable convolution layer reduces the number of channels of the input feature map from c2 to c2; the second convolution layer is a 1*1 convolution layer, which combines the input features The number of channels of the graph changes from c2 to 4*self.reg_max; each detection head is normalized by BN to improve the gradient propagation and training speed. BN normalization normalizes the data of each mini-batch.

Further, the improved PolyLoss described in step (3) includes a combined loss function and a weighted binary cross-entropy loss; PolyLoss combines the binary cross-entropy loss and Focal Loss to improve difficult samples by adjusting the weight and shape of the loss function. and balanced processing capabilities between positive and negative samples; use weighted binary cross-entropy loss to calculate the binary cross-entropy loss between the prediction result and the real label to measure the matching degree between the prediction and the real label; introduce alpha_factor to weight the loss so that The losses of positive samples and negative samples are adjusted to varying degrees in the calculation; a polynomial adjustment factor is added to increase the uncertainty of sample probability prediction.

Further, the "simple" image is a single target image; the "difficult" image is two or more target images.

Based on the same inventive concept, the present invention proposes a device including a memory and a processor, wherein:

Memory for storing computer programs capable of running on the processor;

A processor, configured to execute the steps of the automatic driving target recognition method based on the new dynamic cascade YOLOv8 as described above when running the computer program.

Based on the same inventive concept, the present invention proposes a storage medium with a computer program stored on the storage medium. When the computer program is executed by at least one processor, the above-mentioned automatic driving goal based on the new dynamic cascade YOLOv8 is achieved. Identify method steps.

Beneficial effects: Compared with the existing technology, the beneficial effects of the present invention are: the automatic driving target recognition network based on the new dynamic cascade YOLOv8 constructed by the present invention enables the YOLOv8 backbone network to adaptively select inference routes for input images of different difficulties, improving Feature extraction efficiency; in order to improve YOLOv8 detection accuracy, a new and improved PolyLoss loss function is used, the hyperparameter search space is simplified, and polynomial coefficients are adjusted; in order to upgrade the YOLOv8 detection head, it saves parameters, is more efficient, and improves accuracy, and uses a novel sharing Detection head to enhance model capabilities to achieve higher performance; ultimately making target detection for autonomous driving more accurate.

Description of the drawings

Figure 1 is a schematic diagram of the dynamic cascade backbone network structure;

Figure 2 is a schematic diagram of the detection head structure of shared convolution weights and separated batch normalization layers.

Detailed ways

The present invention will be further described in detail below in conjunction with the accompanying drawings.

The present invention proposes an automatic driving target recognition method based on a new dynamic cascade YOLOv8, which specifically includes the following steps:

Step S1: This invention selects the KITTI data set, which has been divided into data sets, including test sets and training sets. Performance evaluation on autonomous driving datasets.

Step S2: Based on the YOLOv8 network foundation, replace the entire Backbone backbone network with the new Dynamic Cascade backbone network.

As shown in Figure 1, the new Dynamic Cascade backbone network has two cascade backbone networks, and a dynamic router is inserted between the two backbone networks to automatically select the best route for each image to be detected.

Adaptive router: In order to better judge the difficulty of the image, and make a difficulty judgment based on the input multi-scale feature information feature information. Assume that the first backbone network has output multi-scale features. In order to reduce the computational complexity of the dynamic router, information is first compressed to obtain the compressed features, which are global pooling operations and channel dimension splicing operations. Afterwards, the features are mapped to difficulty scores through two linear mapping layers.

Two cascade networks: For the input image x, first extract its multi-scale feature F1, and the first backbone B1 is:

In the formula, L is the number of stages, that is, the number of multi-scale features. Router R will then use these multi-scale features F1 to predict the difficulty score φ∈(0,1) for this image as:

"Simple" images exit at the first trunk, while "complex" images require further processing. "Simple" images are simple images of a single target pedestrian or a single vehicle; "complex" images are two or more images of multiple types. Specifically, if the router classifies the input image as a "simple" image, then the following neck head D1 will output the detection result y as:

On the contrary, if the router classifies the input image as a "complex" image, the multi-scale features will need to be further enhanced by the second backbone instead of being immediately decoded by the neck head D1. In particular, the multi-scale feature F1 is embedded into H through a composite connection module G:

Where G is the DHLC that implements CBNet. Then the input image x is sent to the second backbone, and the features of the second backbone are enhanced by sequentially summing the corresponding elements of the embedded H at each stage, recorded as:

Test results, the second head and neck D2 decoding is:

y=D ₂ (F ₁ ).

Through the above process, a "simple" image will process only one backbone, while a "complex" image will process two backbones. Obviously, with such an architecture, there is a trade-off between computation (i.e. speed) and accuracy.

Step S3: Based on the YOLOv8 network, modify the default CIoU loss function to the new PolyLoss classification loss function to improve detection accuracy.

This loss function combines the ideas of binary cross entropy loss (BCEWithLogitsLoss) and focal loss (FL) and is used for target classification in target detection tasks. Contains the following sections:

Combined loss function: PolyLoss combines binary cross-entropy loss and Focal Loss to improve the model's ability to handle difficult samples and the balance between positive and negative samples by adjusting the weight and shape of the loss function.

Weighted binary cross-entropy loss: PolyLoss first uses nn.BCEWithLogitsLoss to calculate the binary cross-entropy loss between the prediction results and the true label. This part of the loss is used to measure how well the prediction matches the true label.

Focal Loss adjustment: In order to deal with difficult samples, PolyLoss introduces the ideas in Focal Loss. By adjusting the predicted probability value, samples with lower predicted probability play a greater role in the loss calculation, thereby increasing attention to difficult samples.

Loss weight adjustment: PolyLoss weights the loss by introducing alpha_factor. This factor is determined based on the value of the real label, so that the loss of positive samples and negative samples is adjusted to varying degrees in the calculation.

Polynomial adjustment: In the last step, PolyLoss introduces a polynomial adjustment factor to increase the uncertainty of the sample probability prediction. By adjusting the shape and coefficients of the polynomial, you can make the loss increase when the sample probability is lower or higher, further enhancing the focus on difficult samples.

In the target detection task, the PolyLoss loss function combines the ideas of binary cross-entropy loss and focal loss, and provides a loss calculation method that can handle difficult samples and balance positive and negative samples through polynomial adjustment and weight adjustment. This can help the model better learn and handle challenging object classification tasks.

Step S4: Based on the YOLOv8 network, modify the detection head in the last part of the YOLOv8 network to a novel ShareSepHead detection head that shares convolution weights across scales to form an autonomous driving target recognition network based on a new dynamic cascaded YOLOv8.

The original detection head of YOLOv8 is the last layer of the network, responsible for generating prediction results of target detection. It maps feature maps to grids at different scales depending on the size of the input image and the design of the network. Each grid cell is responsible for detecting and locating one or more targets. At each scale, the detection head outputs a set of prediction boxes. Each prediction box is composed of multiple attributes, usually including the coordinates of the bounding box (center coordinates and width and height), the probability of the target category, and the confidence score of the target's existence. . These predicted boxes are post-processed by non-maximum suppression (NMS), which is used to filter overlapping boxes and retain the most accurate detection results. The detection head usually uses a combination of convolutional layers and fully connected layers to adapt to target detection at different scales through different convolution kernel sizes and strides. The output of the detection head usually adopts appropriate activation functions and normalization operations to ensure that the prediction results are within a suitable range and provide good interpretability and robustness.

As shown in Figure 2, the novel ShareSepHead detection head that shares convolution weights across scales: shares the convolution weights between different layers, but independently calculates the statistics of BN (BatchNorm). This is a shared detection head. Real-time object detectors usually use separate detection heads for different feature scales to enhance model capabilities for higher performance, rather than sharing one detection head at multiple scales. This time we choose to share detection head parameters across scales, but use different batch normalization layers BN layers to reduce detection head parameters while maintaining accuracy. BN is also more efficient than other normalization layers because it directly uses the statistics calculated during training during inference.

After the image passes through the YOLOv8head head, it begins to enter the ShareSepHead detection head part to predict the results. Each head includes a first convolutional layer, a first depthwise separable convolutional layer, a second depthwise separable convolutional layer, a second convolutional layer and a BN normalization layer connected in sequence.

The first Conv3*3 convolutional layer is a 3x3 convolutional layer that changes the number of channels of the input feature map from x to c2*2. It helps extract features and increase the number of channels to better capture the target’s information.

The second DWConv3*3 depthwise separable convolution layer first applies convolution operations to each input channel separately, and then combines the features between channels. This helps reduce the amount of calculations and improves the efficiency of the model.

The third part, DWConv3*3 depthwise separable convolution layer, reduces the number of channels of the input feature map from c2 to c2. Similar to the previous step, this layer continues to reduce the number of channels and extract higher-level features.

The fourth part, Conv1*1 convolutional layer, changes the number of channels of the input feature map from c2 to 4*self.reg_max. Responsible for predicting the coordinate information of the bounding box.

The detection head parameter information is shared between each head.

Each detection head undergoes BN normalization to improve gradient propagation and training speed. BN normalization can keep the activation value in the network in a relatively small range by normalizing the data of each mini-batch. within, thus alleviating the problems of gradient disappearance and gradient explosion, promoting the propagation of gradients, and accelerating the network training process.

Step S5: Use the divided data set to train the automatic driving target recognition network based on the new dynamic cascade YOLOv8 constructed in step S4. Evaluate the performance of the trained automatic driving target recognition network based on the new dynamic cascade YOLOv8, and finally achieve target recognition in automatic driving.

Based on the same inventive concept, the present invention proposes a device, including a memory and a processor, wherein: the memory is used to store a computer program that can be run on the processor; the processor is used to when running the computer program, Execute the steps of the automatic driving target recognition method based on the new dynamic cascade YOLOv8 as mentioned above.

Based on the same inventive concept, the present invention also proposes a storage medium with a computer program stored on the storage medium. When the computer program is executed by at least one processor, the automatic driving based on the new dynamic cascade YOLOv8 is implemented as described above. Target identification method steps.

So far, the technical solutions of the present invention have been described in conjunction with the specific experimental processes shown in the drawings, but the protection scope of the present invention is not limited to these specific implementations. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to relevant technical features, and technical solutions after these modifications or substitutions will fall within the protection scope of the present invention.

Claims (9)

1. An automatic driving target recognition method based on the new dynamic cascade YOLOv8, which is characterized by including the following steps: (1) Preprocess the original images of traffic vehicles obtained in advance and divide them into training sets and test sets; (2) Construct an automatic driving target recognition network based on the new dynamic cascade YOLOv8; the automatic driving target recognition network replaces the entire Backbone backbone network in the YOLOv8 network with a new dynamic cascade backbone network; replaces the detection head in the last part of the YOLOv8 network Replaced with the new ShareSepHead detection head that shares convolution weights across scales; (3) Use improved PolyLoss as the loss function of the autonomous driving target recognition network; (4) Use the training set to train the automatic driving target recognition network type; (5) Input the test set into the trained automatic driving target recognition network and evaluate the automatic driving target recognition network. 2. An automatic driving target recognition method based on new dynamic cascade YOLOv8 according to claim 1, characterized in that the new dynamic cascade backbone network in step (2) has two cascade backbone networks, and A dynamic router is inserted between the two backbone networks to automatically select the best route for each image to be detected; the image to be detected will extract the first-level multi-scale features through the first backbone network, and send the multi-scale features into the dynamic The router judges the difficulty of the image; maps the features to difficulty scores through two linear mapping layers; if it is judged to be a "simple" image, the first-level multi-scale features will be sent to the head part of YOLOv8; if it is judged to be "difficult" "image, the image to be detected and its first-level multi-scale features will be sent to the second backbone network to extract the second-level multi-scale features and send them to the head part of YOLOv8. 3. An automatic driving target recognition method based on new dynamic cascade YOLOv8 according to claim 1, characterized in that the implementation process of the new dynamic cascade backbone network in step (2) is as follows: For the input image x, first extract its multi-scale feature F1, and the first backbone B1 is: In the formula, L is the number of stages, that is, the number of multi-scale features; then, the router R will use these multi-scale features F1 to predict the difficulty score φ∈(0,1) of the image as: If the router classifies the input image as a "simple" image, then the following neck head D1 will output the detection result y as: If the router classifies the input image as a "complex" image, the multi-scale features will need to be further enhanced by a second backbone, embedding the multi-scale features F1 into H through a composite connection module G: Among them, G is the DHLC that implements CBNet; the input image x is sent to the second backbone, and the features of the second backbone are enhanced by sequentially summing the corresponding elements of the embedded H at each stage, recorded as: Test results, the second head and neck D2 decoding is: y=D ₂ (F1). 4. A method of automatic driving target recognition based on new dynamic cascade YOLOv8 according to claim 1, characterized in that the ShareSepHead detection head in step (2) shares convolution weights between different layers and calculates them independently. Statistics of BN; the ShareSepHead includes a first convolution layer, a first depth-separable convolution layer, a second depth-separable convolution layer, a second convolution layer and a BN normalization layer connected in sequence. 5. A method of automatic driving target recognition based on new dynamic cascade YOLOv8 according to claim 4, characterized in that the first convolution layer is a 3x3 convolution layer, and the number of channels of the input feature map is From x to c2*2; the first depthwise separable convolution layer first applies a convolution operation to each input channel separately, and then combines the features between channels; the second depthwise separable convolution layer will input the feature map The number of channels is reduced from c2 to c2; the second convolution layer is a 1*1 convolution layer, changing the number of channels of the input feature map from c2 to 4*self.reg_max; each detection head is normalized by BN to improve the gradient Propagation and training speed, BN normalization is performed by normalizing the data of each mini-batch. 6. A kind of automatic driving target recognition method based on new dynamic cascade YOLOv8 according to claim 1, characterized in that the improved PolyLoss in step (3) includes a combined loss function and a weighted binary cross-entropy loss; PolyLoss Binary cross-entropy loss and focal loss are combined to improve the balance between difficult samples and positive and negative samples by adjusting the weight and shape of the loss function; use weighted binary cross-entropy loss to calculate the difference between the prediction result and the true label The binary cross-entropy loss between the two measures the matching degree between the prediction and the real label; alpha_factor is introduced to weight the loss, so that the loss of positive samples and negative samples is adjusted to varying degrees in the calculation; a polynomial adjustment factor is added to increase Uncertainty in sample probability predictions. 7. A method of automatic driving target recognition based on new dynamic cascade YOLOv8 according to claim 2, characterized in that the "simple" image is a single target image; the "difficult" image is two kinds and two More than one target image. 8. An apparatus, characterized by comprising a memory and a processor, wherein: Memory for storing computer programs capable of running on the processor; A processor, configured to execute the steps of the automatic driving target recognition method based on the new dynamic cascade YOLOv8 as described in any one of claims 1-7 when running the computer program. 9. A storage medium, characterized in that a computer program is stored on the storage medium, and when the computer program is executed by at least one processor, the new dynamic cascade-based method as described in any one of claims 1-7 is implemented. YOLOv8 automatic driving target recognition method steps.