CN110852243B - Road intersection detection method and device based on improved YOLOv3 - Google Patents

Abstract

The invention relates to a road intersection detection method and device based on improved YOLOv3, which mainly comprises the steps of firstly obtaining a road image; then, carrying out network training to construct an improved YOLOv3 network model; the improved YOLOv3 network model comprises a feature extraction end and a feature detection end, wherein the feature detection end comprises a plurality of channels, and in each channel, the corresponding convolution module is firstly widened transversely to generate different feature maps, and then is longitudinally aggregated; and identifying the road image to be detected by adopting the improved YOLOv3 network model, and outputting a result. The invention firstly expands the convolution module in each channel of the improved YOLOv3 feature detection end transversely to generate different feature maps, and then carries out longitudinal aggregation, so that the network width of the convolution module of each channel is wider, the expression capability of the network is enhanced, the difficulty in detecting small-size road intersections in complex remote sensing scenes is reduced, and the detection precision is improved.

Description

A road intersection detection method and device based on improved YOLOv3

technical field

The invention belongs to the technical field of image processing, and in particular relates to a road intersection detection method and device based on improved YOLOv3.

Background technique

As the hub of road connections, road intersections provide important information such as accurate location, direction, and topological relationship for the rapid construction of road networks. In the process of road network extraction, due to the interference of various complex factors, the extracted roads will appear discontinuous. At this time, the location of the road intersection is used as the base point, and the information such as direction and topological relationship can be used to assist and guide the construction of the road network.

However, based on the characteristics that road intersections are generally small-shaped surface targets in remote sensing images, the commonly used detection algorithms are mainly based on texture, shape, grayscale and other characteristics to detect, which is simple for the background and the contour features are more obvious. However, it is very difficult to detect small-sized road intersections in complex remote sensing scenarios, and requires more manual intervention, and the degree of automation and detection accuracy is not high.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a road intersection detection method and device based on the improved YOLOv3, which is used to solve the problems that the detection of small-sized road intersections in complex remote sensing scenarios is very difficult and the detection accuracy is not high.

In order to solve the above-mentioned technical problems, the technical scheme of the present invention is: a road intersection detection method based on improved YOLOv3, comprising the following steps:

1), get the road image;

2), carry out network training, and construct an improved YOLOv3 network model; the improved YOLOv3 network model includes a feature extraction end and a feature detection end, and the feature detection end includes multiple channels, and in each channel, the corresponding convolution The module first widens horizontally, generates different feature maps, and then aggregates vertically;

3), using the improved YOLOv3 network model to identify the road image to be detected, and output the result.

The beneficial effects of the present invention are: by first widening the convolution module in each channel of the improved YOLOv3 feature detection end to generate different feature maps, and then performing vertical aggregation, the convolution module of each channel can be The network width is wider and the expressive ability of the network is enhanced, thereby reducing the difficulty of detecting small-sized road intersections in complex remote sensing scenarios and improving the detection accuracy.

Further, the activation function of the feature extraction end is:

μ为动量因子，ε为学习率，∈为常数，a_i的初始化值为0.25，变化范围在(0,1)之间。Among them, x _i is the eigenvalue;

μ is the momentum factor, ε is the learning rate, ∈ is a constant, the initial value of a _i is 0.25, and the variation range is between (0, 1).

Further, the manner of widening the convolution module of each channel includes: symmetrical widening and asymmetric widening.

The present invention also provides a road intersection detection device based on improved YOLOv3, comprising a processor and a memory, the processor is connected with a collection interface for acquiring road images; the processor executes the following method instructions stored in the memory:

1), get the road image;

2), carry out network training, and construct an improved YOLOv3 network model; the improved YOLOv3 network model includes a feature extraction end and a feature detection end, and the feature detection end includes multiple channels, and in each channel, the corresponding convolution The module first widens horizontally, generates different feature maps, and then aggregates vertically;

3), using the improved YOLOv3 network model to identify the road image to be detected, and output the result.

Further, the activation function of the feature extraction end is:

μ为动量因子，ε为学习率，∈为常数，a_i的初始化值为0.25，变化范围在(0,1)之间。Among them, x _i is the eigenvalue;

μ is the momentum factor, ε is the learning rate, ∈ is a constant, the initial value of a _i is 0.25, and the variation range is between (0, 1).

Further, the manner of widening the convolution module of each channel includes: symmetrical widening and asymmetric widening.

Description of drawings

Fig. 1 is the schematic diagram of the improved YOLOv3 network model of the present invention;

Fig. 2 is the symmetrical convolution module channel of the symmetrical widening of the improved YOLOv3 network model of the present invention;

Fig. 3 is the asymmetric convolution module channel of the asymmetric widening of the YOLOv3 network model of the present invention;

Fig. 4 is the multi-scale feature fusion schematic diagram of the YOLOv3 network model of the present invention;

Figure 5-1 is the test curve of the existing YOLOv3 network model;

Fig. 5-2 is the improved YOLOv3 network model test curve of the present invention;

Figure 6-1 is the result of road intersection detection under the occlusion environment of the existing YOLOv3 network model;

Fig. 6-2 is the road intersection detection result graph under the occlusion environment of the improved YOLOv3 network model of the present invention;

Figure 7-1 is the result of road intersection detection under the background color of the existing YOLOv3 network model;

Fig. 7-2 is the road intersection detection result graph under the environment with similar background color of the improved YOLOv3 network model of the present invention;

Figure 8-1 is the detection result of the dense road intersection of the existing YOLOv3 network model;

FIG. 8-2 is a graph of the detection result of the dense road intersection of the improved YOLOv3 network model of the present invention.

Detailed ways

In order to elaborate the objectives, technical solutions and advantages of the present invention, the present invention will be further described in detail below with reference to specific implementation steps and accompanying drawings.

Embodiment of road intersection detection method

The invention provides a road intersection detection method based on improved YOLOv3, which mainly includes first collecting road images; performing network training to construct an improved YOLOv3 network model; the improved YOLOv3 network model includes a feature extraction end and a feature detection end, The feature detection end includes multiple channels. In each channel, the corresponding convolution module is first widened horizontally to generate different feature maps, and then vertically aggregated; the road image to be detected is processed by using the improved YOLOv3 network model. Identify and output the results.

Specifically, the road intersection detection method based on improved YOLOv3 of the present invention includes the following steps:

(1) Road image data acquisition.

Collected 3,106 images of 7 common road intersections including cross, T-shaped, X-shaped, Y-shaped, dislocation, ring, and multi-way through image database, AutoNavi Map, Google Map and other data platforms. Between 2.5 meters, the size is 416 × 416 pixels.

In computer vision, planar objects are usually expressed in the form of bounding boxes. In order to obtain the accurate frame label information of the target, the LambIng platform is used to manually mark the target one by one, obtain the coordinates of the center point, width and height of the frame, and the category to which it belongs, and store it in the xml file corresponding to the image.

Finally, the entire dataset is divided into test set, validation set and training set in a ratio of 1:2:7.

(2) Build an improved YOLOv3 network model and perform network training on it.

The YOLOv3 network belongs to an end-to-end target detection network, including the feature extraction end Darknet and the feature detection end yolo. Among them, the feature extraction end Darknet is a deep network composed of 52 convolution modules and 23 residual modules. Its task is to extract features layer by layer from the original image to form semantic feature maps of different scales.

The improved YOLOv3 network model of this embodiment is further improved on the basis of the existing YOLOv3 network, and the improvement has the following two points:

1. The PReLU function is used in the convolution module to activate the convolution layer.

Compared with the LReLU function that uses a linear function with a fixed small slope to map negative features into weak features, the PReLU function automatically adjusts the slope of the linear function according to the data characteristics and retains more negative features related to the target.

The PReLU function definition expression is as follows:

As shown in formula (1), when the eigenvalue is greater than 0, the PReLU function performs an identity mapping, and when the eigenvalue is less than 0, a non-fixed linear mapping is performed.

During backpropagation of the neural network, the slope a _i is updated by utilizing the momentum factor and the learning rate of the network. In formula (2), μ is the momentum factor, ε is the learning rate, ∈ is a constant, the initial value of a _i is 0.25, and the variation range is between (0, 1).

The PReLU function in this embodiment, in a complex remote sensing scene, if there are many similar interference factors in the detection frame, it will continuously increase the slope of the linear function in the process of backpropagation, improve the attention of negative features, and convert road Intersection features are correlated with disturbance factors. When the target is classified and regressed at the feature detection end, the semantic information of the context and negative features can be integrated to solve the problem of road intersection detection under similar interference factors to a certain extent.

2. The ways of widening the convolution module of each channel in the feature detection end include: symmetrical widening and asymmetric widening.

Specifically, three parallel convolution layers are set in the convolution layer before the output layer Contact. As shown in Figure 2, the size of the convolution kernel is 3 × 3, 1 × 1, 3 × 3, 3 × 3 in turn. ; that is, 3 convolution channels are constructed horizontally with 1×1 and 3×3 convolution kernels, and each channel generates feature maps of different sizes after convolution operations. The BN layer and the PReLU activation function are added after each convolutional layer. The BN layer is a regularization function that keeps the input of each layer of the neural network in the same distribution during the training process of the deep neural network, which can improve the speed of gradient descent and Accuracy, the PReLU activation function performs linear activation on the feature data. Finally, the three convolution layers output feature maps of the same size with the same number of channels; in order to ensure the same size, the SAME algorithm is used in the convolution operation to fill the feature map border (or the setting of border zero padding), and then the feature maps of the same size are used. Input to the output layer Contact for merging.

The BN function and the PReLU function in the above embodiment are set after each convolution kernel, which is actually combined with the convolution layer. It can also be considered that the convolution layer includes the BN layer and the PReLU function.

The three parallel convolutional layers in the above embodiment form a symmetric convolutional module, as shown in FIG. 3 . Of course, as other implementations, an asymmetric convolution module can be constructed by replacing one of the above 3×3 convolution kernels with 1×3 and 3×1 convolution kernels.

It should be noted that, when the size of the feature map in this embodiment is 12×12 to 12×20, the extraction effect of the asymmetric convolution module is better than that of the symmetric convolution module, and the amount of network parameters can be reduced by 1/3. Therefore, in the present invention, in the first detection channel with a feature map size of 13×13, an asymmetric convolution module is used; while other large-scale channels use a symmetric convolution module.

Therefore, the overall structure of the improved YOLOv3 network in the present invention is shown in Figure 1. In the figure, Convolutional-0 represents a symmetric convolution module, Convolutional-1 represents an asymmetric convolution module, and Convolutional-Set is a set of 5 convolution modules. combination.

In addition, in this embodiment, a multi-scale fusion method is used in the feature extraction of road images; the multi-scale fusion method includes bottom-up paths, top-down paths and lateral connections; The path is the feedforward calculation of the network Darknet, a feature hierarchy composed of feature maps of multiple scales, and its scaling step is 2; the output of the last layer of the same network stage is selected as the reference feature map set; from top to bottom The lower paths are fused by feature scale upscaling transfer and then these features are augmented from the bottom-up path by the lateral connections; each lateral connection merges features of the same spatial size from the bottom-up and top-down paths picture.

Specifically, as shown in Figure 4, for the multi-scale fusion in this embodiment, one is to input the 13×13 feature map output by the last layer of the Darknet network into the first channel in YOLO to detect large targets; two The feature map of size 13×13 is passed down to the second channel, and after upsampling operation, it is fused with the 26×26 feature map output by Darknet, and then the second largest target is detected; the third is to use the second channel. The feature map of the third channel continues to be passed to the third channel, and after upsampling, it is fused with the 52 × 52 feature map to detect the sub-small target; fourth, the 52 × 52 feature map of the third channel is upsampled for the last time, and the 104 × 104 The feature map is fused to detect small targets, and the feature extraction after multi-size fusion is realized.

Based on the improved YOLOv3 network model established above, the training set in the road image data is used, and the model obtained by the Darknet network training on the MSCOCO data set is used at the feature extraction end to initialize the parameters.

Among them, the total number of iterations of training is set to 30,000 times, the Adam optimization algorithm is used to update the weight gradient, the iteration batch size is 32, the momentum parameter is 0.9, the learning rate for the first 20,000 iterations is 0.0001, and the learning for the last 10,000 iterations rate dropped to 0.00005.

Specifically, the training process of the improved YOLOv3 network model is as follows:

1) Import training data: Generate training file train.txt, verification file val.txt, and test file test.txt from sample images in order and proportion, and import these three files together with labeled data and sample images as training data into the network;

2) Select the pre-training model: The model obtained by training the Darknet network on the MSCOCO dataset is used as the pre-training model to initialize the network parameters on the feature extraction side.

3) Network training: Iterative training of the network is performed. During the training process, the real-time training accuracy rate, loss value, recall rate and other indicators are stored in the training log.

4) Finally, the improved YOLOv3 network model after training is obtained.

3. Use the trained and improved YOLOv3 network model to detect road intersections on the road image (test set) to be tested.

In order to verify the effectiveness of the algorithm, in the present invention, the network before and after the improvement is trained and optimized on the road intersection data set, and then multiple types of intersections are detected on the multi-source remote sensing image.

In order to accurately evaluate the algorithm of the improved YOLOv3 network of the present invention, the detection performance of the model is measured by the comprehensive accuracy rate, recall rate, PR curve, average accuracy rate AP (Average-Precision) and average accuracy rate mAP.

The formula for calculating the index is:

Among them, TP is the number of road intersections detected correctly; FP is the number of road intersections whose detection results do not match the actual; FN is the number of road intersections that are not detected; N is the number of categories of road intersections; _Pre is the accurate detection R ec is the percentage of correctly detected road intersections in all detection results; R _ec is the recall rate, which indicates the percentage of correctly detected road intersections in all road intersections; the PR curve combines the accuracy and The evaluation standard of the full rate is the recall rate as the abscissa axis, the accuracy as the ordinate axis, the curve and the area of the besieged city on the abscissa and ordinate axes as the AP value; mAP is the average accuracy rate, which is used to measure the network's response to the entire test set. detection effect.

Figure 5-1 and Figure 5-2 show the PR curves drawn by the model trained on the YOLOv3 network before and after the improvement on the test set based on the accuracy and recall. The test set covers crossing, tjunction, There are seven types of road intersections: malposed, X-shape, Y-shape, roundabout and multiple.

Compared with the PR curve in Figure 5-1, the PR curve in Figure 5-2 is smoother, the detection accuracy Pre and the recall rate _Rec are both improved to a certain extent, and the average accuracy rate _mAP is increased by 3.17%. The relative sizes of different types of road intersections in the remote sensing images are different. The smallest size is the T-shaped intersection, and the detection effect is improved most obviously, and the AP value is increased by 8%. For Y-type 4-type road intersections, the AP value is increased by 3.01%, 3.25%, 4.5%, and 2.08%, respectively; the test results of the largest ring and multi-way intersection on the two network models are similar, and the AP value is increased by less than 1 %.

The experimental results show that: using the road intersection test set as the experimental data, the improved YOLOv3 network in this paper has strong robustness, and the test results are relatively stable. The expression ability has been significantly improved.

In order to verify the effectiveness of the algorithm for road intersection detection in complex environments, images with more interference factors were selected in the test set and tested based on the YOLOv3 network model before and after improvement.

For cement roads in township and residential areas, some road intersections are blocked by surrounding trees, and the contour features are incomplete. The test based on the YOLOv3 network model before and after improvement is carried out. The test results are shown in Figure 6-1 and Figure 6-2; The adjacent buildings have similar colors, and the road intersections with insignificant road intersection characteristics under the interference of the ground object background are tested based on the YOLOv3 network model before and after the improvement. The detection results are shown in Figure 7-1 and Figure 7-2. Based on the detection results of the two images by the YOLOv3 network before the improvement, as shown in Figures 6-1 and 7-1, a total of 9 road intersections affected by the interference were not identified, and the missed detection rate was high; based on the improved YOLOv3 The detection results of the network, as shown in Figures 6-2 and 7-2, can accurately identify some road intersections with inconspicuous contour features, and improve the detection effect of road intersections in complex environments.

In order to verify the applicability of the algorithm, city images were randomly intercepted on the Google Earth platform as test data, and migration tests were carried out based on the YOLOv3 model before and after the improvement. The test results are shown in Figure 8-1 and Figure 8-2.

The spatial resolution of the images in Figures 8-1 and 8-2 is about 1 meter, and the corresponding solid area is 1760m×1096m, including 65 road intersections of four types: cross, T-shaped, misplaced and X-shaped. Due to the small number of dislocations and X-shaped road intersections in the image, to avoid the problem of falsely high indicators, all types of road intersections are classified into one category when calculating the recall rate and average accuracy. The statistics of each evaluation index are shown in Table 1. Show.

Table 1 Comparison of detection performance between YOLOv3 and improved YOLOv3

It can be seen from Table 1 that compared with the YOLOv3 algorithm, the improved YOLOv3 algorithm in this paper has obvious advantages in the detection of dense road intersections, reducing the number of missed detections and false detections, and the average accuracy rate is increased by 12.18%. The experimental results show that the improved YOLOv3 algorithm in this paper can effectively migrate the features of road intersections in the dataset, and has strong applicability to other road images. From the statistical results, the average accuracy of road intersection detection is high, which can provide auxiliary information for the rapid construction of road network.

The embodiment of the detection device of the road intersection

The present invention also provides a road intersection detection device based on improved YOLOv3. The device is actually a computer and other equipment with data processing capabilities. The device includes a processor and a memory, and the processor is connected to a collection interface for acquiring road images. , the processor can be a general-purpose processor, or a digital signal processor, an application-specific integrated circuit, etc., the processor is used to execute instructions to implement the road intersection detection method of the present invention. No longer.

While the content of the present invention has been described in detail by way of the above preferred embodiments, it should be appreciated that the above description should not be construed as limiting the present invention. Various modifications and alternatives to the present invention will be apparent to those skilled in the art upon reading the foregoing. Accordingly, the scope of protection of the present invention should be defined by the appended claims.

Claims (4)

1. a road intersection detection method based on improving YOLOv3, is characterized in that, comprises the following steps: 1), get the road image; 2), carry out network training, and construct an improved YOLOv3 network model; the improved YOLOv3 network model includes two parts, a feature extraction end Darknet and a feature detection end yolo, and the feature detection end yolo includes four channels, and in each channel, The corresponding convolution modules are first widened horizontally to generate different feature maps, and then vertical aggregation is performed; The convolution module is a symmetric convolution module or an asymmetric convolution module; the convolution kernel is 3 × 3, 1 × 1 and 3 × 3 parallel convolution layers to form a symmetric convolution module. A 3×3 convolution kernel in the symmetric convolution module is replaced with 1×3 and 3×1 convolution kernels to form an asymmetric convolution module; each channel generates feature maps of different sizes after convolution operations; for features Channels whose image size is in the range of 12×12 to 12×20 use an asymmetric convolution module, and channels of other scales use a symmetric convolution module; When performing feature extraction, a multi-scale fusion method is used, specifically: first, input the feature map with a size of 13 × 13 output from the last layer of the Darknet network into the first channel in YOLO to detect large targets; second The feature map of size 13 × 13 is passed down to the second channel, and after upsampling operation, it is fused with the feature map of size 26 × 26 output by the Darknet network, and then the second largest target is detected; Continue to pass the feature map of the second channel to the third channel, and fuse it with the feature map with a size of 52×52 after upsampling to detect the second-smallest target; fourth, the size of the third channel is 52×52. The feature map is up-sampled for the last time, and it is fused with the feature map of the fourth channel with a size of 104×104 to detect small objects, and the feature extraction after multi-scale fusion is realized; 3), using the improved YOLOv3 network model to identify the road image to be detected, and output the result. 2. the road intersection detection method based on improved YOLOv3 according to claim 1, is characterized in that, the activation function of described feature extraction end is:

Among them, x _i is the eigenvalue;

μ is the momentum factor, ε is the learning rate, ∈ is a constant, the initial value of a _i is 0.25, and the variation range is between (0, 1). 3. a road intersection detection device based on improving YOLOv3, comprises a processor and a memory, and the processor is connected with an acquisition interface that is used to obtain road images; it is characterized in that, the processor executes the following method instructions stored in the memory: 1), get the road image; 2), carry out network training, and construct an improved YOLOv3 network model; the improved YOLOv3 network model includes two parts, a feature extraction end Darknet and a feature detection end yolo, and the feature detection end yolo includes four channels, and in each channel, The corresponding convolution modules are first widened horizontally to generate different feature maps, and then vertical aggregation is performed; The convolution module is a symmetric convolution module or an asymmetric convolution module; the convolution kernel is 3×3, 1×1 and 3×3 parallel convolution layers to form a symmetric convolution module. A 3×3 convolution kernel in the symmetric convolution module is replaced with 1×3 and 3×1 convolution kernels to form an asymmetric convolution module; each channel generates feature maps of different sizes after convolution operations; for features Channels whose image size is in the range of 12×12 to 12×20 use an asymmetric convolution module, and channels of other scales use a symmetric convolution module; When performing feature extraction, a multi-scale fusion method is used, specifically: first, input the feature map with a size of 13 × 13 output from the last layer of the Darknet network into the first channel in YOLO to detect large targets; second The feature map of size 13 × 13 is passed down to the second channel, and after upsampling operation, it is fused with the feature map of size 26 × 26 output by the Darknet network, and then the second largest target is detected; Continue to pass the feature map of the second channel to the third channel, and fuse it with the feature map with a size of 52 × 52 after upsampling to detect the second-smallest target; fourth, the size of the third channel is 52 × 52. The feature map is up-sampled for the last time, and then fused with the feature map of the fourth channel with a size of 104×104 to detect small targets, realizing feature extraction after multi-scale fusion; 3), using the improved YOLOv3 network model to identify the road image to be detected, and output the result. 4. the road intersection detection device based on improved YOLOv3 according to claim 3, is characterized in that, the activation function of described feature extraction end is:

Among them, x _i is the eigenvalue;

μ is the momentum factor, ε is the learning rate, ∈ is a constant, the initial value of a _i is 0.25, and the variation range is between (0, 1).

Images