CN109376580B - A deep learning-based identification method for power tower components - Google Patents

Abstract

The invention discloses an unmanned aerial vehicle power tower part identification method based on deep learning. The invention comprises the following steps: 1. collecting a picture containing an electric power tower component by using unmanned aerial vehicle carried image collection equipment; 2. selecting a proper number of pictures from the collected pictures for preprocessing, and manufacturing a training set, a verification set and a test set according to a certain proportion; 3. training the processed data set by using a modified Yolov2 algorithm; 4. and testing the test set by using the trained model and evaluating the result. Compared with the YOLOv2 algorithm, the improved YOLOv2 algorithm effectively improves the identification accuracy and speed of the model to the power tower part, and has better robustness.

Description

A deep learning-based identification method for power tower components

technical field

The invention belongs to the field of deep learning target recognition, and relates to a method for identifying parts of a power tower, in particular to a method for real-time identification of parts of a power tower based on deep learning.

Background technique

With the rapid development of the UAV industry, the application of UAVs in power inspection operations has received extensive attention. UAV power inspection will generate a large amount of image data including power tower components, which requires a lot of time if only human interpretation is required. Therefore, it is of great significance to use the target recognition algorithm to automatically detect and identify the power tower components in these picture data. Due to the special operation mode and complex operating environment of UAV power inspection, compared with common identification targets such as pedestrians and vehicles, the background of the image obtained during the UAV inspection process is more complex, and the target to be identified is different from the background. Contrast is low, and is often accompanied by large distractions. Traditional electrical equipment identification algorithms rely on manually extracted features, such as SIFT (Scale-Invariant Feature Transform), power line edge detection, HOG (Histogram of Oriented Gridients), and the obtained features are combined with algorithms such as support vector machines or random forests for identification. In addition, there are also image segmentation algorithms such as adaptive threshold and watershed to segment the peripheral contour of electrical equipment. However, due to the complex and irregular structure of electrical equipment, these methods achieve general results with low accuracy and poor generalization ability.

The proposal of Alexnet in 2012 made the application of deep learning in the field of image recognition and object detection attract much attention. The current target detection frameworks based on convolutional neural networks as the basis for feature extraction can be roughly divided into two categories. One class is a two-stage network based on region candidate boxes. The representative ones are R-CNN (Region based Convolutional Neural Network), Fast R-CNN, Faster R-CNN. The second category is one-stage network represented by YOLO (You Only Look Once) and SSD (Single Shot Multibox Detector). RedmonJ et al. proposed the YOLO algorithm, and YOLO lost a certain precision while pursuing speed. Later, YOLOv2 was proposed, which improved the problem of YOLO accuracy loss, and achieved good results in terms of recognition rate and accuracy, and was suitable for the application scenario of real-time recognition of UAV power tower components.

SUMMARY OF THE INVENTION

In order to quickly and accurately realize the identification of power tower components in the UAV power inspection, the present invention proposes a method for identifying power tower components based on deep learning. First, a large number of images containing power tower components are obtained through the image acquisition equipment mounted on the UAV; then images containing three types of targets to be identified, including power towers, signs, and tower bases, are selected for preprocessing, and produced according to a certain proportion. training set, validation set and test set; then use the improved YOLOv2 algorithm to train the data set; finally use the trained model to test the test set and evaluate the results.

The method of the present invention mainly comprises the following steps:

(1) UAV is used to carry image acquisition equipment to collect image information of power tower components in the process of power inspection.

(2). Three types of towers, signboards and tower bases are selected from the images of the power tower components collected in step (1) as identification objects. Images containing the above three types of objects are selected for preprocessing, and made into training, validation and test sets for subsequent training and testing.

(3). Use the improved YOLOv2 algorithm to train the training set prepared in step (2).

The improved YOLOv2 algorithm is as follows:

The improved YOLOv2 network structure contains 24 convolutional layers, 5 pooling layers and two transfer layers. 3×3 and 1×1 convolution kernels are used in the convolutional layers of the network. The convolutional layers with 3×3 convolution kernels and the convolutional layers with 1×1 convolution kernels are alternately placed, making full use of the 1×1 convolution kernel. 1 Convolution kernel compresses the effect of features, and doubles the number of convolution kernels after each pooling layer. The improved network structure removes the redundant two 3×3×1024 convolutional layers at the end of the YOLOv2 network, and reduces the number of convolution kernels in the Conv_3~Conv_6 convolutional layers by half.

The transfer layer consists of the route layer and the reorg layer. The route layer connects the feature maps of different layers, and the reorg is used to adjust the size of the feature map. The combination of the two realizes that the size of the feature maps of other layers is adjusted to be consistent with the size of the feature maps of the current layer, and then spliced to achieve feature fusion of feature maps of different sizes. The improved network structure utilizes two transfer layers at the end of the network to fuse the feature maps of size 26×26 and 52×52 with the feature map of size 13×13. The representation of the transfer layer is as follows:

X _n = fp ₁ +fp ₂ +...+fp _j (1)

X _m = fp ₁ +fp ₂ +...+fp _k (2)

X _n represents all feature maps from the nth convolutional layer, fp ₁ to fp _j correspond to j feature maps respectively, X _m represents all feature maps from the mth convolutional layer, and fp ₁ to fp _k correspond to k respectively feature map.

表示第n层卷积层的特征图通过隔行隔列采样后，将特征图尺寸调整为与第m层卷积层特征图的尺寸一致，S_n表示第n层特征图的尺寸为S_n×S_n，S_m表示第m层特征图尺寸为S_m×S_m，调整后的特征图数量为调整前的2^λ倍，X_merge表示第m、n层卷积层特征图融合得到的结果。

After the feature map of the nth layer convolutional layer is sampled by every row and column, the size of the feature map is adjusted to be consistent with the size of the feature map of the mth layer convolutional layer, and Sn indicates that the size of the _nth layer feature map is _Sn × S _n , S _m indicate that the size of the feature map of the mth layer is S _m ×S _m , the number of feature maps after adjustment is ^2λ times that before the adjustment, and X _merge represents the result obtained by the fusion of the feature maps of the mth and nth layers of convolutional layers .

Use the improved network structure to train until the model converges.

(4). Use the model obtained by training to test the test set, and use the mAP and P-R curves and the average test piece time of each picture to evaluate the obtained results; where mAP is Mean Average Precision, and P-R is Precision-Recall;

The definition of accuracy Precision:

TP is represented as a true class, that is, an instance is a positive class and is also predicted to be a positive class, and FP is a false positive class, that is, an instance is a negative class and is predicted to be a positive class. Precision reflects the ability of the model to predict positive examples.

The definition of recall rate Recall:

FN is represented as a false negative class, i.e. an instance is a positive class and is predicted to be a negative class. Recall reflects the detection ability of the model. The P-R curve reflects the trade-off between the accuracy of the classifier's recognition of positive examples and the ability to cover positive examples.

AP is the area of the graph enclosed by the PR curve and the X-axis, which is defined as:

mAP is defined as:

Q is the number of corresponding categories, that is, the average value of APs of various types, and mAP is the comprehensive ability of the model to recognize multi-category targets.

Compared with the existing method for identifying parts of electric power towers, the present invention has the following characteristics:

The deep learning target detection algorithm can make full use of the deep features of image data, and the model has stronger generalization ability. The improved YOLOv2 algorithm combines the features of different scale feature maps to improve the recognition accuracy; the simplification of the network structure speeds up the recognition rate of the model.

Description of drawings

Fig. 1 is the flow chart of the implementation of the present invention;

Figure 2 is a schematic diagram of the improved YOLOv2 network structure;

Figure 3 is the P-R curve of the three types of target test results.

Detailed ways

The embodiments of the present invention will be described in detail below with reference to the accompanying drawings: the present embodiment is implemented on the premise of the technical solution of the present invention, and provides a detailed implementation manner and a specific operation process.

As shown in Figure 1, this embodiment includes the following steps:

Step 1: UAVs are equipped with high-definition cameras to take a large number of pictures including three types of objects including power towers, signs, and tower bases during power inspection of high-voltage transmission lines.

Step 2: Select 1,200 pictures of three types of objects including power towers, signs, and tower bases obtained in step 1, and distinguish them by 1,000 pictures in the training set, 100 pictures in the verification set and 100 in the test set, and are independent of each other. All images are resized to 600×450, and the objects to be recognized contained in all images are marked for subsequent training and testing.

Step 3, the improved YOLOv2 network structure is shown in Figure 2. Compared with the original YOLOv2 network structure, the two 3×3×1024 convolution layers at the end of the network are removed, and the number of convolution kernels in the middle convolution layer of the network is calculated. The halving process is performed; in addition, two transfer layers are added to fuse feature maps of three sizes of 52×52, 26×26, and 13×13 at the end of the network. Use the original YOLOv2 algorithm and the improved YOLOv2 to train the prepared training set respectively, observe the accuracy of the validation set and the loss value of the training during the training process, and wait until the accuracy of the validation set and the loss value of the training become stable, that is Indicates that the model has converged, save the model, and stop training.

Step 4: Use the YOLOv2 algorithm and the improved YOLOv2 algorithm to test the test set respectively, and evaluate and compare the results.

Table 1 Comparison of test results between improved YOLOv2 and YOLOv2

Table 1 records the mAP, Recall, and average test time per photo of the models trained by the YOLOv2 algorithm and the improved YOLOv2 algorithm on the test set. It can be seen that the improved YOLOv2 has a higher recognition accuracy while having a higher recall rate, and the average test time per photo is also about 35% faster. Figure 3 is the P-R curve drawn according to the test results of the three types of targets in the test set, namely the signboard, the tower, and the tower base, respectively, based on the model trained by the YOLOv2 algorithm and the improved YOLOv2 algorithm. The P-R curve corresponding to the YOLOv2 test result is in the accuracy rate When it rises, the recall rate drops faster, indicating that the model is less robust, while the P-R curve corresponding to the improved YOLOv2 shows that the model has better robustness.

Therefore, the improved YOLOv2 algorithm not only improves the recognition accuracy and rate of power tower components, but also has better robustness of the trained model, which shows that the improved algorithm has more advantages in the identification of power tower components than the YOLOv2 algorithm. greater advantage.

Claims (1)

1. a power tower component identification method based on deep learning, is characterized in that, this method specifically comprises the steps: (1) UAVs are used to carry image acquisition equipment to collect image information of power tower components during the power inspection process; (2) Select three types of towers, signboards, and tower bases from the images of power tower components collected in step (1) as recognition objects; select images containing the above three types of objects for preprocessing, and make them into training set, validation set and test set for subsequent training and testing; (3). Use the improved YOLOv2 algorithm to train the training set prepared in step (2); The improved YOLOv2 algorithm is as follows: The improved YOLOv2 network structure contains 24 convolutional layers, 5 pooling layers and two transfer layers; 3×3, 1×1 convolution kernels are used in the convolutional layers of the network, including 3×3 convolution kernels The convolutional layers are alternately placed with the convolutional layers of 1×1 convolutional kernels, and the number of convolutional kernels is doubled after each pooling layer; the improved network structure removes the redundant end of the YOLOv2 network. Two 3×3×1024 convolutional layers, and the number of convolution kernels of Conv_3~Conv_6 convolutional layers is reduced by half; The transfer layer consists of the route layer and the reorg layer. The route layer connects the feature maps of different layers, and the reorg is used to adjust the size of the feature map; the combination of the two realizes that the size of the feature maps of other layers is adjusted to the current layer. The image size is the same, and then spliced to achieve feature fusion of feature maps of different sizes; the improved network structure uses two transfer layers at the end of the network to combine the 26×26, 52×52 size feature maps and the 13×13 size feature maps. Fusion; the representation of the transfer layer is as follows: X _n =fp _n1 +fp _n2 +...+fp _nj (1) X _m = fp _m1 +fp _m2 +...+fp _mk (2) X _n represents all feature maps from the nth convolutional layer, fp _n1 ~ fp _nj correspond to j feature maps respectively, X _m represents all feature maps from the mth convolutional layer, fp _m1 ~ fp _mk correspond to k feature map;

After the feature map of the nth layer convolutional layer is sampled by every row and column, the size of the feature map is adjusted to be consistent with the size of the feature map of the mth layer convolutional layer, and Sn indicates that the size of the _nth layer feature map is _Sn × S _n , S _m indicate that the size of the feature map of the mth layer is S _m ×S _m , the number of feature maps after adjustment is ^2λ times that before the adjustment, and X _merge represents the result obtained by the fusion of the feature maps of the mth and nth layers of convolutional layers ; Use the improved network structure to train until the model converges; (4). Use the model obtained by training to test the test set, and use the mAP and P-R curves and the average test piece time of each picture to evaluate the obtained results; where mAP is Mean Average Precision, and P-R is Precision-Recall; The definition of accuracy Precision:

TP is represented as a true class, that is, an instance is a positive class and is also predicted to be a positive class, FP is a false positive class, that is, an instance is a negative class and is predicted to be a positive class; Precision reflects the model’s ability to predict positive examples; The definition of recall rate Recall:

FN is represented as a false negative class, that is, an instance is a positive class and is predicted to be a parent class; Recall reflects the detection ability of the model; P-R curve reflects the accuracy of the classifier's recognition of positive examples and the ability to cover positive examples. balance between AP is the area of the graph enclosed by the PR curve and the X-axis, which is defined as:

mAP is defined as:

Q is the number of corresponding categories, that is, the average value of APs of various types, and mAP is the comprehensive ability of the model to recognize multi-category targets.

Images