The High-Precision Detection Method for Insulators’ Self-Explosion Defect Based on the Unmanned Aerial Vehicle with Improved Lightweight ECA-YOLOX-Tiny Model
<p>The flow chart of insulator defect detection based on UAV.</p> "> Figure 2
<p>The structure diagrams of SE and ECA: (<b>a</b>) SE module (<b>b</b>) ECA module.</p> "> Figure 2 Cont.
<p>The structure diagrams of SE and ECA: (<b>a</b>) SE module (<b>b</b>) ECA module.</p> "> Figure 3
<p>The network structure of improved ECA-YOLOX-Tiny.</p> "> Figure 4
<p>Data augmentation: (<b>a</b>) original image, (<b>b</b>) mirroring, (<b>c</b>) rotation 180°, (<b>d</b>) cropping and scaling, (<b>e</b>) adjust contrast, (<b>f</b>) adjust brightness.</p> "> Figure 5
<p>The training loss curve of the ECA-YOLOX-Tiny model.</p> "> Figure 6
<p>The <span class="html-italic">PR</span> curve and <span class="html-italic">AP</span> value of defective and normal insulators: (<b>a</b>) <span class="html-italic">PR</span> curve and <span class="html-italic">AP</span> value of defective insulators (<b>b</b>) <span class="html-italic">PR</span> curve and <span class="html-italic">AP</span> value of normal insulators.</p> "> Figure 7
<p>The recall curve of defective and normal insulators: (<b>a</b>) recall curve of defective insulators, (<b>b</b>) recall curve for normal insulators.</p> "> Figure 8
<p>The <span class="html-italic">F</span>1 score curve of defective and normal insulators: (<b>a</b>) <span class="html-italic">F</span>1 score curve of defective insulators, (<b>b</b>) <span class="html-italic">F</span>1 score curve of normal insulators.</p> "> Figure 9
<p>The class activation mapping process of an insulator image.</p> "> Figure A1
<p>The structure diagram of CSPLayer and Res Unit: (<b>a</b>) CSPLayer structure (<b>b</b>) Res Unit.</p> "> Figure A2
<p>The focus network structure and its feature map transformations: (<b>a</b>) focus network structure, (<b>b</b>) the transformation of feature maps.</p> "> Figure A3
<p>The structure diagram of SPPBottleneck.</p> "> Figure A4
<p>The enhanced feature extraction module fixed by FPN and PAN.</p> "> Figure A5
<p>The comparison between coupled head and decoupled head.</p> "> Figure A6
<p>Some detections missed by ECA-YOLOX-Tiny: (<b>a</b>) the missing detection of insulator defect target, (<b>b</b>) The missing detection of overlap small target 1, (<b>c</b>) the missing detection of overlap small target 2, (<b>d</b>) the missing detection of serious occlusion target.</p> "> Figure A6 Cont.
<p>Some detections missed by ECA-YOLOX-Tiny: (<b>a</b>) the missing detection of insulator defect target, (<b>b</b>) The missing detection of overlap small target 1, (<b>c</b>) the missing detection of overlap small target 2, (<b>d</b>) the missing detection of serious occlusion target.</p> ">
Abstract
:1. Introduction
2. The Construction of ECA-YOLOX-Tiny Network Model
2.1. The Lightweight Attention Module Introduction
2.2. Adjustment of Input Image Resolution
2.3. Optimization of the YOLOX-Tiny Model
3. The Construction of the Dataset
4. Case Verification and Analysis
4.1. Running Environment and Parameter Settings
4.2. The Evaluation Index
4.3. Training Model
4.4. The Analysis of Detection Effect before and after the Model Improved
4.5. The Analysis of Detection Effect before and after the Model Improved
4.6. Visual Analysis of Model Decision Areas
5. Conclusions
- (1)
- Aiming at the problem of insufficient defective samples in the insulator data set, the original data are enhanced by various methods according to the flight shooting conditions of UAV. Through the data enhancement measures, the generalization capability of the network is improved and the characteristic information of the insulators can be fully learned by the model, and then the detection accuracy of the model is improved for small target defective regions and multi-target insulators with different scales.
- (2)
- The higher resolution images are adopted as the network model inputs, so that the feature information of the insulator defect regions as well as small target insulator strings can be better learned by the network model. The examples prove that the detection accuracy indexes of mAP, Pr, Re and F1 score are improved after the resolution of the input images is improved from 416 × 416 to 640 × 640.
- (3)
- By introducing the lightweight attention module (ECA) in the ECA-YOLOX-Tiny model, the semantic information of the feature maps is further enhanced, the nonlinear capability of the network is improved, and the redundant information is reduced. Compared with YOLOX-Tiny_640 model, the mAP of the ECA-YOLOX-Tiny model has been improved by 0.11% due to the introduction of the ECA module, and the mAP is further improved by 0.22% when the model training method of equal interval “step” decay learning rate is replaced by the adaptive cosine annealing learning rate. Compared with the similar models YOLOV4-Tiny and SSD-MobileNet, the mAP of ECA-YOLOX-Tiny has been improved by 1.19% and 2.99%, respectively, and the number of parameters of the ECA-YOLOX-Tiny model can be compressed to best meet the hardware conditions of the UAV.
- (4)
- With the improvement of detection accuracy, the speed indexes of training time and FPS value are certainly decreased, however, it is necessary to improve the model detection accuracy at the cost of less training time and FPS value in the face of high precision detection requirements. The testing results show that the detection speed of the ECA-YOLOX-Tiny model reached 2.10 pictures s−1, which basically meets the requirements of real-time insulator detection.
- (5)
- In order to further verify the superiority of the ECA-YOLOX-Tiny model, the class activation mapping is introduced to visualize the prediction results of different models. Compared with the YOLOV4-Tiny model, the ECA-YOLOX-Tiny model is more accurate in locating the self-explosion areas of defective insulators and has a higher response degree for decision areas. Moreover, the ECA-YOLOX-Tiny model has a higher response to complex backgrounds of overlapping small target insulators, insulators obscured by tower poles, and high similarity background insulators, etc.
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Appendix A
- Backbone Feature Extraction Network (Backbone)
- 2.
- Enhanced Feature Extraction Network (Neck)
- 3.
- Prediction
- (a)
- Decoupled Head
- (b)
- Anchor-free
- (c)
- SimOTA
Appendix B
Models | Characteristics | Who and When | References |
---|---|---|---|
SSD-MobileNetV1 | The accuracy was 59.29%, which was relatively low | Ghoury, S. et al. (2019) | [16] |
SSD-MnasNet | The accuracy was 93.8%, which was high. The model size was 43.73 MB, FPS reached to 36.85 on Server, and the FPS reached to 8.27 on TX2. | Liu, X. et al. (2020) | [21] |
YOLOV3-Tiny | The accuracy was 92.10%, the recall rate was 92.20%, and FPS reached 30. | Han, J. et al. (2020) | [33] |
YOLOV4-Tiny | The mAP was 92.04%, and FPS reached 40. | Qiu, Z. et al. (2021) | [23] |
YOLOV4-MobileNetV1 | The mAP was 97.26%, and FPS reached 53. | Qiu, Z. et al. (2022) | [22] |
YOLOV4-MobileNetV3 | The mAP was 91.3%, and the model size was 116 MB. | Wu, J. et al. (2022) | [34] |
References
- Yang, L.; Fan, J.; Liu, Y.; Li, E.; Peng, J.; Liang, Z. A review on state-of-the-art power line inspection techniques. IEEE Trans. Instrum. Meas. 2020, 69, 9350–9365. [Google Scholar] [CrossRef]
- Sampedro, C.; Martinez, C.; Chauhan, A.; Campoy, P. A supervised approach to electric tower detection and classification for power line inspection. In Proceedings of the 2014 International Joint Conference on Neural Networks (IJCNN), Beijing, China, 6–11 July 2014; pp. 1970–1977. [Google Scholar] [CrossRef]
- Tao, X.; Zhang, D.; Wang, Z.; Liu, X.; Zhang, H.; Xu, D. Detection of power line insulator defects using aerial images analyzed with convolutional neural networks. IEEE Trans. Syst. Man Cybern. Syst. 2018, 50, 1486–1498. [Google Scholar] [CrossRef]
- Varghese, A.; Gubbi, J.; Sharma, H.; Balamuralidhar, P. Power infrastructure monitoring and damage detection using drone captured images. In Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA, 14–19 May 2017; pp. 1681–1687. [Google Scholar] [CrossRef]
- Girshick, R.; Donahue, J.; Darrell, T.; Malik, J. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA, 23–28 June 2014; pp. 580–587. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Spatial pyramid pooling in deep convolutional networks for visual recognition. In Proceedings of the IEEE Transactions on Pattern Analysis and Machine Intelligence, Las Vegas, NV, USA, 1 September 2015; pp. 1904–1916. [Google Scholar] [CrossRef]
- Girshick, R. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision (ICCV), Santiago, Chile, 7–13 December 2015; pp. 1440–1448. [Google Scholar]
- Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In Proceedings of the Advances in Neural Information Processing Systems(NIPS), Montreal, QC, Canada, 7–12 December 2015; pp. 91–99. [Google Scholar]
- Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You Only Look Once: Unified, Real-Time Object Detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 26 June–1 July 2016; pp. 779–788. [Google Scholar]
- Redmon, J.; Farhadi, A. YOLO9000: Better, faster, stronger. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 7263–7271. [Google Scholar]
- Redmon, J.; Farhadi, A. YOLOv3: An incremental improvement. arXiv 2018, arXiv:1804.02767. [Google Scholar]
- Bochkovskiy, A.; Wang, C.Y.; Liao, H.Y.M. YOLOv4: Optimal speed and accuracy of object detection. arXiv 2020, arXiv:2004.10934. [Google Scholar]
- Kasper-Eulaers, M.; Hahn, N.; Berger, S.; Sebulonsen, T.; Myrland, Ø.; Kummervold, P.E. Short Communication: Detecting Heavy Goods Vehicles in Rest Areas in Winter Conditions Using YOLOv5. Algorithms 2021, 14, 114. [Google Scholar] [CrossRef]
- Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.; Fu, C.Y.; Berg, A.C. SSD: Single Shot MultiBox Detector. In Proceedings of the European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands, 11–14 October 2016; pp. 21–37. [Google Scholar] [CrossRef]
- Wen, Q.; Luo, Z.; Chen, R.; Yang, Y.; Li, G. Deep Learning Approaches on Defect Detection in High Resolution Aerial Images of Insulators. Sensors 2021, 21, 1033. [Google Scholar] [CrossRef] [PubMed]
- Ghoury, S.; Sungur, C.; Durdu, A. Real-Time Diseases Detection of Grape and Grape Leaves using Faster R-CNN and SSD-MobileNet Architectures. In Proceedings of the International Conference on Advanced Technologies, Computer Engineering and Science (ICATCES 2019), Alanya, Turkey, 26–28 April 2019; pp. 39–44. [Google Scholar]
- Yan, B.; Fan, P.; Lei, X.; Liu, Z.; Yang, F. A Real-Time Apple Targets Detection Method for Picking Robot Based on Improved YOLOv5. Remote Sens. 2021, 13, 1619. [Google Scholar] [CrossRef]
- Iandola, F.N.; Han, S.; Moskewicz, M.W.; Ashraf, K.; Dally, W.J.; Keutzer, K. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size. Computer Vision and Pattern Recognition (CVPR), Artificial Intelligence. arXiv 2016, arXiv:1602.07360. [Google Scholar]
- Zhang, X.; Zhou, X.; Lin, M.; Sun, J. ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, 18–23 June 2018; pp. 6848–6856. [Google Scholar]
- Wang, Y.; Jia, K.; Liu, P. Impolite Pedestrian Detection by Using Enhanced YOLOv3-Tiny. J. Artif. Intell. 2020, 2, 113–124. [Google Scholar] [CrossRef]
- Liu, X.; Li, Y.; Shuang, F.; Gao, F.; Zhou, X.; Chen, X.Z. ISSD: Improved SSD for insulator and spacer online detection based on UAV system. Sensors 2020, 20, 6961. [Google Scholar] [CrossRef] [PubMed]
- Qiu, Z.; Zhu, X.; Liao, C.; Shi, D.; Qu, W. Detection of Transmission Line Insulator Defects Based on an Improved Lightweight YOLOv4 Model. Appl. Sci. 2022, 12, 1207. [Google Scholar] [CrossRef]
- Qiu, Z.; Zhu, X.; Liao, C.; Shi, D.; Kuang, Y.; Li, Y.; Zhang, Y. Detection of bird species related to transmission line faults based on lightweight convolutional neural network. IET Gener. Transm. Distrib. 2021, 16, 869–881. [Google Scholar] [CrossRef]
- Ge, Z.; Liu, S.; Wang, F.; Li, Z.; Sun, J. YOLOX: Exceeding YOLO Series in 2021. arXiv 2021, arXiv:2107.08430. [Google Scholar]
- Hu, J.; Shen, L.; Sun, G. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 7132–7141. [Google Scholar]
- Woo, S.; Park, J.; Lee, J.Y.; Kweon, I.S. CBAM: Convolutional Block Attention Module. In Proceedings of the European conference on computer vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 3–19. [Google Scholar]
- Wang, Q.; Wu, B.; Zhu, P.; Li, P.; Zuo, W.; Hu, Q. ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13–19 June 2020; pp. 11531–11539. [Google Scholar] [CrossRef]
- Loshchilov, I.; Hutter, F. SGDR: Stochastic Gradient Descent with Warm Restarts. arXiv 2016, arXiv:1608.03983. [Google Scholar]
- Zhou, B.; Khosla, A.; Lapedriza, A.; Oliva, A.; Torralba, A. Learning deep features for discriminative localization. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 2921–2929. [Google Scholar]
- Feng, J.; Sun, Y.; Zhang, K.; Zhao, Y.; Ren, Y.; Chen, Y.; Zhuang, H.; Chen, S. Autonomous Detection of Spodoptera frugiperda by Feeding Symptoms Directly from UAV RGB Imagery. Appl. Sci. 2022, 12, 2592. [Google Scholar] [CrossRef]
- Chien, J.-C.; Lee, J.-D.; Hu, C.-S.; Wu, C.-T. The Usefulness of Gradient-Weighted CAM in Assisting Medical Diagnoses. Appl. Sci. 2022, 12, 7748. [Google Scholar] [CrossRef]
- Elfwing, S.; Uchibe, E.; Doya, K. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural Netw. 2018, 107, 3–11. [Google Scholar] [CrossRef] [PubMed]
- Han, J.; Yang, Z.; Xu, H.; Hu, G.; Zhang, C.; Li, H.; Lai, S.; Zeng, H. Search Like an Eagle: A Cascaded Model for Insulator Missing Faults Detection in Aerial Images. Energies 2020, 13, 713. [Google Scholar] [CrossRef]
- Wu, J.; Li, X.; Zhou, Y. An infrared image detection of power equipment based on super-resolution reconstruction and YOLOv4. J. Eng. 2022, 1–11. [Google Scholar] [CrossRef]
Hardware Configuration | Version or Value | Software Development Environment | Version |
---|---|---|---|
Operating system | Windows 10–64 bit | PyCharm Community Edition | 2020.3.5 |
Graphics card | NVIDIA GeForce GTX 1050 | Anaconda3 | 2020.11 |
Processor | Intel(R) Core (TM) i5–8300H CPU ≅ 2.30 GHz | Python | Python 3.6 |
Operating memory | 8 GB | Keras | Keras 2.2.4 |
Parameters | Value | Parameters | Value |
---|---|---|---|
Input shape | 640 × 640 | Momentum | 0.937 |
Optimizer | Adam | Freeze batch size | 32 |
Freeze epochs | 0~50 | Unfreeze batch size | 16 |
Unfreeze epochs | 50~100 | Maximum value of learning rate | 1 × 10−3 |
Confidence | 0.5 | Minimum value of learning rate | 1 × 10−5 |
Model | mAP (%) | Pr (%) | Re (%) | F1 (%) | TT (h) | FPS (Pictures s−1) |
---|---|---|---|---|---|---|
YOLOX-Tiny_416 | 99.54 | 98.525 | 98.6 | 98.5 | 82.5 | 4.89 |
YOLOX-Tiny_640 | 99.61 | 98.825 | 98.97 | 99.0 | 175.0 | 2.17 |
ECA-YOLOX-Tiny | 99.72 | 98.82 | 98.895 | 99.0 | 198.5 | 2.17 |
ECA-YOLOX-Tiny (ACLR) | 99.94 | 98.98 | 99.63 | 99.5 | 215.0 | 2.10 |
ECA-YOLOX-Tiny | YOLOX-Tiny_640 | YOLOX-Tiny_416 |
---|---|---|
ECA-YOLOX-Tiny | YOLOX-Tiny_640 | YOLOX-Tiny_416 |
---|---|---|
Model | Backbone | mAP (%) | Pr (%) | Re (%) | F1 (%) | Parameters (106) | Training Time (h) | FPS (Pictures s−1) |
---|---|---|---|---|---|---|---|---|
ECA-YOLOX-Tiny | CSPDarknet | 99.94 | 98.98 | 99.63 | 99.5 | 5.1 | 215.0 | 2.10 |
YOLOV4-Tiny | CSPDarknet53-Tiny | 98.75 | 95.93 | 96.12 | 96.0 | 5.9 | 64.5 | 4.87 |
SSD-MobileNet | MobileNetV1 | 96.95 | 99.905 | 62.77 | 76.0 | 6.4 | 52.5 | 6.53 |
Model | Test Images | Successful Detection | Missed Detection | False Detection | Acc (%) | Test Time (s) |
---|---|---|---|---|---|---|
ECA-YOLOX-Tiny | 584 | 578 | 6 | 0 | 98.97 | 418 |
YOLOV4-Tiny | 584 | 561 | 18 | 5 | 96.06 | 127 |
SSD-MobileNet | 584 | 371 | 213 | 0 | 63.53 | 122 |
ECA-YOLOX-Tiny | YOLOV4-Tiny | SSD-MobileNet |
---|---|---|
ECA-YOLOX-Tiny | YOLOV4-Tiny | SSD-MobileNet |
---|---|---|
ECA-YOLOX-Tiny | YOLOV4-Tiny |
---|---|
ECA-YOLOX-Tiny | YOLOV4-Tiny |
---|---|
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ru, C.; Zhang, S.; Qu, C.; Zhang, Z. The High-Precision Detection Method for Insulators’ Self-Explosion Defect Based on the Unmanned Aerial Vehicle with Improved Lightweight ECA-YOLOX-Tiny Model. Appl. Sci. 2022, 12, 9314. https://doi.org/10.3390/app12189314
Ru C, Zhang S, Qu C, Zhang Z. The High-Precision Detection Method for Insulators’ Self-Explosion Defect Based on the Unmanned Aerial Vehicle with Improved Lightweight ECA-YOLOX-Tiny Model. Applied Sciences. 2022; 12(18):9314. https://doi.org/10.3390/app12189314
Chicago/Turabian StyleRu, Chengyin, Shihai Zhang, Chongnian Qu, and Zimiao Zhang. 2022. "The High-Precision Detection Method for Insulators’ Self-Explosion Defect Based on the Unmanned Aerial Vehicle with Improved Lightweight ECA-YOLOX-Tiny Model" Applied Sciences 12, no. 18: 9314. https://doi.org/10.3390/app12189314