Research on Surface Defect Detection of Strip Steel Based on Improved YOLOv7
<p>Improved YOLOv7 network structure.</p> "> Figure 2
<p>Working principle of PConv. <math display="inline"><semantics> <mrow> <mi>*</mi> </mrow> </semantics></math> denotes spatial feature extraction.</p> "> Figure 3
<p>CA module.</p> "> Figure 4
<p>Working principle of SPD convolution module. (<b>a</b>) denotes the original feature map; (<b>b</b>) denotes the spatial-to-depth transformation; (<b>c</b>) denotes channel concatenation; (<b>d</b>) denotes an addition operation; (<b>e</b>) represents non-strided convolution.</p> "> Figure 5
<p>Six defects in NEU-DET dataset.</p> "> Figure 6
<p>Comparison chart of training results. (<b>a</b>) P-R curve of the original YOLOv7; (<b>b</b>) P-R curve of the improved YOLOv7.</p> "> Figure 7
<p>Comparison of Improved Algorithm Effects. (<b>a</b>) Comparison of detection effectiveness for Cr-type defects; (<b>b</b>) Comparison of detection effectiveness for In-type defects; (<b>c</b>) Comparison of detection effectiveness for Pa-type defects; (<b>d</b>) Comparison of detection effectiveness for Ps-type defects; (<b>e</b>) Comparison of detection effectiveness for Rs-type defects; (<b>f</b>) Comparison of detection effectiveness for Sc-type defects.</p> ">
Abstract
:1. Introduction
2. Methodology
2.1. Baseline Networks
2.2. Partial Conv (PConv)
2.3. Coordinate Attention (CA)
2.4. SPD
2.5. EIoU
3. Results and Discussion
3.1. Experimental Preparation
3.1.1. Dataset
3.1.2. Experimental Environment and Parameter Setting
3.1.3. Object Detection Evaluation
- mAP: The average recognition accuracy of all categories is reflected, and the calculation formula is:
- Among them, c represents the total number of categories in the image, i represents the number of detections, and AP represents the average recognition accuracy of a single category. [email protected] refers to the average value obtained by adding the average recognition accuracy AP of each category when IoU is set to 0.5.
- Precision: It reflects the accuracy of model detection, calculated using the formula:
- where TP is the true case and FP is the false certificate case.
- Recall: It represents the proportion of correctly predicted positive examples:
- where FN represents data that were mistakenly identified by the model as negative examples but were actually positive examples.
- FPS represents the number of image frames processed within one second, and the calculation formula is:
- where Processing time per frame represents the processing time of each frame, including the image preprocessing time, the model inference time, and the post-processing time.
- Params reflect the number of parameters occupied by the model’s memory.
- FLOPs reflect the computational complexity of the model.
3.2. Ablation Experiment
3.3. Comparative Experiment
3.3.1. Comparison Experiment of Improvement Effect
3.3.2. Comparisons of Different Attention Mechanism Modules
3.3.3. Comparisons of Different IoU Loss Functions
4. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Czimmermann, T.; Ciuti, G.; Milazzo, M.; Chiurazzi, M.; Roccella, S.; Oddo, C.M.; Dario, P. Visual-Based Defect Detection and Classification Approaches for Industrial Applications—A SURVEY. Sensors 2020, 20, 1459. [Google Scholar] [CrossRef] [PubMed]
- Fang, X.; Luo, Q.; Zhou, B.; Li, C.; Tian, L. Research Progress of Automated Visual Surface Defect Detection for Industrial Metal Planar Materials. Sensors 2020, 20, 5136. [Google Scholar] [CrossRef] [PubMed]
- Li, M.; Wan, S.; Deng, Z. Fabric defect detection based on saliency histogram features. Comput. Intell. 2019, 35, 517–534. [Google Scholar] [CrossRef]
- Luo, Q.; Fang, X.; Sun, Y.; Liu, L.; Ai, J.; Yang, C.; Simpson, O. Surface Defect Classification for Hot-Rolled Steel Strips by Selectively Dominant Local Binary Patterns. IEEE Access 2019, 7, 23488–23499. [Google Scholar] [CrossRef]
- Malek, A.S.; Drean, J.; Bigue, L.; Osselin, J. Optimization of automated online fabric inspection by fast Fourier transform (FFT) and cross-correlation. Text. Res. J. 2013, 83, 256–268. [Google Scholar] [CrossRef]
- Zhou, X.; Wang, Y.; Zhu, Q.; Mao, J.; Xiao, C.; Lu, X.; Zhang, H. A Surface Defect Detection Framework for Glass Bottle Bottom Using Visual Attention Model and Wavelet Transform. IEEE Trans. Ind. Inform. 2020, 16, 2189–2201. [Google Scholar] [CrossRef]
- Ma, J.; Wang, Y.; Shi, C.; Lu, C. Fast Surface Defect Detection Using Improved Gabor Filters. In Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece, 7–10 October 2018; pp. 1508–1512. [Google Scholar]
- Nguyen, H.T.; Nguyen, L.T.; Sidorov, D.N. A Robust Approach for Road Pavement Defects Detection and Classification. J. Comput. Eng. Math. 2016, 3, 40–52. [Google Scholar] [CrossRef]
- Kulkarni, R.; Banoth, E.; Pal, P. Automated surface feature detection using fringe projection: An autoregressive modeling-based approach. Opt. Laser Eng. 2019, 121, 506–511. [Google Scholar] [CrossRef]
- Pasadas, D.J.; Baskaran, P.; Ramos, H.G.; Ribeiro, A.L. Detection and Classification of Defects Using ECT and Multi-Level SVM Model. IEEE Sens. J. 2020, 20, 2329–2338. [Google Scholar] [CrossRef]
- Nguyen, V.H.; Pham, V.H.; Cui, X.; Ma, M.; Kim, H. Design and evaluation of features and classifiers for OLED panel defect recognition in machine vision. J. Inf. Telecommun. 2017, 1, 334–350. [Google Scholar] [CrossRef]
- Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You Only Look Once: Unified, Real-Time Object Detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 779–788. [Google Scholar]
- Liu, W.; Anguelov, D.; Erhan, D. SSD: Single Shot MultiBox Detector. In Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands, 11–14 October 2016; Springer: Amsterdam, The Netherlands, 2016; pp. 21–37. [Google Scholar]
- Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Trans. Pattern Anal. 2017, 39, 1137–1149. [Google Scholar] [CrossRef] [PubMed]
- He, K.; Gkioxari, G.; Dollár, P.; Girshick, R. Mask R-CNN. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 2980–2988. [Google Scholar]
- Fu, H.; Song, Q.; Gong, J.; Jiang, L.; Liu, Z.; Luan, Q.; Wang, H. Automatic detection and pixel-level quantification of surface microcracks in ceramics grinding: An exploration with Mask R-CNN and TransUNet. Measurement 2024, 224, 113895. [Google Scholar] [CrossRef]
- Redmon, J.; Farhadi, A. YOLOv3: An Incremental Improvement; Cornell University Library: Ithaca, NY, USA, 2018. [Google Scholar]
- Zheng, Z.; Zhao, J.; Li, Y. Research on Detecting Bearing-Cover Defects Based on Improved YOLOv3. IEEE Access 2021, 9, 10304–10315. [Google Scholar] [CrossRef]
- Glenn, J. YOLOv5-Master. Available online: https://github.com/ultralytics/yolov5 (accessed on 1 March 2021).
- Fan, J.; Wang, M.; Li, B.; Liu, M.; Shen, D. ACD-YOLO: Improved YOLOv5-based method for steel surface defects detection. IET Image Process 2023, 18, 761–771. [Google Scholar] [CrossRef]
- Zhang, J.; Sui, T.; Lin, B.; Lv, B.; Du, H.; Song, N. Quantification of micro-damage evolution process in ceramics through extensive analysis of multi-source heterogeneous data. Mater. Des. 2024, 237, 112600. [Google Scholar] [CrossRef]
- Zhang, Y.; Liu, X.; Guo, J.; Zhou, P. Surface Defect Detection of Strip-Steel Based on an Improved PP-YOLOE-m Detection Network. Electronics 2022, 11, 2603. [Google Scholar] [CrossRef]
- Wang, C.; Bochkovskiy, A.M.; Liao, H. YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors. In Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada, 18–22 June 2023; pp. 7464–7475. [Google Scholar]
- Gao, C.; Qin, S.; Li, M.; Lv, X. Research on Steel Surface Defect Detection with Improved YOLOv7 Algorithm. Comput. Eng. Appl. 2024, 600, 282–291. [Google Scholar]
- Chen, J.; Kao, S.H.; He, H.; Zhuo, W.; Wen, S.; Lee, C.H.; Chan, S.H.G. Run, Don’t Walk: Chasing Higher FLOPS for Faster Neural Networks. In Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada, 17–24 June 2023; pp. 12021–12031. [Google Scholar]
- Hou, Q.; Zhou, D.; Feng, J. Coordinate Attention for Efficient Mobile Network Design. In Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 20–25 June 2021; pp. 13708–13717. [Google Scholar]
- Sunkara, R.; Luo, T. No More Strided Convolutions or Pooling: A New CNN Building Block for Low-Resolution Images and Small Objects; Cornell University Library: Ithaca, NY, USA, 2022. [Google Scholar]
- Zheng, Z.; Wang, P.; Ren, D.; Liu, W.; Ye, R.; Hu, Q.; Zuo, W. Enhancing Geometric Factors in Model Learning and Inference for Object Detection and Instance Segmentation. IEEE Trans. Cybern. 2022, 52, 8574–8586. [Google Scholar] [CrossRef] [PubMed]
- Zhang, Y.; Ren, W.; Zhang, Z.; Jia, Z.; Wang, L.; Tan, T. Focal and efficient IOU loss for accurate bounding box regression. Neurocomputing 2022, 506, 146–157. [Google Scholar] [CrossRef]
- Hu, J.; Shen, L.; Sun, G. Squeeze-and-Excitation Networks. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 7132–7141. [Google Scholar]
- Woo, S.; Park, J.; Lee, J.Y.; Kweon, I.S. CBAM: Convolutional Block Attention Module. In Proceedings of the Computer Vision-ECCV 2018, Munich, Germany, 8–14 September 2018. [Google Scholar]
Base | PConv | CA | SPD | mAP (%) | FPS | Par (Mb) | GLOPs |
---|---|---|---|---|---|---|---|
√ 1 | 76.4 | 92.6 | 37.2 | 105.2 | |||
√ | √ | 78.6 | 99.0 | 32.7 | 84.9 | ||
√ | √ | √ | 79.3 | 95.2 | 32.9 | 85.3 | |
√ | √ | √ | √ | 80.4 | 90.9 | 33.9 | 82.2 |
Algorithm | mAP (%) | Cr (%) | In (%) | Pa (%) | Ps (%) | Rs (%) | Sc (%) |
---|---|---|---|---|---|---|---|
Yolov7 | 76.4 | 45.0 | 86.1 | 94.2 | 91.1 | 54.7 | 86.9 |
Yolov7 + SE | 77.1 | 46.5 | 87.8 | 96.2 | 90.1 | 52.7 | 89.5 |
Yolov7 + CBAM | 77.8 | 49.2 | 86.0 | 94.6 | 90.5 | 57.4 | 89.4 |
Yolov7 + CA | 78.1 | 49.7 | 85.8 | 95.1 | 89.4 | 57.6 | 91.2 |
Algorithm | mAP (%) | Cr (%) | In (%) | Pa (%) | Ps (%) | Rs (%) | Sc (%) |
---|---|---|---|---|---|---|---|
OurWork + CIoU | 79.3 | 52.4 | 88.0 | 93.6 | 93.6 | 62.9 | 85.5 |
OurWork + SIoU | 79.6 | 59.1 | 88.5 | 93.5 | 89.8 | 58.3 | 88.0 |
OurWork + WIoU | 79.5 | 57.0 | 88.5 | 95.3 | 91.9 | 58.2 | 86.1 |
OurWork + EIoU | 80.4 | 52.1 | 88.7 | 94.6 | 95.7 | 62.0 | 89.2 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lv, B.; Duan, B.; Zhang, Y.; Li, S.; Wei, F.; Gong, S.; Ma, Q.; Cai, M. Research on Surface Defect Detection of Strip Steel Based on Improved YOLOv7. Sensors 2024, 24, 2667. https://doi.org/10.3390/s24092667
Lv B, Duan B, Zhang Y, Li S, Wei F, Gong S, Ma Q, Cai M. Research on Surface Defect Detection of Strip Steel Based on Improved YOLOv7. Sensors. 2024; 24(9):2667. https://doi.org/10.3390/s24092667
Chicago/Turabian StyleLv, Baozhan, Beiyang Duan, Yeming Zhang, Shuping Li, Feng Wei, Sanpeng Gong, Qiji Ma, and Maolin Cai. 2024. "Research on Surface Defect Detection of Strip Steel Based on Improved YOLOv7" Sensors 24, no. 9: 2667. https://doi.org/10.3390/s24092667