Light-YOLO: A Study of a Lightweight YOLOv8n-Based Method for Underwater Fishing Net Detection
<p>Flowchart of the sub-network module of the DA2D module.</p> "> Figure 2
<p>Offset sub-network module diagram.</p> "> Figure 3
<p>CoTAttention module diagram.</p> "> Figure 4
<p>PCM module diagram.</p> "> Figure 5
<p>Block diagram of Light-YOLOv8.</p> "> Figure 6
<p>Schematic diagram of the GAN network principle.</p> "> Figure 7
<p>Comparison of a GAN-generated image as well as the original image; (<b>a</b>) Original image; (<b>b</b>) Generated image.</p> "> Figure 8
<p>Example of an image of the fishing net vulnerability dataset.</p> "> Figure 9
<p>Statistical results for this dataset: (<b>a</b>) Normalized target size plot; (<b>b</b>) Normalized target location plot.</p> "> Figure 10
<p>Various losses of the training process and graphs of some metrics.</p> "> Figure 11
<p>Visualization of underwater target recognition with different models: (<b>a</b>) YOLOv3-tiny; (<b>b</b>) YOLOv7-tiny; (<b>c</b>) Light-YOLO.</p> ">
Abstract
:1. Introduction
- (1)
- A dataset labelled with underwater fishing net vulnerabilities is created, and the dataset is expanded using GAN network generation techniques.
- (2)
- A fast, efficient feature processing module, DA2D, based on sparse linkage and variability convolution is proposed. Using this module, we designed the model to dynamically adjust the position of the attention sampling points according to the input content, allowing the model to focus on local features, reduce computational overhead, and effectively improve accuracy.
- (3)
- This CoTAttention module is introduced. This enhanced self-attention mechanism digs deeper into the contextual information of the key object by combining the local and global contextual information, effectively improving the model’s ability to recognize and model the features of the elongated gaps. The recognition accuracy of the model is improved.
- (4)
- The SEAM module is introduced to optimize the object boundary fitting and enhance the sensitivity of the model to contextual information by constructing a twin network structure with shared weights. The covariant regularization technique is used to ensure the consistency of the class activation maps (CAMs) under different transformations, and the PCM pixel correlation module is used to refine the CAMs.
2. Related Work
3. Model Improvement
3.1. An Improved Approach to Feature Fusion
3.2. Citing an Attention Mechanism
3.3. Integration of the SEAM Module
3.4. Block Diagram of Light-YOLO
4. Experimental
4.1. Fishing Net Vulnerability Dataset Collection and Construction
4.1.1. Expansion of the Fishing Net Vulnerability Dataset
4.1.2. Fishing Net Vulnerability Dataset
4.2. Experimental Environment and Training Strategy
4.3. Evaluation Indicators
4.4. Experimental Results
4.5. Comparison of Conventional Lightweight YOLOs
4.6. Comparison of Other Families
4.7. Model Visualization
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Chu, Y.-I.; Wang, C.-M.; Zhang, H.; Abdussamie, N.; Karampour, H.; Jeng, D.-S.; Baumeister, J.; Aland, P.A. Offshore Fish Farms: A Review of Standards and Guidelines for Design and Analysis. J. Mar. Sci. Eng. 2023, 11, 762. [Google Scholar] [CrossRef]
- Burguera, A.; Bonin-Font, F. Advances in Autonomous Underwater Robotics Based on Machine Learning. J. Mar. Sci. Eng. 2022, 10, 1481. [Google Scholar] [CrossRef]
- Ziaei Nafchi, H.; Cheriet, M. Efficient No-Reference Quality Assessment and Classification Model for Contrast Distorted Images. IEEE Trans. Broadcast. 2018, 64, 518–523. [Google Scholar] [CrossRef]
- Zhang, W.; Wang, Y.; Li, C. Underwater Image Enhancement by Attenuated Color Channel Correction and Detail Preserved Contrast Enhancement. IEEE J. Ocean. Eng. 2022, 47, 718–735. [Google Scholar] [CrossRef]
- Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 1137–1149. [Google Scholar] [CrossRef] [PubMed]
- Girshick, R.; Donahue, J.; Darrell, T.; Malik, J. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 23–28 June 2014; pp. 580–587. [Google Scholar] [CrossRef]
- Girshick, R. Fast R-CNN. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 7–13 December 2015; pp. 1440–1448. [Google Scholar] [CrossRef]
- Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.; Fu, C.-Y.; Berg, A.C. SSD: Single Shot MultiBox Detector. In Computer Vision—ECCV 2016; Leibe, B., Matas, J., Sebe, N., Welling, M., Eds.; Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2016; Volume 9905. [Google Scholar] [CrossRef]
- Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You Only Look Once: Unified, Real-Time Object Detection. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 779–788. [Google Scholar] [CrossRef]
- Li, G.; Ji, Z.; Qu, X.; Zhou, R.; Cao, D. Cross-Domain Object Detection for Autonomous Driving: A Stepwise Domain Adaptative yolo Approach. IEEE Trans. Intell. Veh. 2022, 7, 603–615. [Google Scholar] [CrossRef]
- Qin, L.; Shi, Y.; He, Y.; Zhang, J.; Zhang, X.; Li, Y.; Deng, T.; Yan, H. ID-yolo: Real-Time Salient Object Detection Based on the Driver’s Fixation Region. IEEE Trans. Intell. Transp. Syst. 2022, 23, 15898–15908. [Google Scholar] [CrossRef]
- Wang, Y.; Guo, J.; He, W.; Gao, H.; Yue, H.; Zhang, Z.; Li, C. Is Underwater Image Enhancement All Object Detectors Need? arXiv 2023, arXiv:2311.18814. [Google Scholar] [CrossRef]
- Zhang, J.; Zhang, J.; Zhou, K.; Zhang, Y.; Chen, H.; Yan, X. An Improved yolov5-Based Underwater Object-Detection Framework. Sensors 2023, 23, 3693. [Google Scholar] [CrossRef]
- Lyu, C.; Zhang, W.; Huang, H.; Zhou, Y.; Wang, Y.; Liu, Y.; Zhang, S.; Chen, K. RTMDet: An Empirical Study of Designing Real-Time Object Detectors. arXiv 2022, arXiv:2212.07784. [Google Scholar]
- Wang, C.; Liao, H.M.; Yeh, I.; Wu, Y.; Chen, P.; Hsieh, J. CSPNet: A New Backbone that can Enhance Learning Capability of CNN. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA, 14–19 June 2020; pp. 1571–1580. [Google Scholar]
- Srinivas, A.; Lin, T.; Parmar, N.; Shlens, J.; Abbeel, P.; Vaswani, A. Bottleneck Transformers for Visual Recognition. In Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville. TN, USA, 20–25 June 2021; pp. 16514–16524. [Google Scholar]
- Chen, J.; Er, M.J. Dynamic yolo for Small Underwater Object Detection. Artif. Intell. Rev. 2024, 57, 165. [Google Scholar] [CrossRef]
- Zhou, J.; He, Z.; Lam, K.-M.; Wang, Y.; Zhang, W.; Guo, C.; Li, C. AMSP-UOD: When Vortex Convolution and Stochastic Perturbation Meet Underwater Object Detection. In Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada, 20–28 February 2024; Volume 38, pp. 7659–7667. [Google Scholar] [CrossRef]
- Vaswani, A.; Shazeer, N.M.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, L.; Polosukhin, I. Attention is All You Need. Neural Inform. Process. Syst. 2017, 30. [Google Scholar]
- Dai, J.; Qi, H.; Xiong, Y.; Li, Y.; Zhang, G.; Hu, H.; Wei, Y. Deformable Convolutional Networks. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 764–773. [Google Scholar]
- Liu, L.; Deng, L.; Hu, X.; Zhu, M.; Li, G.; Ding, Y.; Xie, Y. Dynamic Sparse Graph for Efficient Deep Learning. arXiv 2018, arXiv:1810.00859. [Google Scholar]
- Li, Y.; Yao, T.; Pan, Y.; Mei, T. Contextual Transformer Networks for Visual Recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2021, 45, 1489–1500. [Google Scholar] [CrossRef] [PubMed]
- Wang, Y.; Zhang, J.; Kan, M.; Shan, S.; Chen, X. Self-Supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13–19 June 2020; pp. 12272–12281. [Google Scholar]
- Selvaraju, R.R.; Das, A.; Vedantam, R.; Cogswell, M.; Parikh, D.; Batra, D. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. Int. J. Comput. Vis. 2016, 128, 336–359. [Google Scholar] [CrossRef]
- Goodfellow, I.J.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.C.; Bengio, Y. Generative Adversarial Nets. arXiv 2014, arXiv:1406.2661. [Google Scholar]
- Redmon, J.; Farhadi, A. yolov3: An Incremental Improvement. arXiv 2018, arXiv:1804.02767. [Google Scholar]
- Wang, C.; Bochkovskiy, A.; Liao, H.M. yolov7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors. In Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada, 17–24 June 2023; pp. 7464–7475. [Google Scholar]
- Ultralytics. yolov5. Available online: https://github.com/ultralytics/yolov5 (accessed on 1 November 2021).
- Wang, A.; Chen, H.; Liu, L.; Chen, K.; Lin, Z.; Han, J.; Ding, G. YOLOv10: Real-Time End-to-End Object Detection. arXiv 2024, arXiv:2405.14458. [Google Scholar]
- Ge, Z.; Liu, S.; Wang, F.; Li, Z.; Sun, J. yoloX: Exceeding yolo Series in 2021. arXiv 2021, arXiv:2107.08430. [Google Scholar]
- Wang, C.; He, W.; Nie, Y.; Guo, J.; Liu, C.; Han, K.; Wang, Y. Gold-yolo: Efficient Object Detector via Gather-and-Distribute Mechanism. arXiv 2023, arXiv:2309.11331. [Google Scholar]
- Cai, Z.; Vasconcelos, N. Cascade R-CNN: Delving into High Quality Object Detection. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, 18–23 June 2018; pp. 6154–6162. [Google Scholar]
- Lv, W.; Xu, S.; Zhao, Y.; Wang, G.; Wei, J.; Cui, C.; Du, Y.; Dang, Q.; Liu, Y. DETRs Beat yolos on Real-time Object Detection. arXiv 2023, arXiv:2304.08069. [Google Scholar]
- Zhang, S.; Xinjiang, W.; Wang, J.; Pang, J.; Lyu, C.; Zhang, W.; Luo, P.; Chen, K. Dense Distinct Query for End-to-End Object Detection. In Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada, 17–24 June 2023; pp. 7329–7338. [Google Scholar]
Parameters | Generator | Discriminator |
---|---|---|
Learning rate | 0.002 | |
Batchsize | 32 | |
Epoch | 300 | |
Optimizer | Adam | |
Loss | BCEloss | |
Generator input noise | 100 |
Experimental Environment | Configure |
---|---|
GPUs | GeForce RTX 4090 |
CPU | Intel Core i5-10400F (2.9 GHz) |
Systems | Windows 11 Professional |
Framework (deep learning framework) | Torch-2.0.1 |
Programming language versions | Python-3.8.16 |
CUDA version (GPU parallel computing platform) | CUDA 11.7 |
cuDNN version (deep learning GPU acceleration library) | cuDNN 8.5.0 |
Compilers | pycharm |
Model | Depth | Width | Max Channels |
---|---|---|---|
YOLOv8n | 0.33 | 0.25 | 1024 |
YOLOv8s | 0.33 | 0.50 | 1024 |
YOLOv8m | 0.67 | 0.75 | 768 |
YOLOv8l | 1.00 | 1.00 | 512 |
YOLOv8x | 1.00 | 1.25 | 512 |
Parameters | Set Up |
---|---|
Epochs | 300 |
Optimizer | SGD |
Learning rate | 0.001 |
Momentum | 0.937 |
weight_decay | 0.0005 |
batch | 32 |
imgsz | 640 |
iou | 0.7 |
Baseline | Params (MB) | GFLOPs | |||||||
---|---|---|---|---|---|---|---|---|---|
CoTAttention | SEAM | DA2D | |||||||
YOLOv8n | 87.6 | 78.1 | 83.7 | 47.6 | 3.0 | 8.2 | |||
Light-YOLO | √ | 88.5 | 79.0 | 85.9 | 50.1 | 3.1 | 8.3 | ||
√ | √ | 88.7 | 79.1 | 86.0 | 50.1 | 3.1 | 8.3 | ||
√ | √ | √ | 89.3 | 80.7 | 86.7 | 51.8 | 4.4 | 6.1 |
Models | [email protected]/% | [email protected]–0.95 (%) | Params | GFLOPs | FPS | ||
---|---|---|---|---|---|---|---|
YOLOv3-tiny [26] | 91.4 | 77.1 | 83.5 | 43.3 | 17.4 | 12.9 | 434.78 |
YOLOv7-tiny [27] | 91.1 | 81.3 | 85.5 | 47.8 | 12.3 | 13.2 | 123.46 |
YOLOv5n [28] | 91.0 | 77.6 | 85.6 | 49.6 | 9.5 | 7.1 | 833.33 |
YOLOv5s | 90.0 | 77.0 | 84.1 | 49.2 | 34.7 | 23.8 | 263.15 |
YOLOv8n | 87.6 | 78.1 | 83.7 | 47.6 | 3.0 | 8.2 | 322.58 |
YOLOv10n [29] | 90.2 | 78 | 84.5 | 48.7 | 8.06 | 24.8 | 230.49 |
Light-YOLO | 89.3 | 80.7 | 86.7 | 51.8 | 4.4 | 6.1 | 121.95 |
Models | [email protected] (%) | [email protected]–0.95 (%) | Params | GFLOPs | FPS |
---|---|---|---|---|---|
YOLOX [30] | 87 | 50.7 | 8.05 | 21.8 | 227.69 |
Gold_YOLO [31] | 85.3 | 48.9 | 46 | 21.5 | 148.6 |
Cascade RCNN [32] | 79.6 | 42.0 | 69.152 | 209.92 | 50.8 |
RTDETR [33] | 84.6 | 47.8 | 32.80 | 108.0 | 116.32 |
DDQ [34] | 83.9 | 48.7 | 48.31 | 236.2 | 46.15 |
Light-YOLO | 86.7 | 51.8 | 4.4 | 6.1 | 121.95 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Chen, N.; Zhu, J.; Zheng, L. Light-YOLO: A Study of a Lightweight YOLOv8n-Based Method for Underwater Fishing Net Detection. Appl. Sci. 2024, 14, 6461. https://doi.org/10.3390/app14156461
Chen N, Zhu J, Zheng L. Light-YOLO: A Study of a Lightweight YOLOv8n-Based Method for Underwater Fishing Net Detection. Applied Sciences. 2024; 14(15):6461. https://doi.org/10.3390/app14156461
Chicago/Turabian StyleChen, Nuo, Jin Zhu, and Linhan Zheng. 2024. "Light-YOLO: A Study of a Lightweight YOLOv8n-Based Method for Underwater Fishing Net Detection" Applied Sciences 14, no. 15: 6461. https://doi.org/10.3390/app14156461