[go: up one dir, main page]
Skip to main content
SpringerLink
Log in

Fs-yolo: fire-smoke detection based on improved YOLOv7

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Fire has emerged as a major danger to the Earth’s ecological equilibrium and human well-being. Fire detection and alert systems are essential. There is a scarcity of public fire datasets with examples of fire and smoke in real-world situations. Moreover, techniques for recognizing items in fire smoke are imprecise and unreliable when it comes to identifying small objects. We developed a dual dataset to evaluate the model’s ability to handle these difficulties. Introducing FS-YOLO, a new fire detection model with improved accuracy. Training YOLOv7 may lead to overfitting because of the large number of parameters and the limited fire detection object categories. YOLOv7 struggles to recognize small dense objects during feature extraction, resulting in missed detections. The Swin Transformer module has been enhanced to decrease local feature interdependence, obtain a wider range of parameters, and handle features at several levels. The improvements strengthen the model’s robustness and the network’s ability to recognize dense tiny objects. The efficient channel attention was incorporated to reduce the occurrence of false fire detections. Localizing the region of interest and extracting meaningful information aids the model in identifying pertinent areas and minimizing false detections. The proposal also considers using fire-smoke and real-fire-smoke datasets. The latter dataset simulates real-world conditions with occlusions, lens blur, and motion blur. This dataset tests the model’s robustness and adaptability in complex situations. On both datasets, the mAP of FS-YOLO is improved by 6.4\(\%\) and 5.4\(\%\) compared to YOLOv7. In the robustness check experiments, the mAP of FS-YOLO is 4.1\(\%\) and 3.1\(\%\) higher than that of today’s SOTA models YOLOv8s, DINO.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Algorithm 2
Algorithm 3
Algorithm 4
Algorithm 5
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

Our dataset and code will be made public upon acceptance of the paper.

References

  1. Kuang, Hu-Lin, Chan, Leanne Lai Hang, Yan, Hong: Multi-class fruit detection based on multiple color channels, pp 1–7 (2015)

  2. Lowe, David G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60, 91–110 (2004)

    Article  Google Scholar 

  3. Lienhart, Rainer, Maydt, Jochen: An extended set of haar-like features for rapid object detection, 1: I–I (2002)

  4. Lei, Chen, Ji-Feng, Huang: Flame detection method based on video. Comput Eng Des, (2014)

  5. Xuehui, Wu., Xiaobo, Lu., Leung, Henry: A video based fire smoke detection using robust adaboost. Sensors 18(11), 3780 (2018)

    Article  Google Scholar 

  6. Krizhevsky, Alex, Sutskever, Ilya, Hinton, Geoffrey E: Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst, 25 (2012)

  7. Szegedy, Christian, Liu, Wei, Jia, Yangqing, Sermanet, Pierre, Reed, Scott, Anguelov, Dragomir, Erhan, Dumitru, Vanhoucke, Vincent, Rabinovich, Andrew: Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9 (2015)

  8. Simonyan, Karen, Zisserman, Andrew: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. https://doi.org/10.48550/arXiv.1409.1556. (2014). Accessed 1 Aug 2023

  9. He, Kaiming, Zhang, Xiangyu, Ren, Shaoqing, Sun, Jian: Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778 (2016)

  10. Hu, Jie, Shen, Li, Sun, Gang: Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141 (2018)

  11. Ren, Shaoqing, He, Kaiming, Girshick, Ross, Sun, Jian: Faster r-cnn: Towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst, 28 (2015)

  12. Liu, Wei , Anguelov, Dragomir, Erhan, Dumitru, Szegedy, Christian, Reed, Scott, Fu, Cheng-Yang, Berg, Alexander C: Ssd: Single shot multibox detector. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14, pp 21–37 Springer, (2016)

  13. Zhang, Shifeng, Wen, Longyin, Bian, Xiao, Lei, Zhen, Li, Stan Z: Single-shot refinement neural network for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4203–4212 (2018)

  14. Redmon, Joseph, Farhadi, Ali: Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767. https://doi.org/10.48550/arXiv.1804.02767, (2018). Accessed 25 Feb 2024

  15. Long, Jonathan, Shelhamer, Evan, Darrell, Trevor: Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440 (2015)

  16. Ronneberger, Olaf, Fischer, Philipp, Brox, Thomas: U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp 234–241. Springer (2015)

  17. Chen, Liang-Chieh., Papandreou, George, Kokkinos, Iasonas, Murphy, Kevin, Yuille, Alan L.: Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2017)

    Article  Google Scholar 

  18. Girshick, Ross, Donahue, Jeff, Darrell, Trevor, Malik, Jitendra: Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587 (2014)

  19. Girshick, Ross: Fast r-cnn. In Proceedings of the IEEE international conference on computer vision, pp 1440–1448 (2015)

  20. He, Kaiming, Gkioxari, Georgia, Dollár, Piotr, Girshick, Ross: Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pp 2961–2969 (2017)

  21. Carion, Nicolas, Massa, Francisco, Synnaeve, Gabriel, Usunier, Nicolas, Kirillov, Alexander, Zagoruyko, Sergey: End-to-end object detection with transformers. In European conference on computer vision. Springer, New York. pp 213–229 (2020)

  22. Redmon, Joseph, Farhadi, Ali: Yolo9000: better, faster, stronger. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7263–7271 (2017)

  23. Bochkovskiy, Alexey, Wang, Chien-Yao, Mark Liao, Hong-Yuan: Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934, (2020)

  24. Zheng Ge, Songtao Liu, Feng Wang, Zeming Li, and Jian Sun. Yolox: Exceeding yolo series in 2021. arXiv preprint arXiv:2107.08430 2021

  25. Wang, Chien-Yao, Bochkovskiy, Alexey, Mark Liao, Hong-Yuan: Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 7464–7475 (2023)

  26. Sun Li, Y.S., Shi, B Wang, Zhou, Z.Q., Wang, H.L.: Video smoke detection based on color transformation and mser. Trans Beijing Inst Technol 36(10), 1072–1078 (2016)

    Google Scholar 

  27. Wang, Teng, Shi, Lei, Yuan, Peng, Bu, Leping, Hou, Xinguo: A new fire detection method based on flame color dispersion and similarity in consecutive frames. In 2017 Chinese Automation Congress (CAC), IEEE , pp 151–156 (2017)

  28. Emmy Prema, C., Vinsley, S.S., Suresh, S.: Efficient flame detection based on static and dynamic texture analysis in forest fire detection. Fire Technol. 54, 255–288 (2018)

    Article  Google Scholar 

  29. Huizhen, Z.H.A.N.G., Yunyang, Y.A.N., Yian, L.I.U., et al.: Video flame detection based on super pixel segmentation and flash frequency feature discrimination. J Data Acquis Process 33(3), 512 (2018)

    Google Scholar 

  30. Zhong, Zhen, Wang, Minjuan, Shi, Yukun, Gao, Wanlin: A convolutional neural network-based flame detection method in video sequence. SIViP 12, 1619–1627 (2018)

    Article  Google Scholar 

  31. Hui, T., Halidan, A., Du, H., et al.: Multi-type flame detection combined with faster r-cnn. J Image Graphics 24(1), 73–83 (2019)

    Google Scholar 

  32. Zhang, Qi.-xing, Lin, Gao-hua, Zhang, Yong-ming, Gao, Xu., Wang, Jin-jun: Wildland forest fire smoke detection based on faster r-cnn using synthetic smoke images. Proc Eng 211, 441–446 (2018)

    Article  Google Scholar 

  33. Lin, Gaohua, Zhang, Yongming, Gao, Xu., Zhang, Qixing: Smoke detection on video sequences using 3d convolutional neural networks. Fire Technol. 55, 1827–1847 (2019)

    Article  Google Scholar 

  34. Qin, Yue-Yan., Cao, Jiang-Tao., Ji, Xiao-Fei.: Fire detection method based on depthwise separable convolution and yolov3. Int. J. Autom. Comput. 18, 300–310 (2021)

    Article  Google Scholar 

  35. Qian, Huimin, Shi, Fei, Chen, Wei, Ma, Yilong, Huang, Min: A fire monitoring and alarm system based on channel-wise pruned yolov3. Multimed Tools Appl, pp 1–19 (2022)

  36. Renjie, Xu., Lin, Haifeng, Kangjie, Lu., Cao, Lin, Liu, Yunfei: A forest fire detection system based on ensemble learning. Forests 12(2), 217 (2021)

    Article  Google Scholar 

  37. Hongwen, Du., Zhu, Wenzhong, Peng, Ke., Li, Weifu: Improved high speed flame detection method based on yolov7. Open J Appl Sci 12(12), 2004–2018 (2022)

    Google Scholar 

  38. Muksimova, Shakhnoza, Mardieva, Sevara, Cho, Young-Im.: Deep encoder-decoder network-based wildfire segmentation using drone images in real-time. Remote Sens 14(24), 6302 (2022)

    Article  Google Scholar 

  39. Shakhnoza, Muksimova, Sabina, Umirzakova, Sevara, Mardieva, Cho, Young-Im: Novel video surveillance-based fire and smoke classification using attentional feature map in capsule networks. Sensors, 22(1)

  40. Yao JIN and Wei ZHANG: Real-time fire detection algorithm with anchor-free network architecture. J Zhejiang Univ (Eng Sci) 54(12), 2430–2436 (2020)

    Google Scholar 

  41. Niu, Zhaoyang, Zhong, Guoqiang, Hui, Yu.: A review on the attention mechanism of deep learning. Neurocomputing 452, 48–62 (2021)

    Article  Google Scholar 

Download references

Acknowledgements

I would like to acknowledge Professor Wang, who set an example of excellence as a researcher, for her kind support and wise advice.

Funding

The National Natural Science Foundation of China(6210 3096),The Natural Science Foundation of Hainan Province (623MS071),The "Chunhui Plan" cooperative scientific research project of the Ministry of Education(HZKY202 20314),Hainan Province Science and Technology Special Fund (ZDYF2022SHFZ105), The Natural Science Foundation of Heilongjiang Province (LH2022F009), The Hainan Provincial Joint Project of Sanya Yazhou Bay Science and Technology City (2021JJLH0025),Guiding Innovation Foundation of Northeast Petroleum University (15071202 203).

Author information

Authors and Affiliations

  1. College of Electrical and Information Engineering, Northeast Petroleum University, Daging, 163318, China

    Dongmei Wang, Ying Qian, Jingyi Lu, Peng Wang, Zhongrui Hu & Yongkang Chai

  2. SANYA Offshore 0il & Gas Research Institute, Northeast Petroleum University, Sanya, 572023, China

    Dongmei Wang, Ying Qian, Jingyi Lu & Peng Wang

  3. Heilongjiang Provincial Key Laboratory of Networking and Intelligent Control, Daqing, 163318, China

    Jingyi Lu & Peng Wang

  4. Artificial Intelligence Energy Research lnstitute, Northeast Petroleum University, Daqing, 163318, China

    Jingyi Lu & Peng Wang

Authors
  1. Dongmei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ying Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jingyi Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhongrui Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yongkang Chai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jingyi Lu.

Additional information

Communicated by Q. Xu.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, D., Qian, Y., Lu, J. et al. Fs-yolo: fire-smoke detection based on improved YOLOv7. Multimedia Systems 30, 215 (2024). https://doi.org/10.1007/s00530-024-01359-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00530-024-01359-z

Keywords

Search

Navigation