Predict Future Transient Fire Heat Release Rates Based on Fire Imagery and Deep Learning
<p>Image of indoor fire scene.</p> "> Figure 2
<p>Combustion tests of different ignition sources. (<b>a</b>) Cardboard boxes; (<b>b</b>) rubber trash bins; (<b>c</b>) plastic chairs.</p> "> Figure 3
<p>Preprocessing of fire scene video images.</p> "> Figure 4
<p>LSTM and Bi-LSTM. (<b>a</b>) Internal structure of an LSTM cell; (<b>b</b>) Bi-LSTM architecture diagram.</p> "> Figure 5
<p>The attention mechanism. (<b>a</b>) The concept of the attention mechanism; (<b>b</b>) the structure of the attention model.</p> "> Figure 6
<p>Comparative bar chart of some predictive performance indicators of the model.</p> "> Figure 7
<p>The Att-BiLSTM network architecture for predicting future transient HRR of a fire scene.</p> "> Figure 8
<p>Loss during training and validation of the model.</p> "> Figure 9
<p>Test results of the validation set. (<b>a</b>) Residual plot of validation set results; (<b>b</b>) control chart of validation set results.</p> "> Figure 10
<p>Fire scenes under different luminance conditions. (<b>a</b>) Higher luminance; (<b>b</b>) lower luminance.</p> "> Figure 11
<p>Demonstration result images for high-luminance fire scene environments. (<b>a</b>) Cardboard boxes; (<b>b</b>) rubber trash bins; (<b>c</b>) plastic chairs.</p> "> Figure 12
<p>Demonstration result images for other complex combustibles. (<b>a</b>) Box-type gas burner; (<b>b</b>) utility cart with a laptop and printer; (<b>c</b>) propanol liquid.</p> "> Figure 12 Cont.
<p>Demonstration result images for other complex combustibles. (<b>a</b>) Box-type gas burner; (<b>b</b>) utility cart with a laptop and printer; (<b>c</b>) propanol liquid.</p> "> Figure 13
<p>The application of future fire HRR prediction based on fire scene images and deep learning in intelligent firefighting.</p> ">
Abstract
:1. Introduction
2. Materials
2.1. Fire Scene Image Database
2.2. Dataset Preprocessing
- Image Cropping: Given the high quality of the fire video images in the NIST database (1920 × 1080), using the original images directly for training would increase the computational complexity and cost. Therefore, all fire scene images were first resized to 50 × 108. A resolution of 50 pixels in width was selected to accommodate the spatial characteristics of all the fire images in this dataset. This resolution was chosen to maximize the retention of flame features in the field of view and minimize the interference of background noise. Additionally, a height of 108 pixels was selected to ensure that the flame height and upper smoke features were adequately captured, taking into account the height of the original image, which was 1080 pixels. The height of 108 pixels was selected to account for the original image’s height of 1080 pixels, ensuring that the flame height and upper smoke features were adequately captured.
- Random Horizontal Image Flipping: To increase data diversity, reduce redundancy, and improve the model’s generalization ability and robustness, this paper employed a data augmentation technique of random horizontal image flipping. This technique flips the images horizontally with a probability of 0.5, altering the content of the images without changing the pixel values. This renders the model indifferent to the orientation of fire scene images, enabling more accurate recognition of images from different angles.
- Image Pixel Value Normalization: To ensure that the input parameters (pixels) of fire scene images exhibit a similar data distribution, reduce data bias, and accelerate model convergence, the pixel values of fire scene images were normalized. This process converts the pixel values from the original range of [0, 255] to [0, 1]. This precludes the occurrence of excessive discrepancies in pixel values, which could potentially result in model instability or overfitting.
3. Methods
3.1. Bi-LSTM Layer
3.2. Attention Layer
3.3. Model Input
3.4. Prediction Process
4. Results
4.1. Model Training
4.2. Validation Set
5. Discussion
5.1. High-Brightness Fire Scenes
5.2. Complex Combustibles
5.3. Comparative Analysis with Similar Studies
5.4. Applications in Intelligent Firefighting
6. Conclusions
- A new end-to-end method for predicting future fire HRR is proposed. By inputting fire scene images and corresponding HRR label data into the Att-BiLSTM model and employing a sliding window mechanism, it is possible to achieve continuous output of future transient fire HRR predictions.
- In the preprocessing of fire scene images, the quality of images is enhanced while reasonably preserving the information of flames and smoke. This is achieved by fully considering their coexistence characteristics and their impact on fire HRR.
- The model’s generalization ability and reliability were tested in high-brightness environments and fire scenes with complex combustibles. The experimental results demonstrate that the model can accurately predict future transient HRR of fire scenes and can also simulate and predict the development trend of fire situations to a certain extent.
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Johansson, N.; Svensson, S. Review of the Use of Fire Dynamics Theory in Fire Service Activities. Fire Technol. 2018, 55, 81–103. [Google Scholar] [CrossRef]
- Tewarson, A. Heat release rate in fires. Fire Mater. 2004, 4, 185–191. [Google Scholar] [CrossRef]
- Thornton, W.M. XV. The relation of oxygen to the heat of combustion of organic compounds. Lond. Edinb. Dublin Philos. Mag. J. Sci. 2009, 33, 196–203. [Google Scholar] [CrossRef]
- Sun, P.; Wu, C.; Zhu, F.; Wang, S.; Huang, X. Microgravity combustion of polyethylene droplet in drop tower. Combust. Flame 2020, 222, 18–26. [Google Scholar] [CrossRef]
- Xiong, C.; Fan, H.; Huang, X.; Fernandez-Pello, C. Evaluation of burning rate in microgravity based on the fuel regression, flame area, and spread rate. Combust. Flame 2022, 237, 111846. [Google Scholar] [CrossRef]
- Sun, X.; Hu, L.; Zhang, X.; Ren, F.; Yang, Y.; Fang, X. Experimental study on flame pulsation behavior of external venting facade fire ejected from opening of a compartment. Proc. Combust. Inst. 2021, 38, 4485–4493. [Google Scholar] [CrossRef]
- Li, C.; Li, X.; Chen, M.; Sun, X. Deep Learning and Image Recognition. In Proceedings of the 2023 IEEE 6th International Conference on Electronic Information and Communication Technology (ICEICT), Qingdao, China, 21–24 July 2023; pp. 557–562. [Google Scholar]
- Taşyürek, M. ODRP: A new approach for spatial street sign detection from EXIF using deep learning-based object detection, distance estimation, rotation and projection system. Vis. Comput. 2024, 40, 983–1003. [Google Scholar] [CrossRef]
- Hodges, J.L.; Lattimer, B.Y.; Luxbacher, K.D. Compartment fire predictions using transpose convolutional neural networks. Fire Saf. J. 2019, 108, 102854. [Google Scholar] [CrossRef]
- Wu, X.; Park, Y.; Li, A.; Huang, X.; Xiao, F.; Usmani, A. Smart Detection of Fire Source in Tunnel Based on the Numerical Database and Artificial Intelligence. Fire Technol. 2021, 57, 657–682. [Google Scholar] [CrossRef]
- Wu, X.; Zhang, X.; Huang, X.; Xiao, F.; Usmani, A. A real-time forecast of tunnel fire based on numerical database and artificial intelligence. Build. Simul. 2022, 15, 511–524. [Google Scholar] [CrossRef]
- Xiqiang, W.; Zhang, X.; Jiang, Y.; Huang, X.; Huang, G.Q.; Usmani, A. An intelligent tunnel firefighting system and small-scale demonstration. Tunn. Undergr. Space Technol. 2022, 120, 104301. [Google Scholar] [CrossRef]
- Su, L.-c.; Wu, X.; Zhang, X.; Huang, X. Smart performance-based design for building fire safety: Prediction of smoke motion via AI. J. Build. Eng. 2021, 43, 102529. [Google Scholar] [CrossRef]
- Ghosh, R.; Kumar, A. A hybrid deep learning model by combining convolutional neural network and recurrent neural network to detect forest fire. Multimed. Tools Appl. 2022, 81, 38643–38660. [Google Scholar] [CrossRef]
- Choi, H.-S.; Jeon, M.; Song, K.; Kang, M. Semantic Fire Segmentation Model Based on Convolutional Neural Network for Outdoor Image. Fire Technol. 2021, 57, 3005–3019. [Google Scholar] [CrossRef]
- Ban, Y.; Zhang, P.; Nascetti, A.; Bevington, A.R.; Wulder, M.A. Near Real-Time Wildfire Progression Monitoring with Sentinel-1 SAR Time Series and Deep Learning. Sci. Rep. 2020, 10, 1322. [Google Scholar] [CrossRef] [PubMed]
- Wang, Z.; Zhang, T.; Wu, X.; Huang, X. Predicting transient building fire based on external smoke images and deep learning. J. Build. Eng. 2022, 47, 103823. [Google Scholar] [CrossRef]
- Wang, Z.; Zhang, T.; Huang, X. Predicting real-time fire heat release rate by flame images and deep learning. Proc. Combust. Inst. 2023, 39, 4115–4123. [Google Scholar] [CrossRef]
- Fire Calorimetry Database (FCD). Available online: https://www.nist.gov/el/fcd (accessed on 30 April 2024).
- The NIST 20 MW Calorimetry Measurement System for Large-Fire Research. Available online: https://www.nist.gov/publications/nist-20-mw-calorimetry-measurement-system-large-fire-research (accessed on 30 April 2024).
- Jin, C.; Wang, T.; Alhusaini, N.; Zhao, S.; Liu, H.; Xu, K.; Zhang, J. Video Fire Detection Methods Based on Deep Learning: Datasets, Methods, and Future Directions. Fire 2023, 6, 315. [Google Scholar] [CrossRef]
- Wang, Z.; Yang, B. Attention-based Bidirectional Long Short-Term Memory Networks for Relation Classification Using Knowledge Distillation from BERT. In Proceedings of the 2020 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech), Calgary, AB, Canada, 17–22 August 2020; pp. 562–568. [Google Scholar]
- Zhang, Q.; Wang, R.; Qi, Y.; Wen, F. A watershed water quality prediction model based on attention mechanism and Bi-LSTM. Environ. Sci. Pollut. Res. 2022, 29, 75664–75680. [Google Scholar] [CrossRef]
- Luo, J.; Zhang, X. Convolutional neural network based on attention mechanism and Bi-LSTM for bearing remaining life prediction. Appl. Intell. 2022, 52, 1076–1091. [Google Scholar] [CrossRef]
- User’s Guide for Fire Calorimetry Database (FCD). Available online: https://www.nist.gov/system/files/documents/2020/11/19/FCD_User_Guide_v4a.pdf (accessed on 30 April 2024).
- Heat Release Rates of Multiple Transient Combustibles. Available online: https://www.nist.gov/publications/heat-release-rates-multiple-transient-combustibles (accessed on 30 April 2024).
- Heat Release Rate and Fire Characteristics of Fuels Representative of Typical Transient Fire Events in Nuclear Power Plants. Available online: https://www.nrc.gov/docs/ML2009/ML20091L481.pdf (accessed on 30 April 2024).
- NIST Technical Note 2102 Heat Release Rates of Multiple Transient Combustibles. Available online: https://nvlpubs.nist.gov/nistpubs/TechnicalNotes/NIST.TN.2102.pdf (accessed on 30 April 2024).
- Wang, L.; Xiong, Y.; Wang, Z.; Qiao, Y.; Lin, D.; Tang, X.; Van Gool, L. Temporal Segment Networks: Towards Good Practices for Deep Action Recognition. In Proceedings of the Computer Vision—ECCV 2016, Amsterdam, The Netherlands, 11–14 October 2016; Springer: Cham, Switzerland, 2016; pp. 20–36. [Google Scholar]
- Gu, S.; Ding, L. IEEE A Complex-Valued VGG Network Based Deep Learing Algorithm for Image Recognition. In Proceedings of the 9th International Conference on Intelligent Control and Information Processing (ICICIP), Wanzhou, China, 9–11 November 2018; pp. 340–343. [Google Scholar]
- Ho, W.-H.; Huang, T.-H.; Yang, P.-Y.; Chou, J.-H.; Huang, H.-S.; Chi, L.-C.; Chou, F.-I.; Tsai, J.-T. Artificial intelligence classification model for macular degeneration images: A robust optimization framework for residual neural networks. BMC Bioinform. 2021, 22, 148. [Google Scholar] [CrossRef] [PubMed]
- Fu, G.; Wei, Q.; Yang, Y.; Li, C. Bearing fault diagnosis based on CNN-BiLSTM and residual module. Meas. Sci. Technol. 2023, 34, 12. [Google Scholar] [CrossRef]
- Telili, A.; Fezza, S.A.; Hamidouche, W.; Brachemi Meftah, H.F.Z. 2BiVQA: Double Bi-LSTM-based Video Quality Assessment of UGC Videos. ACM Trans. Multimed. Comput. Commun. Appl. 2023, 20, 1–22. [Google Scholar] [CrossRef]
- Soydaner, D. Attention mechanism in neural networks: Where it comes and where it goes. Neural Comput. Appl. 2022, 34, 13371–13385. [Google Scholar] [CrossRef]
- Hochreiter, S.; Schmidhuber, J. Long short-term memory. Neural Comput. 1997, 9, 1735–1780. [Google Scholar] [CrossRef]
- Siami-Namini, S.; Tavakoli, N.; Namin, A.S. The Performance of LSTM and BiLSTM in Forecasting Time Series. In Proceedings of the 2019 IEEE International Conference on Big Data (Big Data), Los Angeles, CA, USA, 9–12 December 2019; pp. 3285–3292. [Google Scholar]
- Graves, A.; Schmidhuber, J. Framewise phoneme classification with bidirectional LSTM networks. In Proceedings of the 2005 IEEE International Joint Conference on Neural Networks, Montreal, QC, Canada, 31 July–4 August 2005; pp. 2047–2052. [Google Scholar]
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, L.; Polosukhin, I. Attention Is All You Need. arXiv 2017, 30. [Google Scholar] [CrossRef]
- Sutskever, I.; Vinyals, O.; Le, Q.V. Sequence to Sequence Learning with Neural Networks. arXiv 2014, 2, 3104–3112. [Google Scholar] [CrossRef]
- Pincott, J.; Tien, P.W.; Wei, S.; Calautit, J.K. Indoor fire detection utilizing computer vision-based strategies. J. Build. Eng. 2022, 61, 105154. [Google Scholar] [CrossRef]
- Zilong, W.; Ding, Y.; Zhang, T.; Huang, X. Automatic real-time fire distance, size and power measurement driven by stereo camera and deep learning. Fire Saf. J. 2023, 140, 103891. [Google Scholar] [CrossRef]
t | R2 | MSE | RMSE | MAE | MAPE |
---|---|---|---|---|---|
9 | 0.96960 | 0.062650 | 0.006998 | 0.006693 | 0.005965 |
10 | 0.99700 | 0.061348 | 0.007832 | 0.005588 | 0.005463 |
11 | 0.97036 | 0.071012 | 0.006698 | 0.004325 | 0.063250 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Xu, L.; Dong, J.; Zou, D. Predict Future Transient Fire Heat Release Rates Based on Fire Imagery and Deep Learning. Fire 2024, 7, 200. https://doi.org/10.3390/fire7060200
Xu L, Dong J, Zou D. Predict Future Transient Fire Heat Release Rates Based on Fire Imagery and Deep Learning. Fire. 2024; 7(6):200. https://doi.org/10.3390/fire7060200
Chicago/Turabian StyleXu, Lei, Jinyuan Dong, and Delei Zou. 2024. "Predict Future Transient Fire Heat Release Rates Based on Fire Imagery and Deep Learning" Fire 7, no. 6: 200. https://doi.org/10.3390/fire7060200