Urban PM2.5 Concentration Prediction via Attention-Based CNN–LSTM
<p>Framework of the proposed approach.</p> "> Figure 2
<p>Illustration of the multiscale predictors.</p> "> Figure 3
<p>(<b>a</b>) The location map of Taiyuan City; (<b>b</b>) distribution of air quality monitoring stations in Taiyuan City.</p> "> Figure 4
<p>Box charts of meteorological features and fine particulate matter (PM<sub>2.5</sub>) concentration: (<b>a</b>) air temperature; (<b>b</b>) humidity; (<b>c</b>) visibility.</p> "> Figure 4 Cont.
<p>Box charts of meteorological features and fine particulate matter (PM<sub>2.5</sub>) concentration: (<b>a</b>) air temperature; (<b>b</b>) humidity; (<b>c</b>) visibility.</p> "> Figure 5
<p>Relationship between wind direction, wind speed, and PM<sub>2.5</sub>.</p> "> Figure 6
<p>Comparison of the autocorrelation coefficients at different time lags for different stations.</p> "> Figure 7
<p>Architecture of long short-term memory (LSTM) memory cell.</p> "> Figure 8
<p>The loss convergence of deep learning methods in one-hour PM<sub>2.5</sub> prediction: (<b>a</b>) loss convergence of LSTM, convolution neural network (CNN)–LSTM, and AC-LSTM models; (<b>b</b>) loss convergence of simple recurrent neural network (RNN) model.</p> "> Figure A1
<p>The results of all models in the one-hour PM<sub>2.5</sub> prediction: (<b>a</b>) SVR, RFR, and MLP; (<b>b</b>) simple RNN, LSTM, CNN–LSTM, and AC-LSTM.</p> "> Figure A2
<p>The results of all models in the 5-h PM<sub>2.5</sub> prediction: (<b>a</b>) SVR, RFR, and MLP; (<b>b</b>) simple RNN, LSTM, CNN–LSTM, and AC-LSTM.</p> "> Figure A3
<p>The results of all models in the 10-h PM<sub>2.5</sub> prediction: (<b>a</b>) SVR, RFR, and MLP; (<b>b</b>) simple RNN, LSTM, CNN–LSTM, and AC-LSTM.</p> "> Figure A4
<p>The results of all models in the 19-h PM<sub>2.5</sub> prediction: (<b>a</b>) SVR, RFR, and MLP; (<b>b</b>) simple RNN, LSTM, CNN–LSTM, and AC-LSTM.</p> "> Figure A5
<p>The goodness-of-fit plots for all models in the one-hour PM<sub>2.5</sub> prediction: (<b>a</b>) SVR; (<b>b</b>) RFR; (<b>c</b>) MLP; (<b>d</b>) simple RNN; (<b>e</b>) LSTM; (<b>f</b>) CNN–LSTM; (<b>g</b>) AC-LSTM.</p> "> Figure A5 Cont.
<p>The goodness-of-fit plots for all models in the one-hour PM<sub>2.5</sub> prediction: (<b>a</b>) SVR; (<b>b</b>) RFR; (<b>c</b>) MLP; (<b>d</b>) simple RNN; (<b>e</b>) LSTM; (<b>f</b>) CNN–LSTM; (<b>g</b>) AC-LSTM.</p> ">
Abstract
:1. Introduction
2. Overview of the AC-LSTM Framework
3. Data and Method
3.1. Data Description
3.2. Data Preprocessing
3.3. Correlation between Meteorological Features and PM2.5
3.4. Spatiotemporal Correlation Analysis
3.5. Method
3.5.1. Convolutional Neural Network
3.5.2. LSTM Network
3.5.3. Attention Layer
4. Results and Discussion
4.1. Experimental Set-Up
4.2. Effects of Different Features
4.3. Model Convergence
4.4. Model Comparison
5. Conclusions and Future Work
- Through the analysis of air quality data, PM2.5 concentration has a strong spatiotemporal correlation. Due to the air flow, PM2.5 concentration in the predicted area can be easily affected by the PM2.5 concentrations of the adjacent monitoring stations. As PM2.5 stays in the air for a long time, the past feature states also affect future PM2.5 concentration. This motivated the design of a spatiotemporal model for effective prediction of PM2.5 concentrations;
- The experimental results indicate that, in addition to using only the pollutant concentrations of the air monitoring stations, adding the meteorological data and the PM2.5 concentrations of the adjacent monitoring stations can improve the prediction accuracy of the model, especially for prediction tasks on time scales over one hour;
- The proposed AC-LSTM model can be applied to multiscale predictors at different time gaps. When compared with the traditional machine learning methods, such as SVR, MLP, and RFR, its prediction accuracy was improved significantly, especially in predicting the PM2.5 concentrations over the gap of one hour. In comparison with deep learning methods, such as simple RNN, LSTM, and CNN–LSTM, AC-LSTM produced improved prediction with lower MAE and RMSE measures due to the introduced attention mechanism in the LSTM model.
Author Contributions
Funding
Conflicts of Interest
Appendix A
References
- Kurt, A.; Oktay, A.B. Forecasting air pollutant indicator levels with geographic models 3 days in advance using neural networks. Expert Syst. Appl. 2010, 37, 7986–7992. [Google Scholar]
- Du, S.; Li, T.; Yang, Y.; Horng, S.J. Deep Air Quality Forecasting Using Hybrid Deep Learning Framework. arXiv 2018, arXiv:1812.04783. [Google Scholar]
- Li, J.; Li, H.; Ma, Y.; Wang, Y.; Abokifa, A.; Lu, C.; Biswas, P. Spatiotemporal distribution of indoor particulate matter concentration with a low-cost sensor network. Build. Environ. 2018, 127, 138–147. [Google Scholar]
- Song, C.; Wu, L.; Xie, Y.; He, J.; Chen, X.; Wang, T.; Lin, Y.; Jin, T.; Wang, A.; Liu, Y.; et al. Air pollution in China: Status and spatiotemporal variations. Environ. Pollut. 2017, 227, 334–347. [Google Scholar]
- Zheng, Y.; Liu, F.; Hsieh, H.P. U-air: When urban air quality inference meets big data. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Chicago, IL, USA, 11–14 August 2013; pp. 1436–1444. [Google Scholar]
- He, H.; Li, M.; Wang, W.; Wang, Z.; Xue, Y. Prediction of PM2.5 concentration based on the similarity in air quality monitoring network. Build. Environ. 2018, 137, 11–17. [Google Scholar]
- Ma, X.; Tao, Z.; Wang, Y.; Yu, H.; Wang, Y. Long short-term memory neural network for traffic speed prediction using remote microwave sensor data. Transp. Res. Part. C Emerg. Technol. 2015, 54, 187–197. [Google Scholar]
- Ong, B.T.; Sugiura, K.; Zettsu, K. Dynamically pre-trained deep recurrent neural networks using environmental monitoring data for predicting PM2.5. Neural Comput. Appl. 2016, 27, 1553–1566. [Google Scholar] [PubMed] [Green Version]
- Feng, R.; Zheng, H.; Gao, H.; Zhang, A.R.; Huang, C.; Zhang, J.X.; Luo, K.; Fan, J.R. Recurrent Neural Network and random forest for analysis and accurate forecast of atmospheric pollutants: A case study in Hangzhou, China. J. Clean. Prod. 2019, 231, 1005–1015. [Google Scholar]
- Hochreiter, S.; Schmidhuber, J. Long short-term memory. Neural Comput. 1997, 9, 1735–1780. [Google Scholar] [PubMed]
- Recurrent models of visual attention. Available online: https://arxiv.org/abs/1406.6247 (accessed on 24 June 2014).
- Bahdanau, D.; Cho, K.; Bengio, Y. Neural machine translation by jointly learning to align and translate. arXiv 2014, arXiv:1409.0473. [Google Scholar]
- Rush, A.M.; Chopra, S.; Weston, J. A neural attention model for abstractive sentence summarization. arXiv 2015, arXiv:1509.00685. [Google Scholar]
- Chaloulakou, A.; Kassomenos, P.; Spyrellis, N.; Demokritou, P.; Koutrakis, P. Measurements of PM10 and PM2.5 particle concentrations in Athens, Greece. Atmos. Environ. 2003, 37, 649–660. [Google Scholar] [CrossRef]
- Hussein, T.; Karppinen, A.; Kukkonen, J.; Härkönen, J.; Aalto, P.P.; Hämeri, K.; Kerminen, V.M.; Kulmala, M. Meteorological dependence of size-fractionated number concentrations of urban aerosol particles. Atmos. Environ. 2006, 40, 1427–1440. [Google Scholar] [CrossRef]
- Chen, J.; Lu, J.; Avise, J.C.; DaMassa, J.A.; Kleeman, M.J.; Kaduwela, A.P. Seasonal modeling of PM2.5, in California’s San Joaquin Valley. Atmos. Environ. 2014, 92, 182–190. [Google Scholar] [CrossRef]
- Zhang, C.; Ni, Z.; Ni, L. Multifractal detrended cross-correlation analysis between PM2.5 and meteorological factors. Phys. A Stat. Mech. Appl. 2015, 438, 114–123. [Google Scholar] [CrossRef]
- Ma, J.; Cheng, J.C.P. Estimation of the building energy use intensity in the urban scale by integrating GIS and big data technology. Appl. Energy 2016, 183, 182–192. [Google Scholar] [CrossRef]
- Junninen, H.; Niska, H.; Tuppurainen, K.; Ruuskanen, J.; Kolehmainen, M. Methods for imputation of missing values in air quality data sets. Atmos. Environ. 2004, 38, 2895–2907. [Google Scholar] [CrossRef]
- Pearson, K. Notes on Regression and Inheritance in the Case of Two Parents. Proc. R. Soc. Lond. 1895, 58, 240–242. [Google Scholar]
- Box, G.E.P.; Jenkins, G.M.; Reinsel, G.C.; Ljung, G.M. Time Series Analysis: Forecasting and Control; John Wiley & Sons: Hoboken, NJ, USA, 2015. [Google Scholar]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 2012, 60, 1097–1105. [Google Scholar] [CrossRef]
- Ma, J.; Ding, Y.; Cheng, J.C.P.; Jiang, F.; Xu, Z. Soft detection of 5-day BOD with sparse matrix in city harbor water using deep learning techniques. Water Res. 2019, 170, 115350. [Google Scholar] [CrossRef]
- Peng, L.; Liu, S.; Liu, R.; Wang, L. Effective long short-term memory with differential evolution algorithm for electricity price prediction. Energy 2018, 162, 1301–1314. [Google Scholar] [CrossRef]
- Jusoh, N.; Ibrahim, W.J.W. Evaluating Fuzzy Time Series and Artificial Neural Network for Air Pollution Index Forecasting. In Proceedings of the Second International Conference on the Future of ASEAN (ICoFA) 2017—Volume 2; Saian, R., Abbas, M.A., Eds.; Springer: Singapore, 2018; pp. 113–121. [Google Scholar]
- Prakash, A.; Kumar, U.; Kumar, K.; Jain, V.K. A Wavelet-based Neural Network Model to Predict Ambient Air Pollutants’ Concentration. Environ. Model. Assess. 2011, 16, 503–517. [Google Scholar] [CrossRef]
- Li, X.; Peng, L.; Yao, X.; Cui, S.; Hu, Y.; You, C.; Chi, T. Long short-term memory neural network for air pollutant concentration predictions: Method development and evaluation. Environ. Pollut. 2017, 231, 997–1004. [Google Scholar] [CrossRef] [PubMed]
- Huang, C.J.; Kuo, P.H. A deep cnn-lstm model for particulate matter (PM2.5) forecasting in smart cities. Sensors 2018, 18, 2220. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Stations | Code | Monitoring Environment | Coordinates |
---|---|---|---|
JianCaoPing | S1 | Urban: residential area | N 37.887, E 112.522 |
JianHe | S2 | Urban: residential area | N 37.910, E 112.573 |
ShangLan | S3 | Rural area | N 38.010, E 112.434 |
JinYuan | S4 | Suburban: residential area | N 37.712, E 112.469 |
XiaoDian | S5 | Urban: residential area | N 37.739, E 112.558 |
TaoYuan | S6 | Urban: residential area | N 37.869, E 112.536 |
WuCheng | S7 | Urban: commercial area | N 37.819, E 112.570 |
NanZhai | S8 | Suburban: industrial park | N 37.985, E 112.549 |
JinSheng | S9 | Suburban: industrial area | N 37.780, E 112.488 |
Station | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 | S9 |
---|---|---|---|---|---|---|---|---|---|
S1 | 1 | 0.95 | 0.2 | 0.69 | 0.8 | 0.95 | 0.86 | 0.95 | 0.72 |
S2 | 0.95 | 1 | 0.24 | 0.76 | 0.86 | 0.82 | 0.82 | 0.95 | 0.77 |
S3 | 0.2 | 0.24 | 1 | 0.29 | 0.57 | 0.08 | 0.53 | 0.37 | 0.4 |
S4 | 0.69 | 0.76 | 0.29 | 1 | 0.87 | 0.51 | 0.76 | 0.8 | 0.96 |
S5 | 0.8 | 0.86 | 0.57 | 0.87 | 1 | 0.62 | 0.93 | 0.93 | 0.9 |
S6 | 0.95 | 0.82 | 0.08 | 0.51 | 0.62 | 1 | 0.78 | 0.84 | 0.57 |
S7 | 0.86 | 0.82 | 0.53 | 0.76 | 0.93 | 0.78 | 1 | 0.94 | 0.85 |
S8 | 0.95 | 0.95 | 0.37 | 0.8 | 0.93 | 0.84 | 0.94 | 1 | 0.83 |
S9 | 0.72 | 0.77 | 0.4 | 0.96 | 0.9 | 0.57 | 0.85 | 0.83 | 1 |
Time Lag | 2 | 4 | 6 | 8 | 10 | 12 | 14 | 16 |
---|---|---|---|---|---|---|---|---|
MAE | 8.21 | 7.89 | 7.9 | 7.82 | 7.75 | 7.68 | 7.61 | 8.04 |
RMSE | 13.81 | 13.2 | 13.22 | 13.08 | 13.01 | 13.06 | 13.09 | 13.42 |
Features | 1 h | 2 h | 3 h | 4 h–6 h | 7 h–12 h | 13 h–24 h |
---|---|---|---|---|---|---|
Fp | 7.61 | 8.12 | 8.21 | 8.53 | 8.57 | 9.04 |
Fp + Fm | 7.98 | 7.99 | 7.99 | 8.51 | 8.6 | 8.89 |
Fp + Fm + Fa | 7.68 | 7.97 | 7.98 | 8.38 | 8.51 | 8.98 |
Features | 1 h | 2 h | 3 h | 4 h–6 h | 7 h–12 h | 13 h–24 h |
---|---|---|---|---|---|---|
Fp | 13.11 | 13.74 | 13.89 | 14.58 | 14.59 | 15.12 |
Fp + Fm | 13.09 | 13.62 | 13.73 | 14.56 | 14.54 | 15.01 |
Fp + Fm + Fa | 13.06 | 13.23 | 13.64 | 14.41 | 14.45 | 14.83 |
Models | 1 h | 2 h | 3 h | 4 h–6 h | 7 h–12 h | 13 h–24 h |
---|---|---|---|---|---|---|
SVR | 7.72 | 12.37 | 15.6 | 22.4 | 26.79 | 30.2 |
RFR | 7.9 | 12.59 | 16.02 | 21.74 | 25.77 | 28.86 |
MLP | 7.82 | 12.27 | 15.71 | 23.02 | 27.2 | 30.43 |
Simple RNN | 8.91 | 8.9 | 8.88 | 9.39 | 9.75 | 9.94 |
LSTM | 8.37 | 8.38 | 8.7 | 8.49 | 8.98 | 9.03 |
CNN–LSTM | 7.79 | 7.97 | 8.05 | 8.38 | 8.79 | 8.92 |
AC-LSTM | 7.68 | 7.97 | 7.98 | 8.38 | 8.51 | 8.98 |
Models | 1 h | 2 h | 3 h | 4 h–6 h | 7 h–12 h | 13 h–24 h |
---|---|---|---|---|---|---|
SVR | 13.46 | 20.92 | 26.14 | 35.48 | 41.59 | 49.09 |
RFR | 13.57 | 20.99 | 26.25 | 33.06 | 38.47 | 43.46 |
MLP | 13.7 | 20.73 | 26.15 | 36.01 | 42.68 | 48.07 |
Simple RNN | 14.1 | 14.24 | 14.62 | 15.19 | 15.38 | 15.15 |
LSTM | 13.91 | 13.97 | 14.32 | 14.58 | 14.99 | 15.11 |
CNN–LSTM | 13.25 | 13.73 | 13.84 | 14.43 | 14.53 | 15.02 |
AC-LSTM | 13.06 | 13.23 | 13.64 | 14.41 | 14.45 | 14.83 |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Li, S.; Xie, G.; Ren, J.; Guo, L.; Yang, Y.; Xu, X. Urban PM2.5 Concentration Prediction via Attention-Based CNN–LSTM. Appl. Sci. 2020, 10, 1953. https://doi.org/10.3390/app10061953
Li S, Xie G, Ren J, Guo L, Yang Y, Xu X. Urban PM2.5 Concentration Prediction via Attention-Based CNN–LSTM. Applied Sciences. 2020; 10(6):1953. https://doi.org/10.3390/app10061953
Chicago/Turabian StyleLi, Songzhou, Gang Xie, Jinchang Ren, Lei Guo, Yunyun Yang, and Xinying Xu. 2020. "Urban PM2.5 Concentration Prediction via Attention-Based CNN–LSTM" Applied Sciences 10, no. 6: 1953. https://doi.org/10.3390/app10061953