Reconstructing Geostationary Satellite Land Surface Temperature Imagery Based on a Multiscale Feature Connected Convolutional Neural Network
<p>Structure of the proposed multiscale feature connection temperature reconstruction-convolutional neural network (MFCTR-CNN) framework.</p> "> Figure 2
<p>The structure of the spatial attention features connection unit (SAU).</p> "> Figure 3
<p>Study areas of (<b>a</b>) FengYun-2G (FY-2G) and (<b>b</b>) the Meteosat Second Generation-Spinning Enhanced Visible and Infrared Imager (MSG-SEVIRI). Different colors represent different land-cover types under the International Geosphere–Biosphere Programme [<a href="#B38-remotesensing-11-00300" class="html-bibr">38</a>]. For the details of the land-cover types, the reader is referred to the web version of this article.</p> "> Figure 4
<p>Masks (size: 40 × 40) with different missing data rates (MDRs) and different distribution characteristics (DCs) of the missing observations (i.e., concentrated or scattered). White pixels represent valid values and black pixels denote invalid values. (<b>a</b>) 53% MDR with concentrated pixels. (<b>b</b>) 53% MDR with scattered pixels. (<b>c</b>) 65% MDR with scattered pixels. (<b>d</b>) 70% MDR with scattered pixels.</p> "> Figure 5
<p>Examples of reconstruction results using MFCTR-CNN and the spline spatial interpolation method for FY-2G LST images during different seasons. (<b>a</b>) August 2015 (summer). (<b>b</b>) November 2015 (fall). (<b>c</b>) February 2016 (winter). (<b>d</b>) May 2016 (spring). S1, S2 and S3 represent periods of 06:00 to 12:00, 12:00 to 18:00, and 18:00 to 06:00 (local time), respectively.</p> "> Figure 5 Cont.
<p>Examples of reconstruction results using MFCTR-CNN and the spline spatial interpolation method for FY-2G LST images during different seasons. (<b>a</b>) August 2015 (summer). (<b>b</b>) November 2015 (fall). (<b>c</b>) February 2016 (winter). (<b>d</b>) May 2016 (spring). S1, S2 and S3 represent periods of 06:00 to 12:00, 12:00 to 18:00, and 18:00 to 06:00 (local time), respectively.</p> "> Figure 5 Cont.
<p>Examples of reconstruction results using MFCTR-CNN and the spline spatial interpolation method for FY-2G LST images during different seasons. (<b>a</b>) August 2015 (summer). (<b>b</b>) November 2015 (fall). (<b>c</b>) February 2016 (winter). (<b>d</b>) May 2016 (spring). S1, S2 and S3 represent periods of 06:00 to 12:00, 12:00 to 18:00, and 18:00 to 06:00 (local time), respectively.</p> "> Figure 6
<p>Examples of reconstruction absolute error maps using MFCTR-CNN for FY-2G LST images in May 2016 (spring). S1, S2 and S3 represent periods of 06:00 to 12:00, 12:00 to 18:00, and 18:00 to 06:00 (local time), respectively.</p> "> Figure 7
<p>Distribution histogram of errors. (<b>a</b>) FY-2G (August 2015, November 2015, February 2016, and May 2016). (<b>b</b>) MSG (August 2010, November 2010, February 2011, and May 2011).</p> "> Figure 7 Cont.
<p>Distribution histogram of errors. (<b>a</b>) FY-2G (August 2015, November 2015, February 2016, and May 2016). (<b>b</b>) MSG (August 2010, November 2010, February 2011, and May 2011).</p> "> Figure 7 Cont.
<p>Distribution histogram of errors. (<b>a</b>) FY-2G (August 2015, November 2015, February 2016, and May 2016). (<b>b</b>) MSG (August 2010, November 2010, February 2011, and May 2011).</p> "> Figure 8
<p>Original images, auxiliary image, and the reconstructed results for FY-2G (top, 5 km spatial resolution) and MSG-SEVIRI (bottom, 5 km spatial resolution) land surface temperatures (LSTs).</p> ">
Abstract
:1. Introduction
2. Reconstruction Architecture
2.1. Temporal Data Combination
2.2. Features of GLST Extraction by the Down-Sampling Procedure
2.3. GLST Image Recovery Using Up-Sampling and a Spatial Attention Unit
3. Experiments
3.1. Datasets
3.2. Network Training Details
3.3. Experiment Results
3.3.1. Visual Performance of MFCTR-CNN in the Simulation Experiment
3.3.2. Quantitative Evaluation under Different MDRs and DCs
3.3.3. Application to Actual LST Data
4. Discussion
5. Conclusions
Author Contributions
Funding
Acknowledgements
Conflicts of Interest
References
- Wan, Z.; Li, Z. A physics-based algorithm for retrieving land-surface emissivity and temperature from EOD/MODIS data. IEEE Trans. Geosci. Remote Sens. 1997, 35, 980–996. [Google Scholar]
- Weng, Q. Thermal infrared remote sensing for urban climate and environmental studies: Methods, applications, and trends. ISPRS J. Photogramm. Remote Sens. 2009, 64, 335–344. [Google Scholar] [CrossRef]
- Zhan, W.; Chen, Y.; Zhou, J.; Wang, J.; Liu, W.; Voogt, J.; Zhu, X.; Quan, J.; Li, J. Disaggregation of remotely sensed land surface temperature: Literature survey, taxonomy, issues, and caveats. Remote Sens. Environ. 2013, 131, 119–139. [Google Scholar] [CrossRef]
- Li, Z.; Tang, B.; Wu, H.; Ren, H.; Yan, G.; Wan, Z.; Trigo, I.; Sobringo, J. Satellite-derived land surface temperature: Current status and perspectives. Remote Sens. Environ. 2013, 131, 14–37. [Google Scholar] [CrossRef] [Green Version]
- Fensholt, R.; Anyamba, A.; Huber, S.; Proud, S.R.; Tucker, C.J.; Small, J.; Pak, E.; Rasmussen, M.O.; Sandholt, I.; Shisanya, C. Analysing the advantages of high temporal resolution geostationary MSG SEVIRI data compared to polar operational environmental satellite data for land surface monitoring in Africa. Int. J. Appl. Earth Observation Geoinf. 2011, 13, 721–729. [Google Scholar] [CrossRef]
- Pergola, N.; Marchese, F.; Tramutoli, V.; Filizzola, C.; Ciampa, M. Advanced satellite technique for volcanic ctivity monitoring and early warning. Ann. Geophys. 2008, 51, 287–300. [Google Scholar]
- Lu, L.; Venus, V.; Skidmore, A.; Wang, T.; Luo, G. Estimating land surface temperature under clouds using MSG/SEVIRI observations. Int. J. Appl. Earth Observ. Geoinf. 2011, 13, 265–276. [Google Scholar] [CrossRef]
- Li, X.; Zhou, Y.; Asrar, G.; Zhu, Z. Creating a seamless 1 km resolution daily land surface temperature dataset for urban and surrounding areas in the conterminous United States. Remote Sens. Environ. 2018, 206, 84–97. [Google Scholar] [CrossRef]
- Zeng, C.; Li, D.; Shen, H.; Wu, P.; Cui, Y.; Hong, Y. A two-step framework for reconstructing remotely sensed land surface temperatures contaminated by cloud. ISPRS J. Photogramm. Remote Sens. 2018, 141, 30–45. [Google Scholar] [CrossRef]
- Zeng, C.; Shen, H.; Zhong, M.; Zhang, L.; Wu, P. Reconstructing MODIS LST Based on Multitemporal Classification and Robust Regression. IEEE Geosci. Remote Sense. Lett. 2015, 12, 512–516. [Google Scholar] [CrossRef]
- Li, X.; Shen, H.; Zhang, L.; Zhang, H.; Yuan, Q. Recovering quantitative remote sensing products contaminated by thick clouds and shadows using multitemporal dictionary learning. IEEE Trans. Geosci. Remote Sens. 2014, 52, 7086–7098. [Google Scholar]
- Jin, M. Interpolation of surface radiative temperature measured from polar orbiting satellites to a diurnal cycle: 2. Cloudy-pixel treatment. J. Geophys. Res. Atoms. 2000, 105, 4061–4076. [Google Scholar] [CrossRef] [Green Version]
- Neteler, M. Estimating daily land surface temperatures in mountainous environments by reconstructed MODIS LST data. Remote Sens. 2010, 2, 333–351. [Google Scholar] [CrossRef]
- Hong, F.; Zhan, W.; Göttsche, F.; Liu, Z.; Zhou, J.; Huang, F.; Lai, J.; Li, M. Comprehensive assessment of four-parameter diurnal land surface temperature cycle models under clear-sky. ISPRS J. Photogramm. Remote Sens. 2018, 142, 190–204. [Google Scholar] [CrossRef]
- Jonsson, P.; Eklundh, L. Seasonality extraction by function fitting to time-series of satellite sensor data. IEEE Trans. Geosci. Remote Sens. 2002, 40, 1824–1832. [Google Scholar] [CrossRef]
- Zhang, G.; Xiao, X.; Dong, J.; Kou, W.; Jin, C.; Qin, Y.; Biradar, C. Mapping paddy rice planting areas through time series analysis of MODIS land surface temperature and vegetation index data. ISPRS J. Photogramm. Remote Sens. 2015, 106, 157–171. [Google Scholar] [CrossRef] [Green Version]
- Yu, W.; Ma, M.; Wang, X.; Tan, J. Estimating the land-surface temperature of pixels covered by clouds in MODIS products. J. Appl. Remote Sens. 2014, 8, 083525. [Google Scholar] [CrossRef]
- Liu, Z.; Wu, P.; Duan, S.; Zhan, W.; Ma, X.; Wu, Y. Spatiotemporal reconstruction of land surface temperature derived from Feng Yun geostationary satellite data. IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens. 2017, 10, 4531–4543. [Google Scholar] [CrossRef]
- Weiss, D.J.; Mappin, B.; Dalrymple, U.; Bhatt, S.; Cameron, E.; Hay, S.I.; Gething, P.W. Re-examining environmental correlates of Plasmodium falciparum malaria endemicity: A data-intensive variable selection approach. Malar. J. 2015, 14, 68. [Google Scholar] [CrossRef]
- Zhang, L.; Zhang, L.; Du, B. Deep learning for remote sensing data: A technical tutorial on the state of the art. IEEE Geosci. Remote Sens. Mag. 2016, 4, 22–40. [Google Scholar] [CrossRef]
- Kussul, N.; Lavreniuk, M.; Skakun, S.; Shelestov, A. Deep learning classification of land cover and crop types using remote sensing data. IEEE Geosci. Remote Sense. Lett. 2017, 14, 778–782. [Google Scholar] [CrossRef]
- Nogueira, K.; Fadel, S.; Dourado, Í.; Werneck, R.; Muñoz, J.; Penatti, O.; Calumby, R.; Li, L.; Santos, J.; Torres, R. Exploiting ConvNet diversity for flooding identification. IEEE Geosci. Remote Sense. Lett. 2018, 15, 1446–1450. [Google Scholar] [CrossRef]
- Malek, S.; Melgani, F.; Bazi, Y.; Alajlan, N. Reconstructing cloud-contaminated multispectral images with contextualized autoencoder neural networks. IEEE Trans. Geosci. Remote Sens. 2018, 56, 2270–2282. [Google Scholar] [CrossRef]
- Zhang, Q.; Yuan, Q.; Zeng, C.; Li, X.; Wei, Y. Missing data reconstruction in remote sensing image with a unified spatial–temporal–spectral deep convolutional neural network. IEEE Trans. Geosci. Remote Sens. 2018, 56, 4274–4288. [Google Scholar] [CrossRef]
- Isikdogan, F.; Bovik, A.; Passalacqua, P. Surface water mapping by deep learning. IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens. 2017, 10, 4909–4918. [Google Scholar] [CrossRef]
- Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 5–9 October 2015; pp. 234–241. [Google Scholar]
- Wang, H.; Wang, Y.; Zhang, Q.; Xiang, S.; Pan, C. Gated convolutional neural network for semantic segmentation in high-resolution images. Remote Sens. 2017, 9, 446. [Google Scholar] [CrossRef]
- Wang, F.; Jiang, M.; Qian, C.; Yang, S.; Li, C.; Zhang, H.; Wang, X.; Tang, X. Residual attention network for image classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 6450–6458. [Google Scholar]
- Yang, H.; Wu, P.; Yao, X.; Wu, Y.; Wang, B.; Xu, Y. Building Extraction in Very High Resolution Imagery by Dense-Attention Networks. Remote Sens. 2018, 10, 1768. [Google Scholar] [CrossRef]
- Fu, J.; Zheng, H.; Mei, T. Look closer to see better: Recurrent attention convolutional neural network for fine-grained image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 4476–4484. [Google Scholar]
- Xu, K.; Ba, J.; Kiros, R.; Cho, K.; Courville, A.; Salakhutdinov, R.; Zemel, R.; Bengio, Y. Show, attend and tell: Neural image caption generation with visual attention. arXiv, 2016; arXiv:1502.03044v3. [Google Scholar]
- Chen, L.; Zhang, H.; Xiao, J.; Nie, L.; Shao, J.; Liu, W.; Chua, T.S. SCA-CNN: Spatial and channel-wise attention in convolutional networks for image captioning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 6298–6306. [Google Scholar]
- Yang, Z.; He, X.; Gao, J.; Deng, L.; Smola, A. Stacked attention networks for image question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 26 June–1 July 2016; pp. 21–29. [Google Scholar]
- Chen, K.; Wang, J.; Chen, L.C.; Gao, H.; Xu, W.; Nevatia, R. ABC-CNN: An attention based convolutional neural network for visual question answering. arXiv, 2015; arXiv:1511.05960v2. [Google Scholar]
- Yao, L.; Torabi, A.; Cho, K.; Ballas, N.; Pal, C.; Larochelle, H.; Courville, A. Describing videos by exploiting temporal structure. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 7–13 December 2015; pp. 4507–4515. [Google Scholar]
- Kuen, J.; Wang, Z.; Wang, G. Recurrent attentional networks for saliency detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 26 June–1 July 2016; pp. 3668–3677. [Google Scholar]
- Pede, T.; Mountrakis, G. An empirical comparison of interpolation methods for MODIS 8-day land surface temperature composites across the conterminous Unites States. ISPRS J. Photogramm. Remote Sens. 2018, 142, 137–150. [Google Scholar] [CrossRef]
- Friedl, M.A.; Mciver, D.K.; Hodges, J.C.F.; Zhang, X.Y.; Muchoney, D.; Strahler, A.H.; Woodcock, C.E.; Gopal, S.; Schneider, A.; Cooper, A.; et al. Global land cover mapping from modis: Algorithms and early results. Remote Sens. Environ. 2002, 83, 287–302. [Google Scholar] [CrossRef]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 7–13 December 2015; pp. 1026–1034. [Google Scholar]
- Shen, H.; Li, T.; Yuan, Q.; Zhang, L. Estimating regional ground-level PM2.5 directly from satellite top-of-atmosphere reflectance using deep belief networks. J. Geophys. Res. 2018, 123, 875–886. [Google Scholar]
Down-Sampling | Up-Sampling | ||||
---|---|---|---|---|---|
Level | Layer | Size | Layer | Level | |
L1 | {conv 3 × 3, 64, batch normalization} × 2 | 1 | 1 | {conv 3 × 3, 64, batch normalization} × 2 conv 3 × 3, 32, batch normalization conv 3 × 3, 1 | L7 |
L2 | {conv 3 × 3, 128, batch normalization} × 2 | 1/2 | 1/2 | {conv 3 × 3, 128, batch normalization} × 2 | L6 |
L3 | {conv 3 × 3, 256, batch normalization} × 2 | 1/4 | 1/4 | {conv 3 × 3, 256, batch normalization} × 2 | L5 |
L4 | {conv 3 × 3, 512, batch normalization} × 2 | 1/8 |
RMSEs with Different MDR | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
53% Concentrated | 53% Scattered | 65% | 70% | |||||||
MFCTR-CNN | Spline | MFCTR-CNN | Spline | MFCTR-CNN | Spline | MFCTR-CNN | Spline | |||
Next Summer | 201607-S1 | 0.98 | 1.24 | 0.88 | 0.99 | 1.02 | 1.07 | 0.97 | 1.13 | |
201607-S2 | 0.98 | 1.54 | 1.02 | 1.39 | 1.12 | 1.45 | 1.19 | 1.58 | ||
201607-S3 | 0.82 | 1.05 | 0.70 | 0.82 | 0.76 | 0.90 | 0.78 | 0.92 | ||
FY-2G | Summer | 201508-S1 | 0.95 | 1.37 | 0.88 | 1.05 | 1.05 | 1.14 | 0.96 | 1.17 |
201508-S2 | 0.74 | 1.87 | 0.75 | 1.43 | 0.85 | 1.45 | 0.89 | 1.59 | ||
201508-S3 | 0.81 | 1.10 | 0.67 | 0.84 | 0.76 | 0.89 | 0.76 | 0.94 | ||
Fall | 201511-S1 | 0.89 | 1.22 | 0.95 | 0.97 | 1.07 | 1.00 | 0.99 | 1.07 | |
201511-S2 | 0.85 | 1.22 | 0.89 | 1.00 | 1.02 | 1.05 | 0.95 | 1.08 | ||
201511-S3 | 0.59 | 1.09 | 0.61 | 0.89 | 0.64 | 0.94 | 0.67 | 0.97 | ||
Winter | 201602-S1 | 0.97 | 1.44 | 0.95 | 1.16 | 0.98 | 1.24 | 0.99 | 1.30 | |
201602-S2 | 0.84 | 1.50 | 0.81 | 1.23 | 0.85 | 1.23 | 0.89 | 1.35 | ||
201602-S3 | 0.82 | 1.44 | 0.81 | 1.15 | 0.84 | 1.21 | 0.84 | 1.29 | ||
Spring | 201605-S1 | 0.74 | 1.48 | 0.80 | 1.26 | 1.04 | 1.42 | 0.95 | 1.36 | |
201605-S2 | 0.68 | 1.78 | 0.83 | 1.47 | 0.93 | 1.55 | 0.85 | 1.65 | ||
201605-S3 | 0.85 | 1.13 | 0.89 | 1.04 | 0.81 | 1.19 | 0.76 | 1.13 | ||
Next Summer | 201107-S1 | 0.91 | 2.50 | 1.02 | 2.25 | 0.92 | 2.06 | 0.95 | 2.26 | |
201107-S2 | 0.89 | 2.60 | 1.04 | 2.42 | 0.98 | 2.17 | 0.97 | 2.38 | ||
201107-S3 | 0.75 | 1.84 | 1.13 | 1.72 | 0.89 | 1.57 | 0.79 | 1.74 | ||
MSG | Summer | 201008-S1 | 0.86 | 2.38 | 0.87 | 2.06 | 0.87 | 1.99 | 0.90 | 2.20 |
201008-S2 | 0.82 | 2.33 | 0.87 | 1.99 | 0.94 | 1.97 | 0.90 | 2.13 | ||
201008-S3 | 0.69 | 1.75 | 0.99 | 1.57 | 0.81 | 1.60 | 0.73 | 1.71 | ||
Fall | 201011-S1 | 0.89 | 1.64 | 0.91 | 1.53 | 0.96 | 1.52 | 0.95 | 1.61 | |
201011-S2 | 0.79 | 1.75 | 0.87 | 1.60 | 0.83 | 1.58 | 0.85 | 1.72 | ||
201011-S3 | 0.70 | 1.99 | 0.70 | 1.79 | 0.69 | 1.72 | 0.71 | 1.93 | ||
Winter | 201102-S1 | 0.79 | 1.60 | 0.93 | 1.43 | 0.92 | 1.42 | 0.91 | 1.52 | |
201102-S2 | 0.75 | 1.71 | 0.82 | 1.55 | 0.84 | 1.57 | 0.87 | 1.67 | ||
201102-S3 | 0.73 | 1.89 | 0.75 | 1.63 | 0.77 | 1.59 | 0.75 | 1.72 | ||
Spring | 201105-S1 | 0.79 | 1.81 | 0.72 | 1.38 | 0.83 | 1.63 | 0.86 | 1.74 | |
201105-S2 | 0.62 | 1.85 | 0.57 | 1.43 | 0.68 | 1.64 | 0.75 | 1.77 | ||
201105-S3 | 1.06 | 1.11 | 0.55 | 0.89 | 0.91 | 1.09 | 1.00 | 1.12 |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Wu, P.; Yin, Z.; Yang, H.; Wu, Y.; Ma, X. Reconstructing Geostationary Satellite Land Surface Temperature Imagery Based on a Multiscale Feature Connected Convolutional Neural Network. Remote Sens. 2019, 11, 300. https://doi.org/10.3390/rs11030300
Wu P, Yin Z, Yang H, Wu Y, Ma X. Reconstructing Geostationary Satellite Land Surface Temperature Imagery Based on a Multiscale Feature Connected Convolutional Neural Network. Remote Sensing. 2019; 11(3):300. https://doi.org/10.3390/rs11030300
Chicago/Turabian StyleWu, Penghai, Zhixiang Yin, Hui Yang, Yanlan Wu, and Xiaoshuang Ma. 2019. "Reconstructing Geostationary Satellite Land Surface Temperature Imagery Based on a Multiscale Feature Connected Convolutional Neural Network" Remote Sensing 11, no. 3: 300. https://doi.org/10.3390/rs11030300