Attentively Learning Edge Distributions for Semantic Segmentation of Remote Sensing Imagery
<p>Illustration of non-local block.</p> "> Figure 2
<p>Overall framework of EDENet.</p> "> Figure 3
<p>Pipeline of EDA.</p> "> Figure 4
<p>Pipeline of HAM.</p> "> Figure 5
<p>Illustration of ISPRS data samples. (<b>a</b>) Raw image of Potsdam dataset, (<b>b</b>) annotated ground truth of (<b>a</b>), (<b>c</b>) raw image of Vaihingen dataset, (<b>d</b>) annotated ground truth of (<b>c</b>).</p> "> Figure 6
<p>Illustration of DeepGlobe data samples. (<b>a</b>) Raw image, (<b>b</b>) annotated ground truth of (<b>a</b>), (<b>c</b>) raw image, (<b>d</b>) annotated ground truth of (<b>c</b>).</p> "> Figure 7
<p>Visual inspections of random samples from Vaihingen test set. (<b>a</b>) raw image, (<b>b</b>) ground truth, (<b>c</b>) SegNet, (<b>d</b>) U-Net, (<b>e</b>) DeepLab V3+, (<b>f</b>) CBAM, (<b>g</b>) DANet, (<b>h</b>) NLNet, (<b>i</b>) OCRNet, (<b>j</b>) ResUNet-a, (<b>k</b>) SCAttNet, (<b>l</b>) EDENet.</p> "> Figure 8
<p>Visual inspections of random samples from Potsdam test set. (<b>a</b>) raw image, (<b>b</b>) ground truth, (<b>c</b>) SegNet, (<b>d</b>) U-Net, (<b>e</b>) DeepLab V3+, (<b>f</b>) CBAM, (<b>g</b>) DANet, (<b>h</b>) NLNet, (<b>i</b>) OCRNet, (<b>j</b>) ResUNet-a, (<b>k</b>) SCAttNet, (<b>l</b>) EDENet.</p> "> Figure 9
<p>Visual inspections of random samples from DeepGlobe test set. (<b>a</b>) raw image, (<b>b</b>) ground truth, (<b>c</b>) SegNet, (<b>d</b>) U-Net, (<b>e</b>) DeepLab V3+, (<b>f</b>) CBAM, (<b>g</b>) DANet, (<b>h</b>) NLNet, (<b>i</b>) OCRNet, (<b>j</b>) ResUNet-a, (<b>k</b>) SCAttNet, (<b>l</b>) EDENet.</p> "> Figure A1
<p>Training loss of Vaihingen dataset.</p> "> Figure A2
<p>Training mIoU of Vaihingen dataset.</p> "> Figure A3
<p>Training loss of Potsdam dataset.</p> "> Figure A4
<p>Training mIoU of Potsdam dataset.</p> "> Figure A5
<p>Training loss of DeepGlobe dataset.</p> "> Figure A6
<p>Training mIoU of DeepGlobe dataset.</p> ">
Abstract
:1. Introduction
- (1)
- Inspired by the image covariance analysis of 2DPCA, the covariance matrix (CM) is re-defined with learnt feature maps in the network. Then, the edge distribution attention module (EDA) is devised based on the covariance matrix analysis, modeling the dependencies of edge distributions in a self-attentive way explicitly. Through the column-wise and row-wise edge attention maps, the vertical and horizontal relationships are both quantified and leveraged. Specifically, in EDA, the handcrafted feature is successfully combined with learnt ones.
- (2)
- A hybrid attention module (HAM) that emphasizes the edge distributions and position-wise dependencies is devised. Thereby, more complementary edge and contextual information are collected and injected. This module supports independent and flexible embedding by a parallel architecture.
- (3)
- A conceptually end-to-end neural network, named edge distribution-enhanced semantic segmentation neural network (EDENet), is proposed. EDENet hierarchically integrates HAM to generate representative and discriminative encoded features, providing available and reasonable cues for dense prediction.
- (4)
2. Preliminaries
2.1. Attention Mechanism
2.2. Revisiting 2DPCA
2.3. Non-Local Block
3. The Proposed Method
3.1. Overview
3.2. Edge Distribution Attention Module
3.2.1. Re-Defining Covariance Matrix for Feature Matrix
3.2.2. Edge Distribution Attention Module
3.3. Hybrid Attention Module
4. Experiments and Results
4.1. Experimental Settings
4.1.1. Datasets
- ISPRS Vaihingen dataset
- 2.
- ISPRS Potsdam dataset
- 3.
- DeepGlobe dataset
4.1.2. Hyper-Parameters and Implementation Details
4.1.3. Numerical Metrics
4.2. Comparison with State-of-the-Art
4.2.1. Results on Vaihingen Dataset
4.2.2. Results on Potsdam Dataset
4.2.3. Results on DeepGlobe Dataset
4.3. Ablation Study of EDA
5. Discussions
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Appendix A. Ablation Study on the Vaihingen Dataset
Appendix B. Ablation Study on the Potsdam Dataset
Appendix C. Ablation Study on the DeepGlobe Dataset
References
- Xia, M.; Cui, Y.; Zhang, Y.; Xu, Y.; Xu, Y. DAU-Net: A novel water areas segmentation structure for remote sensing image. Int. J. Remote Sens. 2021, 42, 2594–2621. [Google Scholar] [CrossRef]
- Weng, L.; Xu, Y.; Xia, M.; Zhang, Y.; Xu, Y. Water areas segmentation from remote sensing images using a separable Residual SegNet network. ISPRS Int. J. Geo-Inf. 2020, 9, 256. [Google Scholar] [CrossRef]
- Yang, Q.; Liu, M.; Zhang, Z.; Yang, S.; Ning, J.; Han, W. Mapping Plastic mulched farmland for high resolution images of unmanned aerial vehicle using deep semantic segmentation. Remote Sens. 2019, 11, 2008. [Google Scholar] [CrossRef] [Green Version]
- Tong, X.; Xia, G.; Lu, Q.; Shen, H.; Li, S.; You, S.; Zhang, L. Land-cover classification with high-resolution remote sensing images using transferable deep models. Remote Sens. Environ. 2020, 237, 11132. [Google Scholar] [CrossRef] [Green Version]
- Henry, C.J.; Storie, C.D.; Palaniappan, M.; Alhassan, V.; Swamy, M.; Aleshinloye, D.; Curtis, A.; Kim, D. Automated LULC map production using deep neural networks. Int. J. Remote Sens. 2019, 40, 4416–4440. [Google Scholar] [CrossRef]
- Shi, H.; Chen, L.; Bi, F.; Chen, H.; Yu, Y. Accurate urban area detection in remote sensing images. IEEE Geosci. Remote Sens. Lett. 2015, 12, 1948–1952. [Google Scholar] [CrossRef]
- Wegner, J.D.; Branson, S.; Hall, D.; Schindler, K.; Perona, P. Cataloging public objects using aerial and street-level images-Urban trees. In Proceedings of the 29th IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 26 June–1 July 2016; pp. 6014–6023. [Google Scholar]
- Ozdarici, A.; Schindler, K. Mapping of agricultural crops from single high-resolution multispectral images data-driven smoothing vs. parcel-based smoothing. Remote Sens. 2015, 7, 5611–5638. [Google Scholar] [CrossRef] [Green Version]
- Gibril, M.B.A.; Shafri, H.Z.M.; Shanableh, A.; Al-Ruzouq, R.; Wayayok, A.; Hashim, S.J. Deep convolutional neural network for large-scale date palm tree mapping from UAV-based images. Remote Sens. 2021, 13, 2787. [Google Scholar] [CrossRef]
- Minaee, S.; Boykov, Y.; Porikli, F.; Plaza, A.J.; Kehtarnavaz, N.; Terzopoulos, D. Image segmentation using deep learning: A survey. IEEE Trans. Pattern Anal. Mach. Intell. 2021. accepted. [Google Scholar] [CrossRef]
- Arivazhagan, S.; Ganesan, L.; Priyal, S. Texture classification using Gabor wavelets based rotation invariant features. Pattern Recognit. Lett. 2006, 27, 1976–1985. [Google Scholar] [CrossRef]
- Gitelson, A.; Merzlyak, M. Signature analysis of leaf reflectance spectra: Algorithm development for remote sensing of chlorophyll. J. Plant Physiol. 1996, 148, 494–500. [Google Scholar] [CrossRef]
- Peñ, A.; José, M.; Ngugi, M.; Richard, E.P.; Johan, S. Object-based crop identification using multiple vegetation indices, textural features and crop phenology. Remote Sens. Environ. 2011, 115, 1301–1316. [Google Scholar]
- Blaschke, T. Object based image analysis for remote sensing. ISPRS J. Photogramm. Remote Sens. 2010, 65, 2–16. [Google Scholar] [CrossRef] [Green Version]
- Yang, J.; Jiang, Z.; Zhou, Q. Remote sensing image semantic labeling based on conditional random field. Acta Aeronaut. Astronaut. Sin. 2015, 36, 3069–3081. [Google Scholar]
- Mountrakis, G.; Im, J.; Ogole, C. Support vector machines in remote sensing: A review. ISPRS J. Photogramm. Remote Sens. 2011, 66, 247–259. [Google Scholar] [CrossRef]
- Belgiu, M.; Lucian, D. Random forest in remote sensing: A review of applications and future directions. ISPRS J. Photogramm. Remote Sens. 2016, 114, 24–31. [Google Scholar] [CrossRef]
- Long, J.; Shelhamer, E.; Darrell, T. Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 2015, 39, 640–651. [Google Scholar]
- Badrinarayanan, V.; Kendall, A.; Cipolla, R. SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 2481–2495. [Google Scholar] [CrossRef] [PubMed]
- Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional networks for biomedical image segmentation. In Proceedings of the Medical Image Computing and Computer-Assisted Intervention (MCCAI), Munich, Germany, 5–9 October 2015; Volume 9351, pp. 234–241. [Google Scholar]
- Chen, L.; Papandreou, G.; Kokkinos, L.; Murphy, K.; Yuille, A. Semantic image segmentation with deep convolutional nets and fully connected CRFs. In Proceedings of the 3rd International Conference on Learning Representations (ICLR), San Diego, CA, USA, 7–9 May 2015. [Google Scholar]
- Chen, L.; Papandreou, G.; Kokkinos, I.; Murphy, K.; Yuille, A. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 2018, 40, 834–848. [Google Scholar] [CrossRef] [PubMed]
- Chen, L.; Papandreou, G.; Schroff, F.; et al. Rethinking atrous convolution for semantic image segmentation. arXiv 2017, arXiv:1706.05587. [Google Scholar]
- Chen, L.; Zhu, Y.; Papandreou, G.; Adam, H. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the 15th European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 833–851. [Google Scholar]
- Zhang, J.; Lin, S.; Ding, L.; Bruzzone, L. Multi-scale context aggregation for semantic segmentation of remote sensing images. Remote Sens. 2020, 12, 701. [Google Scholar] [CrossRef] [Green Version]
- Sun, K.; Xiao, B.; Liu, D.; Wang, J. Deep high-resolution representation learning for human pose estimation. In Proceedings of the 32nd IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 16–20 June 2019; pp. 5686–5696. [Google Scholar]
- Chen, B.; Xia, M.; Huang, J. MFANet: A multi-level feature aggregation network for semantic segmentation of land cover. Remote Sens. 2021, 13, 731. [Google Scholar] [CrossRef]
- Wang, E.; Jiang, Y. MFCSNet: Multi-scale deep features fusion and cost-sensitive loss function based segmentation network for remote sensing images. Appl. Sci. 2019, 9, 4043. [Google Scholar] [CrossRef] [Green Version]
- Hu, J.; Shen, L.; Albanie, S.; Sun, G.; Wu, E. Squeeze-and-excitation networks. IEEE Trans. Pattern Anal. Mach. Intell. 2020, 42, 2011–2023. [Google Scholar] [CrossRef] [Green Version]
- Woo, S.; Park, J.; Lee, J.; Kweon, I. CBAM: Convolutional block attention module. In Proceedings of the 15th European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 3–19. [Google Scholar]
- Fu, J.; Liu, J.; Tian, H.; Li, Y.; Bao, Y.; Fang, Z.; Lu, H. Dual attention network for scene segmentation. In Proceedings of the 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 16–20 June 2019; pp. 3141–3149. [Google Scholar]
- Wang, X.; Girshick, R.; Gupta, A.; He, K. Non-local neural networks. In Proceedings of the 31st Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, 18–22 June 2018; pp. 7794–7803. [Google Scholar]
- Zhang, F.; Chen, Y.; Li, Z.; Hong, Z.; Ding, E. ACFNet: Attentional class feature network for semantic segmentation. In Proceedings of the 17th IEEE International Conference on Computer Vision (ICCV), Seoul, Korea, 27 October–2 November 2019; pp. 6797–6806. [Google Scholar]
- Yuan, Y.; Chen, X.; Wang, J. Object-contextual representations for Semantic Segmentation. In Proceedings of the 16th European Conference on Computer Vision (ECCV), 23–28 August 2020; pp. 173–190. [Google Scholar]
- Ghaffarian, S.; Valente, J.; Voort, M.; Tekinerdogan, B. Effect of attention mechanism in deep learning-based remote sensing image processing: A systematic literature review. Remote Sens. 2021, 13, 2965. [Google Scholar] [CrossRef]
- Luo, H.; Chen, C.; Fang, L.; Zhu, X.; Lu, L. High-resolution aerial images semantic segmentation using deep fully convolutional network with channel attention mechanism. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2019, 12, 3492–3507. [Google Scholar] [CrossRef]
- Niu, R.; Sun, X.; Tian, Y.; Diao, W.; Chen, K.; Fu, K. Hybrid multiple attention network for semantic segmentation in aerial images. IEEE Trans. Geosci. Remote Sens. 2021. accepted. [Google Scholar] [CrossRef]
- Li, H.; Qiu, K.; Chen, L.; Mei, X.; Hong, L.; Tao, C. SCAttNet: Semantic segmentation network with spatial and channel attention mechanism for high-resolution remote sensing images. IEEE Geosci. Remote Sens. Lett. 2021, 18, 905–909. [Google Scholar] [CrossRef]
- Ding, L.; Tang, H.; Bruzzone, L. LANet: Local attention embedding to improve the semantic segmentation of remote sensing images. IEEE Trans. Geosci. Remote Sens. 2020. accepted. [Google Scholar] [CrossRef]
- Marmanis, D.; Schindler, K.; Wegner, J.; Galliani, S.; Datcu, M.; Stilla, U. Classification with an edge: Improving semantic image segmentation with boundary detection. ISPRS J. Photogramm. Remote Sens. 2018, 135, 158–172. [Google Scholar] [CrossRef] [Green Version]
- Pan, S.; Tao, Y.; Nie, C.; Chong, Y. PEGNet: Progressive edge guidance network for semantic segmentation of remote sensing images. IEEE Geosci. Remote Sens. Lett. 2021, 18, 637–641. [Google Scholar] [CrossRef]
- Yang, J.; Zhang, D.; Frangi, A.; Yang, J. Two-dimensional PCA: A new approach to appearance-based face representation and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2004, 26, 131–137. [Google Scholar] [CrossRef] [Green Version]
- ISPRS Vaihingen 2D Semantic Labeling Dataset. Available online: http://www2.isprs.org/commissions/comm3/wg4/2d-sem-label-vaihingen.html (accessed on 10 December 2017).
- ISPRS Potsdam 2D Semantic Labeling Dataset. Available online: http://www2.isprs.org/commissions/comm3/wg4/2d-sem-label-potsdam.html (accessed on 10 December 2017).
- Ilke, D.; et al. DeepGlobe 2018: A challenge to parse the Earth through satellite images. In Proceedings of the 31th IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA, 18–22 June 2018; pp. 172–181. [Google Scholar]
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.; Kaiser, L.; Polosukhin, I. Attention is all you need. In Proceedings of the 31st Annual Conference on Neural Information Processing Systems (NIPS), Long Beach, CA, USA, 4–9 December 2017; pp. 5999–6009. [Google Scholar]
- Wei, J.; He, J.; Zhou, Y.; Chen, K.; Tang, Z.; Xiong, Z. Enhanced object detection with deep convolutional neural networks for advanced driving assistance. IEEE Trans. Intell. Transp. Syst. 2020, 21, 1572–1583. [Google Scholar] [CrossRef] [Green Version]
- Techimann, M.; Weber, M.; Zollnr, M.; Cipolla, R.; Urtasun, R. MultiNet: Real-time joint semantic reasoning for autonomous driving. In Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China, 26–30 June 2018. [Google Scholar]
- Haut, J.; Fernandez-Beltran, R.; Paolett, M.; Plaza, J.; Plaza, A. Remote sensing image super-resolution using deep residual channel attention. IEEE Trans. Geosci. Remote Sens. 2019, 57, 9277–9289. [Google Scholar] [CrossRef]
- Li, H.; Liu, Y.; Ouyang, W.; Wang, X. Zoom out-and-in network with map attention decision for region proposal and object detection. Int. J. Comput. Vis. 2019, 127, 225–238. [Google Scholar] [CrossRef] [Green Version]
- Liu, S.; Johns, E.; Davison, A. End-to-End Multi-Task Learning with Attention. arXiv arXiv:1803.10704, 2019.
- Maninis, K.; Radosavovic, I.; Kokkinos, I. Attentive single-tasking of multiple tasks. In Proceedings of the 32nd IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 16–20 June 2019; pp. 1851–1860. [Google Scholar]
- Huang, Z.; Wang, X.; Huang, L.; Huang, C.; Wei, Y.; Liu, W. CCNet: Criss-cross attention for semantic segmentation. In Proceedings of the 17th IEEE International Conference on Computer Vision (ICCV), Seoul, Korea, 27 October–2 November 2019; pp. 603–612. [Google Scholar]
- Yang, Z.; He, X.; Gao, J.; Deng, L.; Smola, A. Stacked attention networks for image question answering. In Proceedings of the 29th IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 26 June–1 July 2016; pp. 21–29. [Google Scholar]
- Li, H.; Xiong, P.; An, J.; Wang, L. Pyramid attention network for semantic segmentation. In Proceedings of the 29th British Machine Vision Conference (BMVC), Newcastle, UK, 3–6 September 2018. [Google Scholar]
- Zhu, L.; Wang, T.; Aksu, E.; Kamarainen, J. Cross-granularity attention network for semantic segmentation. In Proceedings of the 17th IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Korea, 27–28 October 2019; pp. 1920–1930. [Google Scholar]
- Yang, J.; Yang, J. From image vector to matrix: A straightforward image projection technique—IMPCA vs. PCA. Pattern Recognit. 2002, 35, 1997–1999. [Google Scholar] [CrossRef]
- Foivos, D.; François, W.; Peter, C.; Wu, C. ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data. ISPRS J. Photogramm. Remote Sens. 2020, 162, 94–114. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the 29th IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 26 June–1 July 2016; pp. 770–778. [Google Scholar]
- Ganguly, S. Multi-objective distributed generation penetration planning with load model using particle swarm optimization. Decis. Mak. Appl. Manag. Eng. 2020, 3, 30–42. [Google Scholar] [CrossRef]
- Xie, Q.; Zhou, M.; Zhao, Q.; Meng, D.; Zuo, W.; Xu, Z. Multispectral and hyperspectral image fusion by MS/HS fusion net. In Proceedings of the 32nd IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 16–20 June 2019. [Google Scholar]
- Chen, Y.; Huang, T.; He, W.; Zhao, X.; Zhang, H.; Zeng, J. Hyperspectral image denoising using factor group sparsity-regularized nonconvex low-rank approximation. IEEE Trans. Geosci. Remote Sens. 2021, 1–16. [Google Scholar] [CrossRef]
- He, W.; Yao, Q.; Li, C.; Yokoya, N.; Zhao, Q.; Zhang, H.; Zhang, L. Non-local meets global: An integrated paradigm for hyperspectral image restoration. IEEE Trans. Pattern Anal. Mach. Intell. 2020, 1. [Google Scholar] [CrossRef] [PubMed]
Datasets | Vaihingen | Potsdam | DeepGlobe |
---|---|---|---|
Bands used | NIR, R, G | NIR, R, G | R, G, B |
GSD | 9 cm | 5 cm | 0.5 m |
Number of available images | 16 | 24 | 803 |
Spatial size | 2500 × 2500 | 6000 × 6000 | 2448 × 2448 |
Imaging sensors | Airborne | Airborne | Satellite |
Datasets | Vaihingen | Potsdam | DeepGlobe |
---|---|---|---|
Backbone | ResNet 101 | ResNet 101 | ResNet 101 |
Batch size | 16 | 16 | 16 |
Learning strategy | Poly decay | Poly decay | Poly decay |
Initial learning rate | 0.002 | 0.002 | 0.002 |
Loss Function | Cross-entropy | Cross-entropy | Cross-entropy |
Optimizer | Adam | Adam | Adam |
Max epoch | 500 | 500 | 200 |
Sub-patch size | 256 × 256 | 256 × 256 | 256 × 256 |
Total number of sub-patches | 1520 | 8576 | 65043 |
Training set (number of sub-patches) | 1216 | 6860 | 52035 |
Validation set (number of sub-patches) | 152 | 858 | 6504 |
Test set (number of sub-patches) | 152 | 858 | 6504 |
Data augmentation | Rotate 90, 180 and 270 degrees, horizontally and vertically flip |
Methods | Categories | References |
---|---|---|
SegNet | Classical networks | A deep convolutional encoder-decoder architecture for image segmentation [19] |
U-Net | Convolutional networks for biomedical image segmentation [20] | |
DeepLab V3+ | Encoder-decoder with atrous separable convolution for semantic image segmentation [24] | |
CBAM | Attention-based networks | Convolutional block attention module [30] |
DANet | Dual attention network for scene segmentation [31] | |
NLNet | Non-local neural networks [32] | |
OCRNet | Object-contextual representations for Semantic Segmentation [34] | |
ResUNet-a | RSI-specific networks | A deep learning framework for semantic segmentation of remotely sensed data [58] |
SCAttNet | Semantic segmentation network with spatial and channel attention mechanism for high-resolution remote sensing images [38] | |
EDENet | Ours | / |
Methods | Impervious Surfaces | Building | Low Vegetation | Tree | Car | Clutter | OA | mIoU |
---|---|---|---|---|---|---|---|---|
SegNet [19] | 92.26/78.77 | 90.85/75.62 | 80.51/62.04 | 77.13/57.91 | 60.54/51.05 | 73.30/54.73 | 79.10 | 63.35 |
U-Net [20] | 92.60/78.63 | 90.73/76.52 | 79.85/62.17 | 77.58/58.26 | 69.79/55.20 | 74.32/59.41 | 80.81 | 65.03 |
DeepLab V3+ [24] | 93.44/80.41 | 89.85/77.43 | 81.05/66.53 | 78.02/62.94 | 70.41/58.76 | 77.43/70.19 | 81.70 | 69.38 |
CBAM [30] | 93.34/82.98 | 89.92/78.41 | 82.18/65.70 | 77.89/62.25 | 71.04/66.74 | 75.51/63.49 | 81.65 | 69.93 |
DANet [31] | 93.52/83.76 | 90.04/78.15 | 83.22/69.21 | 78.46/63.21 | 70.87/65.35 | 76.54/61.59 | 82.11 | 70.21 |
NLNet [32] | 93.21/84.00 | 91.12/79.21 | 84.40/68.33 | 79.53/63.14 | 72.17/67.97 | 79.45/65.66 | 83.31 | 71.38 |
OCRNet [34] | 96.13/86.12 | 91.62/80.38 | 89.11/70.87 | 83.88/65.19 | 72.08/67.52 | 78.65/65.67 | 85.25 | 72.62 |
ResUNet-a [54] | 93.50/87.17 | 97.12/81.29 | 85.21/70.68 | 85.83/66.55 | 79.92/71.17 | 81.91/75.74 | 87.25 | 75.43 |
SCAttNet [37] | 89.13/84.50 | 92.58/80.59 | 86.97/70.29 | 85.31/63.56 | 75.50/68.45 | 82.83/69.21 | 85.39 | 72.77 |
EDENet | 96.69/88.06 | 97.15/82.22 | 89.44/71.15 | 90.52/70.48 | 84.84/73.70 | 84.17/75.84 | 90.47 | 76.91 |
Methods | Impervious Surfaces | Building | Low Vegetation | Tree | Car | Clutter | OA | mIoU |
---|---|---|---|---|---|---|---|---|
SegNet [19] | 92.12/78.66 | 90.72/75.51 | 80.40/61.94 | 77.01/57.83 | 60.45/50.97 | 73.19/54.65 | 78.98 | 63.26 |
U-Net [20] | 92.46/78.51 | 90.60/76.41 | 79.74/62.08 | 77.46/58.17 | 69.68/55.12 | 74.21/59.32 | 80.69 | 64.94 |
DeepLab V3+ [24] | 93.30/80.30 | 89.72/77.31 | 80.93/66.43 | 77.90/62.85 | 70.30/58.67 | 77.31/70.08 | 81.58 | 69.28 |
CBAM [30] | 93.21/82.86 | 89.78/78.30 | 82.06/65.60 | 77.77/62.16 | 70.93/66.64 | 75.40/63.40 | 81.53 | 69.83 |
DANet [31] | 93.39/83.63 | 89.90/78.04 | 83.10/69.11 | 78.34/63.12 | 70.76/65.25 | 76.43/61.50 | 81.99 | 70.11 |
NLNet [32] | 93.08/83.87 | 90.99/79.10 | 84.28/68.23 | 79.41/63.05 | 72.07/67.87 | 79.33/65.56 | 83.19 | 71.28 |
OCRNet [34] | 96.15/86.12 | 91.63/80.38 | 89.12/70.87 | 83.90/65.19 | 72.09/67.52 | 78.66/65.67 | 85.27 | 72.62 |
ResUNet-a [54] | 93.51/87.19 | 97.14/81.30 | 85.22/70.70 | 85.84/66.56 | 79.93/71.18 | 81.92/75.75 | 87.28 | 75.45 |
SCAttNet [37] | 89.14/84.51 | 92.59/80.60 | 86.98/70.30 | 85.32/63.57 | 75.51/68.46 | 82.84/69.22 | 85.41 | 72.78 |
EDENet | 96.70/88.07 | 97.17/82.23 | 89.46/71.16 | 90.53/70.49 | 84.85/73.71 | 84.18/75.85 | 90.50 | 76.92 |
Methods | Urban Land | Agriculture Land | Rangeland | Forest Land | Water | Barren Land | Unknown | OA | mIoU |
---|---|---|---|---|---|---|---|---|---|
SegNet | 70.25/50.95 | 81.17/58.87 | 69.09/50.11 | 67.78/49.16 | 83.52/60.57 | 61.13/44.34 | 58.06/42.11 | 70.14 | 50.87 |
U-Net | 76.54/55.51 | 85.66/62.13 | 75.28/54.59 | 73.86/53.57 | 85.15/61.76 | 59.22/42.95 | 58.07/42.12 | 73.40 | 53.23 |
DeepLab V3+ | 77.23/56.01 | 86.18/62.50 | 77.62/56.29 | 74.48/54.02 | 87.14/63.20 | 65.29/47.35 | 61.11/44.32 | 75.58 | 54.81 |
CBAM | 79.51/57.67 | 87.75/63.64 | 79.19/57.43 | 75.56/54.80 | 87.58/63.52 | 67.14/48.69 | 62.02/44.98 | 76.96 | 55.82 |
DANet | 79.48/57.64 | 86.98/63.08 | 79.02/57.31 | 75.79/54.97 | 88.13/63.92 | 67.01/48.60 | 63.17/45.81 | 77.08 | 55.91 |
NLNet | 79.92/57.96 | 87.52/63.48 | 79.97/58.00 | 76.61/55.56 | 88.06/63.87 | 66.95/48.56 | 63.59/46.12 | 77.52 | 56.22 |
OCRNet | 80.73/58.55 | 88.35/64.08 | 81.10/58.82 | 77.91/56.51 | 89.33/64.79 | 68.08/49.38 | 65.57/47.56 | 78.72 | 57.10 |
ResUNet-a | 79.03/57.32 | 90.13/65.37 | 79.67/57.78 | 79.92/57.96 | 88.21/63.98 | 77.02/55.86 | 70.88/51.41 | 80.69 | 58.52 |
SCAttNet | 76.55/55.52 | 87.30/63.31 | 77.16/55.96 | 80.44/58.34 | 91.23/66.17 | 70.29/50.98 | 68.53/49.70 | 78.79 | 57.14 |
EDENet | 81.59/59.17 | 93.05/67.49 | 82.25/59.65 | 82.51/59.84 | 91.27/66.19 | 79.52/57.67 | 73.18/53.07 | 83.34 | 60.44 |
Models | Vaihingen | Potsdam | DeepGlobe |
---|---|---|---|
non-EDA version | 85.60/73.34 | 85.48/73.24 | 79.65/57.77 |
EDENet | 90.47/76.91 | 90.50/76.92 | 83.34/60.44 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Li, X.; Li, T.; Chen, Z.; Zhang, K.; Xia, R. Attentively Learning Edge Distributions for Semantic Segmentation of Remote Sensing Imagery. Remote Sens. 2022, 14, 102. https://doi.org/10.3390/rs14010102
Li X, Li T, Chen Z, Zhang K, Xia R. Attentively Learning Edge Distributions for Semantic Segmentation of Remote Sensing Imagery. Remote Sensing. 2022; 14(1):102. https://doi.org/10.3390/rs14010102
Chicago/Turabian StyleLi, Xin, Tao Li, Ziqi Chen, Kaiwen Zhang, and Runliang Xia. 2022. "Attentively Learning Edge Distributions for Semantic Segmentation of Remote Sensing Imagery" Remote Sensing 14, no. 1: 102. https://doi.org/10.3390/rs14010102