Semantic Segmentation of SLAR Imagery with Convolutional LSTM Selectional AutoEncoders
"> Figure 1
<p>Scheme of the aircraft data acquisition system during flight.</p> "> Figure 2
<p>Representative image of a Side-Looking Airborne Radar (SLAR) sequence from our dataset (<b>top</b>) and its corresponding ground truth (<b>bottom</b>). Unlike in the previous work [<a href="#B27-remotesensing-11-01402" class="html-bibr">27</a>] in which only oil spills were labeled, in this ground-truth seven classes are labeled: Ships in fuchsia, oil spills in red, lookalikes in green, coast in blue, artifacts below the airplane in light gray, artifacts caused by its turns in dark gray, and water in white. Ships are marked with circles in the top image to help the reader to locate them.</p> "> Figure 3
<p>Architecture of the proposed network. Each scanline is supplied to a series of ConvLSTM Selectional AutoEncoders (CMSAE) networks to process each of the different classes in parallel. The results of these networks are given to a final classifier which performs the prediction.</p> "> Figure 4
<p>Scheme of the CMSAE network specialized for the segmentation of oil spills. In this figure, the layer type is labeled with colors according to the side legend. The size of each layer for convolutions and transposed convolutions is <math display="inline"><semantics> <mrow> <mi>h</mi> <mo>×</mo> <mi>w</mi> </mrow> </semantics></math>, where <span class="html-italic">h</span> is the height and <span class="html-italic">w</span> the width. The size for the ConvLSTM layer is <math display="inline"><semantics> <mrow> <mi>s</mi> <mo>×</mo> <mi>h</mi> <mo>×</mo> <mi>w</mi> </mrow> </semantics></math>, where <span class="html-italic">s</span> is the sequence length. The number of filters (<span class="html-italic">f</span>), the kernel size (<span class="html-italic">k</span>) and the stride value (<math display="inline"><semantics> <mrow> <mi>s</mi> <mi>t</mi> </mrow> </semantics></math>) applied for each layer are also shown. The CMSAE networks used for the segmentation of the other classes follow the same scheme, although their topologies vary.</p> "> Figure 5
<p>Representation of the feature vector extraction process to classify one pixel (marked in red in the figure). From the <span class="html-italic">t</span> neural codes (NC) obtained for a class (using the current <math display="inline"><semantics> <msub> <mi>t</mi> <mi>n</mi> </msub> </semantics></math> and the previous <math display="inline"><semantics> <mrow> <mo>[</mo> <msub> <mi>t</mi> <mrow> <mi>n</mi> <mo>-</mo> <mn>1</mn> </mrow> </msub> <mo>,</mo> <msub> <mi>t</mi> <mrow> <mi>n</mi> <mo>-</mo> <mi>s</mi> <mo>-</mo> <mn>1</mn> </mrow> </msub> <mo>]</mo> </mrow> </semantics></math> scanlines), the <math display="inline"><semantics> <mrow> <mi>s</mi> <mo>×</mo> <mn>5</mn> </mrow> </semantics></math> neighbors to the pixel to be classified are copied to the feature vector. This process is repeated for the NCs obtained for the rest of the 7 classes, finally forming a vector of <math display="inline"><semantics> <mrow> <mi>s</mi> <mo>×</mo> <mn>5</mn> <mo>×</mo> <mn>7</mn> </mrow> </semantics></math>.</p> "> Figure 6
<p>Comparison of the result obtained for each of the classes when varying: (<b>a</b>) The input size of the CMSAE networks and (<b>b</b>) the number of scanlines used as input. See <a href="#remotesensing-11-01402-t005" class="html-table">Table 5</a> for standard deviation of results based on the settings shown in <a href="#remotesensing-11-01402-t002" class="html-table">Table 2</a>.</p> "> Figure 7
<p>Normalized confusion matrix calculated for the seven classes considered at the pixel level. Rows show the current label and columns the prediction given by the specific CMSAE network for the class detection.</p> "> Figure 8
<p>Receiver operating characteristic (ROC) curves of the proposed method for ships and oil spills classes.</p> "> Figure 9
<p>Results of processing two SLAR input images with the proposed method (zooming in the area of interest). The first column shows the original SLAR images and the second column shows the detection results. Black areas depict correct detections of oil spills, red and blue pixels depict FP and FN of oil spills, respectively, and white pixels represents correct detections of the background.</p> "> Figure 10
<p>Comparison of the result obtained from a Multi-objective Optimization Problem (MOP) perspective, comparing the result of the classification and the runtime for the ship and oil spill classes.</p> ">
Abstract
:1. Introduction
- A method designed to work in real time (flight time), in contrast to previous techniques such as [27] which can only be used offline (once all the scanlines are available). For this, a SAE topology was modified in order to work directly with SLAR scanlines, and recurrent neurons were added to take advantage of the information in the previous readings.
- The proposed method uses a parallel set of specialized supervised classifiers for each of the classes, and finally combines their outputs to provide an answer. By combining the classifiers’ decisions, it is possible to consistently improve the results, as demonstrated with statistical tests in Section 3.
- The model was evaluated using 51 different flight sequences with a total of 5.4 flight hours, including a wide range of examples of the different elements to be classified as well as different meteorological and flight conditions (altitude, flight speed, wind speed, etc.).
- The proposed approach is compared with other state-of-the-art methods, reporting better results in both detection and segmentation tasks at the pixel level, as well as better processing time.
2. Materials and Methods
2.1. Materials
2.2. Method
2.2.1. CMSAE
2.2.2. Classifier Integration
- k-Nearest-Neighbors (kNN) [38]: This classifier is one of the most widely used schemes for supervised learning tasks. It classifies a given input element by assigning the most common label among its k-nearest prototypes of the training set according to a similarity function. Different numbers of neighbors have been evaluated in this work.
- Support-Vector Machines (SVM) [39]: It learns a hyperplane that tries to maximize the distance to the nearest samples (support vectors) of each class. In our case, we use the “one-against-rest” approach [40] for multi-class classification and a Radial Basis Function (or Gaussian) kernel to handle non-linear decision boundaries. Typically, an SVM also considers a parameter that measures the cost of learning a non-optimal hyperplane, which is usually referred to as parameter c. For these experiments, we tuned this parameter in the range .
- Random Forest (RaF) [41]: It builds an ensemble classifier by generating several random decision trees at the training stage. The final output is taken by combining the individual decisions of each tree. The number of random trees has been established by experimenting in the range .
2.3. Training Process
3. Results
3.1. Hyperparameters Evaluation
3.2. Final Classifier Evaluation
4. Discussion
- TSCNN [2]: This method employs a two-stage architecture composed of three pairs of CNNs. Each pair of networks is trained to recognize a single class (ship, oil spill, or coast) by following two steps: a first network performs a coarse detection, and then, a second CNN obtains the precise localization. After classification, a postprocessing stage is performed to improve the results.
- U-Net [47]: This CNN was developed for biomedical image segmentation. This network uses a FCN divided into two phases: a contracting path and an expansive path. The feature activations of the contracting path are concatenated with the corresponding layers from the expansive path. The last layer uses a 1x1 convolution with a Softmax activation function to output class labels.
- SegNet [20]: It uses a fully convolutional neural network architecture for semantic pixel-wise segmentation, containing an encoder network, a corresponding decoder network, and a pixel-wise multiclass classification layer. The architecture of the encoder network is topologically identical to the 13 convolutional layers in the VGG16 network [48].
- DeepLabv3 [49]: It uses atrous spatial pyramid pooling to robustly segment objects at multiple scales with filters at multiple sampling rates to explicitly control the resolution at which feature responses are computed within Deep Convolutional Neural Networks. It also includes a image-level feature to capture longer range information and uses batch normalization to facilitate the training.
- SelAE [27]: This approach uses a SAE network specialized in the segmentation of oil spills. It returns a probability distribution over which they apply a threshold to select the pixels to segment.
Runtime Analysis
5. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Lupidi, A.; Staglianò, D.; Martorella, M.; Berizzi, F. Fast Detection of Oil Spills and Ships Using SAR Images. Remote Sens. 2017, 9, 230. [Google Scholar] [CrossRef]
- Nieto-Hidalgo, M.; Gallego, A.; Gil, P.; Pertusa, A. Two-Stage Convolutional Neural Network for Ship and Spill Detection Using SLAR Images. IEEE Trans. Geosci. Remote Sens. 2018, 56, 5217–5230. [Google Scholar] [CrossRef] [Green Version]
- Brekke, C.; Solberg, A.H. Oil spill detection by satellite remote sensing. Remote Sens. Environ. 2005, 95, 1–13. [Google Scholar] [CrossRef]
- Fingas, M.; Brown, C. Review of oil spill remote sensing. Mar. Pollut. Bull. 2014, 83, 9–23. [Google Scholar] [CrossRef] [Green Version]
- Fingas, M.; Brown, C.E. A Review of Oil Spill Remote Sensing. Sensors 2018, 18, 91. [Google Scholar] [CrossRef]
- Leifer, I.; Lehr, W.J.; Simecek-Beatty, D.; Bradley, E.; Clark, R.; Dennison, P.; Hu, Y.; Matheson, S.; Jones, C.E.; Holt, B.; et al. State of the art satellite and airborne marine oil spill remote sensing: Application to the BP Deepwater Horizon oil spill. Remote Sens. Environ. 2012, 124, 185–209. [Google Scholar] [CrossRef] [Green Version]
- Gil, P.; Alacid, B. Oil Spill Detection in Terma-Side-Looking Airborne Radar Images Using Image Features and Region Segmentation. Sensors 2018, 18, 151. [Google Scholar] [CrossRef]
- Topouzelis, K.N. Oil Spill Detection by SAR Images: Dark Formation Detection, Feature Extraction and Classification Algorithms. Sensors 2008, 8, 6642–6659. [Google Scholar] [CrossRef] [Green Version]
- Chang, L.; Tang, Z.; Chang, S.; Chang, Y.L. A region-based GLRT detection of oil spills in SAR images. Pattern Recognit. Lett. 2008, 29, 1915–1923. [Google Scholar] [CrossRef]
- Shu, Y.; Li, J.; Yousif, H.; Gomes, G. Dark-spot detection from SAR intensity imagery with spatial density thresholding for oil-spill monitoring. Remote Sens. Environ. 2010, 114, 2026–2035. [Google Scholar] [CrossRef] [Green Version]
- Zhao, Q.; Li, Y.; Liu, Z. SAR Image Segmentation Using Voronoi Tessellation and Bayesian Inference Applied to Dark Spot Feature Extraction. Sensors 2013, 13, 14484–14499. [Google Scholar] [CrossRef] [Green Version]
- Ren, P.; Xu, M.; Yu, Y.; Chen, F.; Jiang, X.; Yang, E. Energy Minimization With One Dot Fuzzy Initialization for Marine Oil Spill Segmentation. IEEE J. Ocean. Eng. 2018, 1–14. [Google Scholar] [CrossRef]
- Jing, Y.; An, J.; Liu, Z. A Novel Edge Detection Algorithm Based on Global Minimization Active Contour Model for Oil Slick Infrared Aerial Image. IEEE Trans. Geosci. Remote Sens. 2011, 49, 2005–2013. [Google Scholar] [CrossRef]
- Song, H.; Huang, B.; Zhang, K. A Globally Statistical Active Contour Model for Segmentation of Oil Slick in SAR Imagery. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2013, 6, 2402–2409. [Google Scholar] [CrossRef]
- Li, Y.; Cui, C.; Liu, Z.; Liu, B.; Xu, J.; Zhu, X.; Hou, Y. Detection and Monitoring of Oil Spills Using Moderate/High-Resolution Remote Sensing Images. Arch. Environ. Contam. Toxicol. 2017, 73, 154–169. [Google Scholar] [CrossRef]
- Guo, Y.; Zhang, H.Z. Oil spill detection using synthetic aperture radar images and feature selection in shape space. Int. J. Appl. Earth Obs. Geoinf. 2014, 30, 146–157. [Google Scholar] [CrossRef]
- Singha, S.; Bellerby, T.J.; Trieschmann, O. Satellite Oil Spill Detection Using Artificial Neural Networks. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2013, 6, 2355–2363. [Google Scholar] [CrossRef]
- LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436–444. [Google Scholar] [CrossRef]
- Guo, H.; Wu, D.; An, J. Discrimination of Oil Slicks and Lookalikes in Polarimetric SAR Images Using CNN. Sensors 2017, 17, 1837. [Google Scholar] [CrossRef]
- Badrinarayanan, V.; Kendall, A.; Cipolla, R. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 2481–2495. [Google Scholar] [CrossRef]
- Chen, G.; Li, Y.; Sun, G.; Zhang, Y. Polarimetric SAR oil spill detection based on deep networks. In Proceedings of the 2017 IEEE International Conference on Imaging Systems and Techniques (IST), Beijing, China, 18–20 October 2017; pp. 1–5. [Google Scholar] [CrossRef]
- Yu, X.; Zhang, H.; Luo, C.; Qi, H.; Ren, P. Oil Spill Segmentation via Adversarialf-Divergence Learning. IEEE Trans. Geosci. Remote Sens. 2018, 56, 4973–4988. [Google Scholar] [CrossRef]
- Li, Y.; Yang, X.; Ye, Y.; Cui, L.; Jia, B.; Jiang, Z.; Wang, S. Detection of Oil Spill Through Fully Convolutional Network. In Geo-Spatial Knowledge and Intelligence; Yuan, H., Geng, J., Liu, C., Bian, F., Surapunt, T., Eds.; Springer: Singapore, 2018; pp. 353–362. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
- Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going deeper with convolutions. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7–12 June 2015; pp. 1–9. [Google Scholar] [CrossRef]
- Long, J.; Shelhamer, E.; Darrell, T. Fully Convolutional Networks for Semantic Segmentation ppt. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7–12 June 2015; pp. 3431–3440. [Google Scholar] [CrossRef]
- Gallego, A.J.; Gil, P.; Pertusa, A.; Fisher, R.B. Segmentation of Oil Spills on Side-Looking Airborne Radar Imagery with Autoencoders. Sensors 2018, 18, 797. [Google Scholar] [CrossRef]
- Shi, X.; Chen, Z.; Wang, H.; Yeung, D.Y.; Wong, W.k.; Woo, W.c. Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting. In Proceedings of the 28th International Conference on Neural Information Processing Systems—Volume 1; MIT Press: Cambridge, MA, USA, 2015; NIPS’15; pp. 802–810. [Google Scholar]
- Hinton, G.E.; Zemel, R.S. Autoencoders, Minimum Description Length and Helmholtz Free Energy. In Proceedings of the Advances in Neural Information Processing Systems, Denver, CO, USA, 28 November–1 December 1994; pp. 3–10. [Google Scholar]
- Baldi, P. Autoencoders, Unsupervised Learning, and Deep Architectures. JMLR W&CP 2012, 27, 37–49. [Google Scholar]
- Bengio, Y.; Yao, L.; Alain, G.; Vincent, P. Generalized denoising auto-encoders as generative models. In Proceedings of the 26th International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA, 5–10 December 2013; pp. 899–907. [Google Scholar]
- Vincent, P.; Larochelle, H.; Bengio, Y.; Manzagol, P.A. Extracting and Composing Robust Features with Denoising Autoencoders. In Proceedings of the 25th International Conference on Machine Learning; ACM: New York, NY, USA, 2008; pp. 1096–1103. [Google Scholar] [CrossRef]
- Vincent, P.; Larochelle, H.; Lajoie, I.; Bengio, Y.; Manzagol, P.A. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 2010, 11, 3371–3408. [Google Scholar]
- Ioffe, S.; Szegedy, C. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. JMLR W&CP 2015, 37, 448–456. [Google Scholar]
- Glorot, X.; Bordes, A.; Bengio, Y. Deep Sparse Rectifier Neural Networks. JMLR W&CP 2011, 15, 315–323. [Google Scholar]
- Bergstra, J.; Bengio, Y. Random Search for Hyper-Parameter Optimization. J. Mach. Learn. Res. 2012, 13, 281–305. [Google Scholar]
- Zheng, L.; Zhao, Y.; Wang, S.; Wang, J.; Tian, Q. Good Practice in CNN Feature Transfer. CoRR 2016, arXiv:abs/1604.00133. [Google Scholar]
- Cover, T.; Hart, P. Nearest neighbor pattern classification. IEEE Trans. Inf. Theory 1967, 13, 21–27. [Google Scholar] [CrossRef]
- Vapnik, V.N. Statistical Learning Theory, 1st ed.; Wiley: Hoboken, NJ, USA, 1998. [Google Scholar]
- Knerr, S.; Personnaz, L.; Dreyfus, G. Single-layer learning revisited: A stepwise procedure for building and training a neural network. In Neurocomputing; Soulié, F.F., Hérault, J., Eds.; Springer: Berlin/Heidelberg, Germany, 1990; pp. 41–50. [Google Scholar]
- Breiman, L. Random Forests. Mach. Learn. 2001, 45, 5–32. [Google Scholar] [CrossRef] [Green Version]
- Bottou, L. Large-scale machine learning with stochastic gradient descent. In Proceedings of COMPSTAT’2010; Springer: Berlin/Heidelberg, Germany, 2010; pp. 177–186. [Google Scholar]
- Zeiler, M.D. ADADELTA: An Adaptive Learning Rate Method. CoRR 2012, arXiv:abs/1212.5701. [Google Scholar]
- Kohavi, R. A Study of Cross-validation and Bootstrap for Accuracy Estimation and Model Selection. In Proceedings IJCAI; Morgan Kaufmann Publishers Inc.: San Francisco, CA, USA, 1995; Volume 2, pp. 1137–1143. [Google Scholar]
- Oprea, S.; Gil, P.; Mira Martínez, D.; Alacid Soto, B. Candidate Oil Spill Detection in SLAR Data—A Recurrent Neural Network-based Approach. In Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods—Volume 1: ICPRAM; INSTICC, SciTePress: Setúbal, Portugal, 2017; pp. 372–377. [Google Scholar] [CrossRef]
- Schuster, M.; Paliwal, K. Bidirectional Recurrent Neural Networks. Trans. Sig. Proc. 1997, 45, 2673–2681. [Google Scholar] [CrossRef]
- Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Proceedings of the Medical Image Computing and Computer-Assisted Intervention (MICCAI), Munich, Germany, 5–9 October 2015. [Google Scholar]
- Simonyan, K.; Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. CoRR 2014, arXiv:abs/1409.1556. [Google Scholar]
- Chen, L.C.; Papandreou, G.; Schroff, F.; Adam, H. Rethinking Atrous Convolution for Semantic Image Segmentation. CoRR 2017, arXiv:abs/1706.05587. [Google Scholar]
- Demsar, J. Statistical Comparisons of Classifiers over Multiple Data Sets. J. Mach. Learn. Res. 2006, 7, 1–30. [Google Scholar]
Class | #Instances | #Scanlines/Class | % of Pixels () | Avg. BB in px. () | ||
---|---|---|---|---|---|---|
Central noise | 51 | 24,582 | 6.06 | |||
Maneuvers | 93 | 2091 | 8.18 | |||
Ship | 233 | 796 | 0.01 | |||
Spill | 380 | 3732 | 0.22 | |||
Lookalike | 3452 | 3351 | 0.38 | |||
Coast | 493 | 4798 | 6.69 | |||
Water | 51 | 22,635 | 79.89 |
Class | Input Size (px) height × width | Sequence Length (s) | # Encoder– Decoder Layers | # Filters (f) | Kernel Size (k) |
---|---|---|---|---|---|
Central noise | 1 × 512 | 14 | 16 | 1 × 7 | |
Maneuvers | 1 × 128 | 20 | 128 | 1 × 7 | |
Ships | 1 × 1160 | 14 | 128 | 1 × 5 | |
Spills | 1 × 512 | 25 | 128 | 1 × 5 | |
Lookalikes | 1 × 512 | 15 | 64 | 1 × 7 | |
Coast | 1 × 512 | 15 | 64 | 1 × 7 | |
Water | 1 × 512 | 12 | 128 | 1 × 7 |
# Filters/Kernel Size | |||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
16 | 32 | 64 | 128 | ||||||||||||||||
Class | # Layers | 3 | 5 | 7 | 3 | 5 | 7 | 3 | 5 | 7 | 3 | 5 | 7 | Avg. | |||||
Central | 2 | 82.0 | 83.7 | 86.0 | 82.3 | 85.4 | 86.0 | 81.9 | 85.6 | 86.2 | 83.8 | 85.4 | 86.0 | 84.5 | |||||
4 | 81.9 | 85.5 | 86.6 | 81.0 | 85.7 | 85.1 | 81.9 | 84.6 | 86.2 | 82.8 | 84.0 | 85.2 | 84.2 | ||||||
6 | 79.2 | 80.7 | 81.0 | 78.3 | 80.0 | 80.5 | 75.7 | 83.0 | 83.4 | 76.6 | 81.4 | 82.5 | 80.2 | ||||||
Avg. kernel | 81.0 | 83.3 | 84.5 | 80.5 | 83.7 | 83.9 | 79.8 | 84.4 | 85.3 | 81.0 | 83.6 | 84.6 | |||||||
Avg. filters | 83.0 | 82.7 | 83.2 | 83.1 | |||||||||||||||
Maneuvers | 2 | 47.9 | 48.3 | 58.8 | 49.1 | 54.4 | 59.0 | 52.5 | 53.4 | 58.9 | 50.0 | 55.3 | 54.8 | 53.5 | |||||
4 | 50.5 | 55.0 | 60.7 | 50.4 | 52.8 | 57.1 | 50.3 | 57.6 | 58.3 | 49.9 | 56.3 | 59.0 | 54.8 | ||||||
6 | 52.6 | 58.8 | 64.9 | 52.9 | 54.6 | 62.5 | 53.0 | 65.2 | 65.4 | 53.5 | 65.4 | 65.4 | 59.5 | ||||||
Avg. kernel | 50.3 | 54.0 | 61.5 | 50.8 | 53.9 | 59.6 | 52.0 | 58.8 | 60.9 | 51.1 | 59.0 | 59.7 | |||||||
Avg. filters | 55.3 | 54.8 | 57.2 | 56.6 | |||||||||||||||
Ships | 2 | 45.2 | 49.7 | 42.4 | 42.2 | 47.2 | 48.5 | 42.7 | 47.8 | 49.5 | 46.4 | 48.4 | 47.4 | 46.4 | |||||
4 | 42.8 | 45.2 | 45.5 | 42.4 | 44.7 | 48.0 | 47.1 | 48.7 | 51.0 | 47.1 | 47.2 | 48.1 | 46.5 | ||||||
6 | 46.2 | 49.8 | 49.5 | 51.2 | 53.8 | 54.1 | 49.2 | 55.9 | 54.9 | 53.7 | 57.8 | 56.1 | 52.7 | ||||||
Avg. kernel | 44.7 | 48.2 | 45.8 | 45.3 | 48.6 | 50.2 | 46.3 | 50.8 | 51.8 | 49.0 | 51.1 | 50.5 | |||||||
Avg. filters | 46.3 | 48.0 | 49.7 | 50.2 | |||||||||||||||
Spills | 2 | 17.4 | 18.4 | 17.9 | 18.1 | 18.6 | 18.0 | 18.9 | 19.6 | 20.1 | 21.6 | 23.7 | 23.0 | 19.6 | |||||
4 | 33.4 | 34.0 | 33.7 | 32.8 | 33.8 | 30.8 | 30.6 | 34.6 | 33.5 | 34.2 | 35.9 | 35.1 | 33.5 | ||||||
6 | 49.1 | 50.9 | 51.6 | 50.9 | 51.1 | 50.0 | 50.6 | 51.7 | 50.9 | 51.5 | 52.6 | 52.1 | 51.1 | ||||||
Avg. kernel | 33.3 | 34.5 | 34.4 | 33.9 | 34.5 | 32.9 | 33.4 | 35.3 | 34.8 | 35.7 | 37.4 | 36.8 | |||||||
Avg. filters | 34.1 | 33.8 | 34.5 | 36.6 | |||||||||||||||
Lookalikes | 2 | 21.2 | 21.7 | 21.8 | 21.8 | 21.9 | 20.8 | 22.0 | 22.0 | 21.4 | 23.0 | 19.8 | 19.3 | 21.4 | |||||
4 | 23.2 | 23.5 | 23.2 | 23.4 | 23.1 | 23.2 | 20.8 | 20.0 | 24.1 | 18.1 | 19.5 | 20.1 | 21.9 | ||||||
6 | 23.8 | 23.1 | 23.8 | 23.5 | 23.1 | 23.2 | 23.7 | 23.8 | 24.5 | 23.0 | 23.2 | 23.2 | 23.5 | ||||||
Avg. kernel | 22.7 | 22.7 | 22.9 | 22.9 | 22.7 | 22.4 | 22.1 | 21.9 | 23.3 | 21.4 | 20.8 | 20.9 | |||||||
Avg. filters | 22.8 | 22.7 | 22.5 | 21.0 | |||||||||||||||
Coast | 2 | 71.4 | 76.6 | 79.8 | 71.9 | 77.7 | 79.6 | 72.5 | 79.8 | 80.1 | 74.2 | 78.9 | 82.5 | 77.1 | |||||
4 | 72.3 | 77.6 | 79.8 | 72.7 | 75.6 | 80.5 | 72.6 | 78.0 | 81.6 | 73.4 | 77.5 | 79.7 | 76.8 | ||||||
6 | 75.3 | 75.9 | 80.1 | 74.8 | 78.1 | 79.7 | 73.2 | 77.9 | 82.9 | 75.7 | 80.0 | 82.1 | 78.0 | ||||||
Avg. kernel | 73.0 | 76.7 | 79.9 | 73.1 | 77.1 | 79.9 | 72.8 | 78.6 | 81.5 | 74.4 | 78.8 | 81.5 | |||||||
Avg. filters | 76.5 | 76.7 | 77.6 | 78.2 | |||||||||||||||
Water | 2 | 94.1 | 93.7 | 94.4 | 94.0 | 94.4 | 94.2 | 94.2 | 94.4 | 94.7 | 93.9 | 94.3 | 94.7 | 94.2 | |||||
4 | 94.0 | 94.3 | 94.6 | 94.1 | 94.5 | 94.5 | 94.1 | 94.5 | 94.9 | 94.1 | 94.4 | 94.7 | 94.4 | ||||||
6 | 94.1 | 94.5 | 94.2 | 93.8 | 93.9 | 94.7 | 93.8 | 94.8 | 94.7 | 94.2 | 94.3 | 95.0 | 94.3 | ||||||
Avg. kernel | 94.1 | 94.2 | 94.4 | 94.0 | 94.2 | 94.5 | 94.0 | 94.6 | 94.8 | 94.0 | 94.3 | 94.8 | |||||||
Avg. filters | 94.2 | 94.2 | 94.5 | 94.4 |
Class | CMSAE | CMSAE+kNN | CMSAE+SVM | CMSAE+RaF |
---|---|---|---|---|
Central noise | 86.60 | 87.11 | 87.92 | 87.32 |
Maneuvers | 65.44 | 65.37 | 65.45 | 65.43 |
Ships | 57.79 | 57.53 | 58.03 | 58.17 |
Spills | 52.58 | 53.12 | 54.36 | 53.91 |
Lookalikes | 24.49 | 26.37 | 26.54 | 26.61 |
Coast | 82.90 | 83.45 | 83.92 | 83.51 |
Water | 94.97 | 95.12 | 95.49 | 95.35 |
Average | 66.40 | 66.87 | 67.39 | 67.19 |
Class | BiRNN | TSCNN | U-Net | SegNet | DeepLabv3 | SelAE | Our Method |
---|---|---|---|---|---|---|---|
Central noise | 75.37 ± 2.5 | – | 85.64 ± 1.5 | 84.19 ± 0.9 | 85.95 ± 1.1 | 87.84 ± 0.7 | 87.92 ± 0.7 |
Maneuvers | 50.32 ± 2.2 | – | 62.47 ± 2.1 | 60.20 ± 1.0 | 65.71 ± 1.3 | 65.55 ± 0.9 | 65.45 ± 0.8 |
Ships | 26.50 ± 3.1 | 50.35 ± 1.1 | 27.02 ± 2.9 | 2.53 ± 1.4 | 46.56 ± 1.0 | 57.29 ± 0.9 | 58.03 ± 0.8 |
Spills | 32.38 ± 3.0 | 53.86 ± 1.0 | 33.39 ± 2.6 | 12.08 ± 1.3 | 49.33 ± 1.8 | 54.52 ± 0.9 | 54.36 ± 0.9 |
Lookalikes | 14.90 ± 3.3 | – | 15.13 ± 3.1 | 1.67 ± 1.8 | 13.32 ± 2.3 | 17.74 ± 1.3 | 26.54 ± 1.0 |
Coast | 65.41 ± 2.8 | 72.99 ± 1.3 | 68.25 ± 1.6 | 68.95 ± 1.2 | 84.57 ± 1.2 | 80.20 ± 1.0 | 83.92 ± 1.0 |
Water | 90.39 ± 1.3 | – | 94.24 ± 1.2 | 94.79 ± 1.0 | 94.84 ± 1.1 | 95.37 ± 0.9 | 95.49 ± 0.7 |
Average | 50.75 ± 2.5 | – | 55.16 ± 2.1 | 46.35 ± 1.2 | 62.90 ± 1.4 | 65.50 ± 1.0 | 67.39 ± 0.9 |
Class | BiRNN | TSCNN | U-Net | SegNet | DeepLabv3 | SelAE | Our Method |
---|---|---|---|---|---|---|---|
Central noise | 100.00 | – | 100.00 | 100.00 | 100.00 | 100.00 | 100.00 |
Maneuvers | 96.77 | – | 100.00 | 100.00 | 100.00 | 100.00 | 100.00 |
Ships | 17.17 | 75.11 | 19.37 | 2.15 | 65.04 | 87.98 | 90.13 |
Spills | 26.32 | 89.47 | 27.45 | 13.16 | 84.29 | 93.42 | 92.11 |
Lookalikes | 43.45 | – | 45.63 | 1.30 | 41.12 | 56.49 | 59.39 |
Coast | 76.06 | 86.21 | 80.21 | 81.14 | 95.67 | 93.31 | 94.32 |
Water | 100.00 | – | 100.00 | 100.00 | 100.00 | 100.00 | 100.00 |
Average | 65.68 | – | 67.52 | 56.82 | 83.73 | 90.17 | 90.85 |
Method | Lag Time | Runtime | Total Time |
---|---|---|---|
SegNet | 24 | 0.2929 | 24.2929 |
DeepLabv3 | 24 | 0.1871 | 24.1871 |
U-Net | 24 | 0.1044 | 24.1044 |
SelAE | 24 | 0.0007 | 24.0007 |
TSCNN | 19 | 0.0078 | 19.0078 |
BiRNN | 2.28 | 0.0006 | 2.2806 |
Our approach | 0.76 | 0.5259 | 1.2859 |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Gallego, A.-J.; Gil, P.; Pertusa, A.; Fisher, R.B. Semantic Segmentation of SLAR Imagery with Convolutional LSTM Selectional AutoEncoders. Remote Sens. 2019, 11, 1402. https://doi.org/10.3390/rs11121402
Gallego A-J, Gil P, Pertusa A, Fisher RB. Semantic Segmentation of SLAR Imagery with Convolutional LSTM Selectional AutoEncoders. Remote Sensing. 2019; 11(12):1402. https://doi.org/10.3390/rs11121402
Chicago/Turabian StyleGallego, Antonio-Javier, Pablo Gil, Antonio Pertusa, and Robert B. Fisher. 2019. "Semantic Segmentation of SLAR Imagery with Convolutional LSTM Selectional AutoEncoders" Remote Sensing 11, no. 12: 1402. https://doi.org/10.3390/rs11121402