[go: up one dir, main page]
Next Article in Journal
Airborne Validation Experiment of 1.57-μm Double-Pulse IPDA LIDAR for Atmospheric Carbon Dioxide Measurement
Next Article in Special Issue
Flood Detection Using Multi-Modal and Multi-Temporal Images: A Comparative Study
Previous Article in Journal
Hyperspectral Inversion of Phragmites Communis Carbon, Nitrogen, and Phosphorus Stoichiometry Using Three Models
Previous Article in Special Issue
A Coarse-to-Fine Deep Learning Based Land Use Change Detection Method for High-Resolution Remote Sensing Images
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Remote Sensing
Volume 12
Issue 12
10.3390/rs12122000
remotesensing-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Deep Learning for Land Cover Classification Using Only a Few Bands

by
Chiman Kwan
1,*,
Bulent Ayhan
1,
Bence Budavari
1,
Yan Lu
2,
Daniel Perez
2,
Jiang Li
2,
Sergio Bernabe
3 and
Antonio Plaza
4
1
Applied Research LLC, Rockville, MD 20850, USA
2
Department of Electrical and Computer Engineering, Old Dominion University, Norfolk, VA 23529, USA
3
Department of Computer Architecture and Automation, Complutense University of Madrid, 28040 Madrid, Spain
4
Department of Technology of Computers and Communications, University of Extremadura, 10003 Cáceres, Spain
*
Author to whom correspondence should be addressed.
Remote Sens. 2020, 12(12), 2000; https://doi.org/10.3390/rs12122000
Submission received: 26 May 2020 / Revised: 14 June 2020 / Accepted: 18 June 2020 / Published: 22 June 2020
(This article belongs to the Special Issue Recent Advances in Land Cover Classification and Change Detection in 2D and 3D)
Browse Figures
Graphical abstract
"> Figure 1
<p>Our customized conventional neural network (CNN) model.</p> "> Figure 2
<p>CNN-3D [<a href="#B50-remotesensing-12-02000" class="html-bibr">50</a>].</p> "> Figure 3
<p>Color image for the University of Houston area with ground truth land cover annotations of test data.</p> "> Figure 4
<p>Performance metrics for the investigated band combinations with our customized CNN model.</p> "> Figure 5
<p>Classification accuracy for each land cover type with our customized CNN model using the investigated band combinations.</p> "> Figure 6
<p>Estimated land cover maps by our customized CNN method for the whole image.</p> "> Figure 7
<p>Performance metrics for the investigated band combinations with the CNN-3D model.</p> "> Figure 8
<p>Classification accuracy for each land cover type with CNN-3D model using the investigated band combinations.</p> "> Figure 9
<p>CNN-3D method estimated land cover maps for the whole image.</p> "> Figure 9 Cont.
<p>CNN-3D method estimated land cover maps for the whole image.</p> ">
Versions Notes

Abstract

:
There is an emerging interest in using hyperspectral data for land cover classification. The motivation behind using hyperspectral data is the notion that increasing the number of narrowband spectral channels would provide richer spectral information and thus help improve the land cover classification performance. Although hyperspectral data with hundreds of channels provide detailed spectral signatures, the curse of dimensionality might lead to degradation in the land cover classification performance. Moreover, in some practical applications, hyperspectral data may not be available due to cost, data storage, or bandwidth issues, and RGB and near infrared (NIR) could be the only image bands available for land cover classification. Light detection and ranging (LiDAR) data is another type of data to assist land cover classification especially if the land covers of interest have different heights. In this paper, we examined the performance of two Convolutional Neural Network (CNN)-based deep learning algorithms for land cover classification using only four bands (RGB+NIR) and five bands (RGB+NIR+LiDAR), where these limited number of image bands were augmented using Extended Multi-attribute Profiles (EMAP). The deep learning algorithms were applied to a well-known dataset used in the 2013 IEEE Geoscience and Remote Sensing Society (GRSS) Data Fusion Contest. With EMAP augmentation, the two deep learning algorithms were observed to achieve better land cover classification performance using only four bands as compared to that using all 144 hyperspectral bands.

Graphical Abstract">

Graphical Abstract

1. Introduction

Hyperspectral data have been used for chemical agent detection and classification [1,2], small target detection [3,4], fire damage assessment [5,6], anomaly detection [7,8,9,10,11,12,13], border monitoring [14], change detection [15,16,17,18], and mineral map abundance estimation on Mars [19,20]. Land cover classification is another application area where hyperspectral data are used extensively [21]. To improve land cover classification, the use of light detection and ranging (LiDAR) data and the fusion of LiDAR and hyperspectral data have been considered by several works [22,23,24,25,26,27]. Because some land covers differ among themselves with respect to their height, height could be a valuable piece of information.
There is also an increasing interest in adapting deep learning methods for land cover classification after several breakthroughs have been reported in a variety of computer vision tasks such as image classification, object detection and tracking, and semantic segmentation. The authors of [28] described an unsupervised feature extraction framework, with the name “patch-to-patch convolutional neural network (PToP CNN)”, which was used for land cover classification using hyperspectral and LiDAR data. The authors of [29] introduced a spectral-spatial attention network for image classification using hyperspectral data. In this network, while the recurrent neural network (RNN) part with attention can learn inner spectral correlations within a continuous spectrum, the CNN part is for the saliency features and aims to extract spatial relevance between neighboring pixels in the spatial dimension. The authors of [30] used convolutional networks for multispectral image classification. The authors of [31] provided a comprehensive review on land cover classification and object detection approaches using high-resolution imagery. The performances of deep learning models were evaluated against traditional approaches, and the authors concluded that the deep-learning-based methods provided an end-to-end solution and showed better performance than the traditional pixel-based methods by utilizing both spatial and spectral information, whereas traditional pixel-based methods resulted in salt-and-pepper-type noisy land cover map estimates. A number of other works have also shown that semantic segmentation classification with deep learning methods are quite promising in land cover classification [32,33,34,35].
Researchers also utilized spatial information in hyperspectral data via the use of Extended Multi-attribute Profiles (EMAPs) [36]. EMAPs, which are based on morphological attribute filters, generate augmented bands from hyperspectral image bands and have found a considerable interest in land cover classification [37,38,39,40,41,42]. In the event that limited image bands are available, EMAP augmented bands from these limited image bands could be beneficial. A recent application of using EMAP-augmented bands for soil detection due to illegal tunnel excavation can be found in [43]. Because the number of image bands is too high in hyperspectral data, the number of EMAP augmented bands becomes too high as well. For this reason, works utilizing EMAP have generally applied dimension reduction techniques to extract the most useful information from the EMAP augmented bands [38]. The fusion of hyperspectral and LiDAR data was proposed by the authors [44] and was applied to the 2013 IEEE Geoscience and Remote Sensing Society (GRSS) Data Fusion Contest dataset. All 144 hyperspectral bands were used in EMAP augmentation and the overall accuracy was found to be 90.65%.
Even though using hyperspectral data provides rich and distinctive information that can point subtle differences for land cover classification that would otherwise be impossible to identify using RGB image bands only or even using multispectral image bands, there is a price that comes with it. The high dimensionality in hyperspectral data due to the presence of hundreds of bands could result in the curse of dimensionality phenomena [37,45], especially if there is limited training data when training models with the classifiers for land cover classification. In this situation, the available training data could look sparse relative to the large space volume formed by the high dimensional data. This could potentially result in lower classification performances than anticipated when all hyperspectral bands are used. To mitigate this issue, several works have applied dimension reduction techniques to hyperspectral data [45,46], as well as to the EMAP augmented bands [36,38]. Moreover, hyperspectral sensors are expensive and demand large data storage and computational resources and times. Considering that some applications, such as pest damage assessment by farmers, have limited budgets and the use of hyperspectral sensors could be costly and off-limits for them, using fewer image bands like RGB and near infrared (NIR) for land cover classification via advanced algorithms that can utilize both spatial and spectral information from these fewer image bands would be invaluable.
In this paper, we summarized the evaluation of land cover classification performance of two CNN-based deep learning algorithms when only a few bands, namely RGB+NIR bands and RGB+NIR+LiDAR bands, were available, and when EMAP was applied to those limited bands to generate the augmented bands. This is different from the EMAP application described by the authors of [36,37,38,39,40,41,42,43] since, in those works, all hyperspectral image bands were augmented followed by dimension reduction of the features extracted from the augmented bands. Extensive performance comparisons were conducted for the cases when all hyperspectral bands were used for land cover classification in contrast to EMAP-augmented bands only and RGB+NIR bands alone. Additionally, the impact of adding LiDAR on the classification performance of the three cases was examined. These three cases are: (a) Limited number of bands (RGB+NIR) vs. (RGB+NIR+LiDAR), (b) EMAP-augmentation of RGB+NIR vs. EMAP augmentation of RGB+NIR+LiDAR, and (c) hyperspectral bands vs. hyperspectral bands+LiDAR. We used the 2013 IEEE GRSS Data Fusion Contest dataset [44] in the investigations. Two conventional classification algorithms, Joint Sparse Representation (JSR) [14] and Support Vector Machine (SVM) [47], were also applied to the same dataset for the EMAP-augmented cases. The contributions of this paper are as follows:
  • We provided a comprehensive performance evaluation of two CNN-based deep learning methods for land cover classification when a limited number of bands (RGB+NIR and RGB+NIR+LiDAR) were used for augmentation with EMAP. The evaluation also included detailed classification accuracy comparisons when limited bands were used only and when all hyperspectral bands were used without EMAP.
  • We showed that, with deep learning methods using fewer number of bands and utilizing EMAP-based augmentation, it is quite possible to arrive at highly decent accuracies for land cover classification. This eliminates the need for hundreds of hyperspectral bands and reduces it to four bands.
  • We demonstrated that, even though adding LiDAR band to RGB+NIR bands for EMAP augmentation made a significant impact with conventional classifiers, JSR and SVM, for land cover classification, no considerable impact was observed with the deep learning methods since their classification performances were already good when using the EMAP-augmented RGB+NIR bands.
The rest of this paper is organized as follows. In Section 2, we review the two deep learning methods, EMAP, and the 2013 IEEE GRSS Data Fusion Contest Data. In Section 3, we summarize our findings. Finally, we conclude our paper with a few remarks.

2. Methods

2.1. Our Customized CNN Method

For this CNN-based method, we used the same structure that was used in our previous work [39]. Only the filter size in the first convolution layer was changed accordingly to be consistent with the input patch sizes. This CNN model has four convolutional layers, with various filter sizes and a fully connected layer with 100 hidden units as shown in Figure 1. When we designed the network, we tried different configurations for the number of layers and the size of each layer and selected the one that provided the best results. We did this for all the layers (convolutional and FC layers). The choice of “100 hidden units” was the outcome of our design studies. This model was also used for soil detection due to illegal tunnel excavation, as described in [48]. Each convolutional layer utilizes the Rectified Linear Unit (ReLu) as the activation function. The last fully connected layer uses the SoftMax function for classification. We added dropout layer for each convolutional layer with a dropout rate of 0.1 to mitigate overfitting [49]. It should be noted that the network size changes depending on the input size. In Figure 1, the network should be the same for 5 × 5 as for 7 × 7. The only difference would be that the first layer is deleted. For 3 × 3, the first two layers are deleted. The number of bands (N) in the input image can be any integer number.

2.2. CNN-3D

CNN-3D is a CNN-based method [50] with two convolutional layers and two fully connected layers. The two convolutional layers have pooling units and all of its four layers have ReLu units. Figure 2 shows the architecture of this second CNN-based method that is used for land cover classification. This network was used in hyperspectral pixel classification before. The details have been described by the authors of [50]. Due to the use of pooling layers, the patch size cannot be too small. Otherwise, the later layers will not have any impacts.

2.3. EMAP

For a grayscale image f and a sequence of threshold levels { T h 1 ,   T h 2 ,   …   T h n } , the attribute profile (AP) of f was obtained by applying a sequence of thinning and thickening attribute transformations to every pixel in f as follows:
A P ( f ) = { ϕ 1 ( f ) ,   ϕ 2 ( f ) ,   …   ϕ n ( f ) , f ,   γ 1 ( f ) ,   γ 2 ( f ) ,   …   γ n ( f ) }
where ϕ i and γ i   ( i = 1 ,   2 ,   … n ) are the thickening and thinning operators at threshold T h i , respectively. The EMAP of f was then acquired by stacking two or more APs using any feature reduction technique on multispectral/hyperspectral data, such as purely geometric attributes (e.g., area, length of the perimeter, image moments, shape factors), or textural attributes (e.g., range, standard deviation, entropy) [36,39,40,41] as shown in (2). Further technical details about EMAP can be found in [36,39,40,41].
E M A P ( f ) = { A P 1 ( f ) ,     A P 2 ( f )   …   A P m ( f ) }
When generating the EMAP-augmented bands in the conducted investigations, no feature dimension reduction process was applied to the hyperspectral data (such as Principal Component Analysis) because the hyperspectral bands of interest were only RGB+NIR instead of all the hyperspectral bands available. EMAP was directly applied to these limited number of bands (RGB+NIR) with the selected attribute profiles, which were the ‘area (a)’ and ‘length of the diagonal of the bounding box’ (d) [43]. The lambda parameters in EMAP for both ‘area’ and ‘length of the diagonal of the bounding box’ attributes were arbitrarily chosen. For the ‘area’ attribute, which is related to modeling spatial information, the higher number of extrema (lambda parameters) contains more detail, whereas the smaller number smooths out the input data. In this work, smoothing was favored and the lambda parameters for the area attribute of EMAP, which is a sequence of thresholds used by the morphological attribute filters, were set to 10 and 15, respectively. For the ‘length of the diagonal of the bounding box’ attribute, which is related to the shape of the regions, the lambda parameters were set to 50, 100, and 500. With these two attributes and their parameter settings, 10 augmented bands were generated for a single band image. The total number of image bands becomes 11 when including the original single band image, Because the resultant EMAP augmented bands were few in number, no feature dimension reduction process was applied to the augmented EMAP bands and all of the EMAP augmented bands were used with the deep learning classifiers.

2.4. Dataset

The hyperspectral image dataset for the University of Houston area and the corresponding LiDAR data were used in this paper. This dataset, together with its ground truth land cover maps, were obtained from the IEEE GRSS Data Fusion package [44] and were used in the 2013 IEEE Geoscience and Remote Sensing Society Data Fusion Contest. The hyperspectral data in this dataset contain 144 bands ranging in wavelength values from 380 mm to 1050 nm with a spectral width of 4.65 nm and a spatial resolution of 2.5 m per pixel. The LiDAR data contain the height information and have a resolution of 2.5 m per pixel. Table 1 displays the number of training and test data pixels per land cover class provided in this dataset. There were 15 land cover classes which were identified and named by the contest. These classes and their corresponding training and test datasets (each pixel in the training and test dataset had its own land cover class label) were fixed. The training data set included 2832 pixels and the test dataset includes the remaining 12,197 labeled pixels. Figure 3a shows the color image, and Figure 3b,c show the color image with ground truth land cover annotations for both the training and test dataset. The brightness and contrast of the color image in Figure 3 was adjusted for better visual assessment.
In the hyperspectral image, the investigations with the four bands (RGB+NIR) correspond to the narrow hyperspectral bands, which were directly retrieved from hyperspectral data. These bands are Red (R), Green (G), Blue (Blue), and NIR, and the band numbers in the hyperspectral data for RGB and NIR bands are (R), #30 (G), #22 (B), and #103 (NIR).
We would like to mention that there is another dataset known as Trento data that contains both hyperspectral and LiDAR images. However, the Trento dataset is no longer publicly available to researchers.

2.5. Performance Metrics

For classification performance evaluation of the deep learning methods with EMAP-augmented bands and all other band combinations, we used overall accuracy, OA, which is the ratio of the sum of correctly classified pixels from all classes to the total number of pixels in the test data. In addition to OA, we also generated average accuracy, AA, value which corresponds to the average of individual class accuracies, which is also known as ‘balanced accuracy’. The last performance metric was the well-known Kappa (K) coefficient [51].

3. Results

3.1. Our Customized CNN Results

Table 2 shows our customized CNN model [48] classification results for the test dataset with six different sets of image bands. These six different sets of image bands are: (a) RGB+NIR (four bands), (b) RGB+NIR+LiDAR (5 bands), (c) 44 EMAP-augmented bands (RGB+NIR bands augmented to 44 bands with EMAP, 11 × 4 = 44), (d) 55 EMAP-augmented bands (RGB+NIR+LiDAR bands augmented to 55 bands with EMAP), (e) 144 hyperspectral bands, and (f) 144 hyperspectral bands+LiDAR (145 bands). The results in Table 2 consist of two parts. The first part corresponds to the resultant performance metrics (OA, AA, and Kappa), and the second part corresponds to the correct classification accuracy for each land cover type using the investigated band combinations with our customized CNN model. Figure 4 and Figure 5 show these results using bar chart type of plots. When applying the customized CNN model, two different patch sizes were investigated, 3 × 3 and 5 × 5, respectively. In Figure 4 and Figure 5, the value in parenthesis after the band combination type corresponds to patch size. As an example, (p3) indicates a patch size of 3 × 3.
The highest classification accuracy was found to be with the 44 EMAP-augmented bands case using the 3 × 3 patch size, followed by the 55 EMAP-augmented bands case with 3 × 3 patch size. Among the 15 different land covers, two land covers, synthetic grass and tennis court, had perfect correct classification followed by close-to-perfect classification accuracy for soil land cover. The lowest classification accuracy was found to be for highway class. Figure 6 shows the estimated land cover maps for the whole image with our customized CNN model. The estimated land cover map with the highest overall accuracy for each image band set is shown in Figure 6.

3.2. CNN-3D (C3D) Results

Table 3 shows the classification results of the CNN-3D method [49] for the test dataset with six different sets of image bands. With this method, two different patch sizes, 13 × 13 and 17 × 17, were investigated for three of the five sets of bands. As mentioned earlier, the patch sizes cannot be too small due to the pooling layers in C3D. For instance, if the patch size is 7 × 7, the patch size will drop to single digits after the first two layers, hence the network cannot function as desired. For the two remaining band sets, which were 144 hyperspectral bands and the 144 hyperspectral bands+LiDAR, the GPU memory was not sufficient for CNN-3D to run. Consequently, only the patch size of 7 × 7 was considered for 144 hyperspectral bands and the 144 hyperspectral bands+LiDAR cases. Even though comparing the 7 × 7 patch size results of these two image band sets with the 13 × 13 and 17 × 17 patch size results of the other three image band sets would not be fair, we still included these results in Table 3 to provide some rough idea about the impact of adding LiDAR band to the hyperspectral bands. Figure 7 corresponds to the bar chart of the resultant performance metrics (OA, AA, and Kappa), and Figure 8 shows the bar chart plot of the correct classification accuracy for each land cover type using the investigated band combinations with CNN-3D model in Table 3.
With CNN-3D, the highest classification accuracy was obtained with the 55 EMAP-augmented bands and 44 EMAP-augmented bands cases using a patch size of 13 × 13. The 55 EMAP-augmented bands case was observed to have slightly higher accuracy than the 44 EMAP-augmented bands case. The classification results with all other image band sets had relatively lower values. For the 55 EMAP-augmented bands case, the top three close-to-perfect correct classifications were for the soil, synthetic grass, and tennis court land covers, whereas the lowest classification accuracy was for highway class. In our customized CNN model results, similar observations were made with respect to the land covers with best top three and the worst classification accuracies. Figure 9 shows the estimated land cover maps with this method for the whole image. For image band sets with more than one investigated patch window size, the one providing better classification performance is shown in Figure 9.

3.3. Performance Comparison of Deep Learning Methods

Table 4 shows the highest classification accuracy values obtained with our customized CNN model and CNN-3D method for each of the six image band sets. In five of these six cases, our customized CNN method was observed to perform better than CNN-3D method. The CNN-3D performed better than our customized CNN method only in the 55 EMAP-augmented bands case.
Considering the best classification results from any of the two deep learning methods in each image band set, one interesting observation from Table 5 is that using RGB+NIR bands (four bands) without LiDAR and using hyperspectral bands (144 bands) without the LiDAR resulted in slightly better classification accuracies than the same cases which included LiDAR. For the 44 EMAP-augmented bands case and 55 EMAP-augmented bands case, a similar observation was made when the average accuracy and Kappa metrics were taken into consideration instead of the overall accuracy metric, since these two metrics were found to be slightly higher in the 44 EMAP-augmented bands case in comparison to the 55 EMAP-augmented bands case. For the overall accuracy metric, they were also very close to each other in value. Considering the results with and without LiDAR in these three cases, it was observed that including LiDAR did not seem to have a considerable impact with the deep learning methods. This is somewhat not intuitive, since LiDAR was expected to bring valuable information with respect to height. This could be because several of the 15 land covers in this dataset are surface-level type land covers, meaning they have similar heights with the exception of trees, residential, and commercial land covers. When looking at the classification accuracies of these three land cover classes for the LiDAR-added cases to see whether LiDAR had any impact, mixed results were also seen. It was anticipated that no significant impact of LiDAR would be observed with the two deep learning methods because their classification performances were already good when using RGB+NIR bands.
By comparing the classification maps of Figure 6 and Figure 9, it can be seen that our customized CNN results look crisper due to the use of small patch sizes. In a nutshell, the two CNNs differed only in the pooling layers. In our investigations, we found that the pooling layers did not help the overall classification performance.

3.4. Comparison with Conventional Methods

The deep learning methods’ best classification performances for 44 EMAP-augmented bands and 55 EMAP-augmented bands cases were compared with two conventional methods, Joint Sparse Representation (JSR) [14] and Support Vector Machine (SVM) [47], using the same two sets of EMAP-augmented bands (44 and 55). JSR is more computationally demanding, as it exploits neighborhood pixels for joint land type classification. In JSR, a 3 × 3 or 5 × 5 patch of pixels is used in the S target matrix. The parameter s 0 in Equation (13) of [52] is the design parameter, which controls the number of sparse elements. Details of the mathematics have been described by the authors of [52]. We chose that parameter to be 10e-4. There were two SVM parameters, the penalty factor C and the radial basis function (RBF) kernel width, γ , which were chosen to be 10 and 0.1, respectively. In addition to these two conventional methods, classification results reported by two papers, which used this same dataset with hyperspectral and LiDAR bands augmented with EMAP, were also considered for comparison as well. Details of the JSR and SVM results have been described by the authors of [52].
The resultant overall accuracies of these methods are shown in Table 5. The two conventional methods were observed to perform extremely close in accuracy to the deep learning methods for the 55 EMAP-augmented bands case. For the 44 EMAP-augmented bands case, on the other hand, the deep learning methods’ performances were considerably better than the conventional methods. From Table 5, it can be noticed that adding the LiDAR band to the RGB+NIR bands for EMAP augmentation made a significant impact in overall classification accuracy with the conventional classifiers, JSR and SVM, for land cover classification. However, no considerable impact was observed when adding the LiDAR band to the the EMAP augmentation with the deep learning methods as discussed before.
In [44], the authors investigated three different combinations of hyperspectral bands for land cover classification. Among these, the investigation which used the hyperspectral bands, additional bands from LiDAR data, and the EMAP-augmented hyperspectral bands resulted in an overall accuracy values of 90.65%, whereas the overall accuracy when using only the hyperspectral bands and EMAP-augmented hyperspectral bands was 84.40%. In another paper [53], a graph-based approach was proposed to fuse HS and LiDAR data for land cover classification. In [53], the authors used the same dataset with three different band combinations. Their investigation with hyperspectral bands only using SVM classifier had an accuracy of 80.72%. Our results using only hyperspectral bands are in line with this result, since we reached an accuracy of 81.48% using our customized CNN method, which provided slightly higher accuracy than their result. A second investigation described by the authors of [53] used the morphological profiles of hyperspectral and LiDAR bands and reached an accuracy of 86.39%.
In our case, even though only four bands (RGB+NIR) and five bands (RGB+NIR+LiDAR) were used with EMAP for augmentation, decent classification accuracies of 87.92% (Our CNN) and 87.96% (CNN-3D) were reached, respectively. These overall accuracy values are higher than the investigations described by the authors of [44,53], which used all hyperspectral bands and additional LiDAR bands together with EMAP augmentation. This is quite impressive considering that we augmented only four bands with EMAP instead of using all hyperspectral bands and additional bands from LiDAR and still reached an overall accuracy of 87.92%. Our work shows that with deep learning methods using a fewer number of bands and utilizing EMAP augmentation, it is possible to arrive at very decent correct classification accuracies for land cover classification, eliminating the need of using hundreds of hyperspectral bands. This could ultimately reduce computation and data storage needs.
Table
Table 5. Comparison of the overall accuracies (%) of several methods.
Table 5. Comparison of the overall accuracies (%) of several methods.
ReferenceDataset AdoptedAlgorithm AdoptedOverall Accuracy
[52]44 EMAP (RGB+NIR)JSR80.77
[52]44 EMAP (RGB+NIR)SVM82.64
This paper44 EMAP (RGB+NIR)Our CNN87.92
This paper44 EMAP (RGB+NIR)CNN-3D87.64
[52]55 EMAP (RGB+NIR+LiDAR)JSR86.86
[52]55 EMAP (RGB+NIR+LiDAR)SVM86.00
This paper55 EMAP (RGB+NIR+LiDAR)Our CNN86.02
This paper55 EMAP (RGB+NIR+LiDAR)CNN-3D87.96
[44]Hyperspectral data; EMAP augmentation applied hyperspectral data (Xh + EMAP(Xh))MLRsub84.40
Hyperspectral data; Additional bands from LiDAR data (Xh + AP(XL))MLRsub87.91
Hyperspectral data; EMAP augmentation applied hyperspectral data; Additional bands from LiDAR data (Xh + AP(XL) + EMAP(Xh))MLRsub90.65
[53]Hyperspectral dataSVM80.72
Morphological Profile of hyperspectral and LiDAR data (MPSHSLi)SVM86.39
Here, we compared the computational costs of different methods. We used a desktop PC with i7 quadcore CPU and a GPU (NVIDIA GeForce Titan X Pascal 12GB GDDR5X). The JSR and SVM results only used CPU, and the CNN and C3D used GPU. Table 6 summarizes the computational times in minutes. It can be seen that SVM were much faster than JSR because JSR needed a lot more time in minimizing the sparseness constraint. The CNN and C3D times are similar and less than three minutes, which might be misleading. It should be noted that the deep learning methods all used GPU, hence the computational times are less than those conventional methods.

4. Conclusions

In this paper, we investigated the classification performance of two deep learning methods for land cover classification, where only four bands (RGB+NIR) and five bands (RGB+NIR+LiDAR) were used in terms of EMAP augmentation. In the performance evaluations, the RGB+NIR bands and hyperspectral bands were used without EMAP as well. The results showed that, using deep learning methods and EMAP augmentation, RGB+NIR (four bands) or RGB+NIR+LiDAR (five bands) can produce very good classification performance, and these results are better than those obtained by conventional classifiers such as SVM and JSR. We observed that, even though adding LiDAR band to RGB+NIR bands for augmentation made a significant classification performance impact with respect to conventional classifiers (JSR and SVM), no significant impact was observed with the deep learning methods, since their classification performances were already very good when using RGB+NIR bands. This work demonstrated that using RGB+NIR (four bands) or RGB+NIR+LiDAR (five bands) with EMAP augmentation is feasible for land cover classification, as the resultant accuracies are only a few percentage points lower than some of the best performing methods in the literature which used the same dataset and utilized all the available hyperspectral bands.
The fusion of different classifiers using decision level fusion [54] or pixel level fusion [55] could be a good future direction. Parallel implementation of JSR and SVM using parallel processing techniques could be another future direction.

Author Contributions

Conceptualization, C.K.; methodology, C.K., B.B., Y.L., D.P., J.L., S.B., A.P.; writing—original draft preparation, B.A. and C.K.; supervision, C.K.; project administration, C.K.; Writing—review & editing—B.A., C.K., S.B., A.P., J.L.; funding acquisition, C.K. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported by US Department of Energy under grant # DE-SC0019936. The views, opinions and/or findings expressed are those of the author and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S. Government.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Ayhan, B.; Kwan, C.; Jensen, J.O. Remote vapor detection and classification using hyperspectral images. In Proceedings of the SPIE 11010, Chemical, Biological, Radiological, Nuclear, and Explosives (CBRNE) Sensing XX, Baltimore, MD, USA, 14 May 2019; p. 110100U. [Google Scholar]
  2. Kwan, C.; Ayhan, B.; Chen, G.; Chang, C.; Wang, J.; Ji, B. A Novel Approach for Spectral Unmixing, Classification, and Concentration Estimation of Chemical and Biological Agents. IEEE Trans. Geosci. Remote Sens. 2006, 44, 409–419. [Google Scholar] [CrossRef]
  3. Harsanyi, J.C.; Chang, C.I. Hyperspectral image classification and dimensionality reduction: An orthogonal subspace projection approach. IEEE Trans. Geosci. Remote Sens. 1994, 32, 779–785. [Google Scholar] [CrossRef] [Green Version]
  4. Heinz, D.C.; Chang, C.I. Fully constrained least squares linear spectral mixture analysis method for material quantification in hyperspectral imagery. IEEE Geosci. Remote Sens. Soc. 2001, 39, 529–545. [Google Scholar] [CrossRef] [Green Version]
  5. Dao, M.; Kwan, C.; Ayhan, B.; Tran, T. Burn Scar Detection Using Cloudy MODIS Images via Low-rank and Sparsity-based Models. In Proceedings of the IEEE Global Conference on Signal and Information Processing (GlobalSIP), Washington, DC, USA, 7–9 December 2016; pp. 177–181. [Google Scholar]
  6. Veraverbeke, S.; Dennison, P.; Gitas, I.; Hulley, G.; Kalashnikova, O.; Katagis, T.; Kuai, L.; Meng, R.; Roberts, D.; Stavros, N. Hyperspectral remote sensing of fire: State-of-the-art and future perspectives. Remote Sens. Environ. 2018, 216, 105–121. [Google Scholar] [CrossRef]
  7. Wang, W.; Li, S.; Qi, H.; Ayhan, B.; Kwan, C.; Vance, S. Identify Anomaly Component by Sparsity and Low Rank. In Proceedings of the 7th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS), Tokyo, Japan, 2–5 June 2015. [Google Scholar]
  8. Chang, C.-I. Hyperspectral Imaging; Springer: New York, NY, USA, 2003. [Google Scholar]
  9. Li, S.; Wang, W.; Qi, H.; Ayhan, B.; Kwan, C.; Vance, S. Low-rank Tensor Decomposition based Anomaly Detection for Hyperspectral Imagery. In Proceedings of the IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada, 27–30 September 2015; pp. 4525–4529. [Google Scholar]
  10. Yang, Y.; Zhang, J.; Song, S.; Liu, D. Hyperspectral Anomaly Detection via Dictionary Construction-Based Low-Rank Representation and Adaptive Weighting. Remote Sens. 2019, 11, 192. [Google Scholar] [CrossRef] [Green Version]
  11. Qu, Y.; Guo, R.; Wang, W.; Qi, H.; Ayhan, B.; Kwan, C.; Vance, S. Anomaly Detection in Hyperspectral Images through Spectral Unmixing and Low Rank Decomposition. In Proceedings of the IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Beijing, China, 10–15 July 2016; pp. 1855–1858. [Google Scholar]
  12. Li, F.; Zhang, L.; Zhang, X.; Chen, Y.; Jiang, D.; Zhao, G.; Zhang, Y. Structured Background Modeling for Hyperspectral Anomaly Detection. Sensors 2018, 18, 3137. [Google Scholar] [CrossRef] [Green Version]
  13. Qu, Y.; Qi, H.; Ayhan, B.; Kwan, C.; Kidd, R. Does Multispectral/Hyperspectral Pansharpening Improve the Performance of Anomaly Detection? In Proceedings of the IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Fort Worth, TX, USA, 23–28 July 2017; pp. 6130–6133. [Google Scholar]
  14. Dao, M.; Kwan, C.; Koperski, K.; Marchisio, G. A Joint Sparsity Approach to Tunnel Activity Monitoring Using High Resolution Satellite Images. In Proceedings of the IEEE Ubiquitous Computing, Electronics & Mobile Communication Conference, New York, NY, USA, 19–21 October 2017; pp. 322–328. [Google Scholar]
  15. Radke, R.J.; Andra, S.; Al-Kofani, O.; Roysam, B. Image change detection algorithms: A systematic survey. IEEE Trans. Image Proc. 2005, 14, 294–307. [Google Scholar] [CrossRef]
  16. İlsever, M.; Unsalan, C. Two-Dimensional Change Detection Methods; Springer: Berlin, Germany, 2012. [Google Scholar]
  17. Zhou, J.; Kwan, C.; Ayhan, B.; Eismann, M. A Novel Cluster Kernel RX Algorithm for Anomaly and Change Detection Using Hyperspectral Images. IEEE Trans. Geosci. Remote Sens. 2016, 54, 6497–6504. [Google Scholar] [CrossRef]
  18. Bovolo, F.; Bruzzone, L. The time variable in data fusion: A change detection perspective. IEEE Geosci. Remote Sens. Mag. 2015, 3, 8–26. [Google Scholar] [CrossRef]
  19. Kwan, C.; Haberle, C.; Echavarren, A.; Ayhan, B.; Chou, B.; Budavari, B.; Dickenshied, S. Mars Surface Mineral Abundance Estimation Using THEMIS and TES Images. In Proceedings of the IEEE Ubiquitous Computing, Electronics & Mobile Communication Conference, New York, NY, USA, 8–10 November 2018. [Google Scholar]
  20. CRISM. Available online: http://crism.jhuapl.edu/ (accessed on 18 December 2019).
  21. Lindgren, D. Land Use Planning and Remote Sensing; Taylor & Francis: Milton Park, UK, 1984; Volume 2. [Google Scholar]
  22. Gu, Y.; Jin, X.; Xiang, R.; Wang, Q.; Wang, C.; Yang, S. UAV-Based Integrated Multispectral-LiDAR Imaging System and Data Processing. Sci. China Technol. Sci. 2020, 1–9. [Google Scholar] [CrossRef]
  23. Jahan, F.; Zhou, J.; Awrangjeb, M.; Gao, Y. Inverse Coefficient of Variation Feature and Multilevel Fusion Technique for Hyperspectral and LiDAR Data Classification. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2020, 13, 367–381. [Google Scholar] [CrossRef] [Green Version]
  24. Wu, Y.; Zhang, X. Object-Based Tree Species Classification Using Airborne Hyperspectral Images and LiDAR Data. Forests 2020, 11, 32. [Google Scholar] [CrossRef] [Green Version]
  25. Hänsch, R.; Hellwich, O. Fusion of Multispectral LiDAR, Hyperspectral, and RGB Data for Urban Land Cover Classification. IEEE Geosci. Remote Sens. Lett. 2020. [Google Scholar] [CrossRef]
  26. Tusa, E.; Laybros, A.; Monnet, J.M.; Dalla Mura, M.; Barré, J.B.; Vincent, G.; Dalponte, M.; Feret, J.-P.; Chanussot, J. Fusion of hyperspectral imaging and LiDAR for forest monitoring. In Data Handling in Science and Technology; Elsevier: Amsterdam, The Netherlands, 2020; Volume 32, pp. 281–303. [Google Scholar]
  27. Du, Q.; Ball, J.E.; Ge, C. Hyperspectral and LiDAR data fusion using collaborative representation. Algorithms Technol. Appl. Multispectr. Hyperspectr. Imagery XXVI 2020, 11392, 1139208. [Google Scholar] [CrossRef]
  28. Zhang, M.; Li, W.; Du, Q.; Gao, L.; Zhang, B. Feature extraction for classification of hyperspectral and LiDAR data using patch-to-patch CNN. IEEE Trans. Cybern. 2018, 50, 100–111. [Google Scholar] [CrossRef]
  29. Mei, X.; Pan, E.; Ma, Y.; Dai, X.; Huang, J.; Fan, F.; Du, Q.; Zheng, H.; Ma, J. Spectral-spatial attention networks for hyperspectral image classification. Remote Sens. 2019, 11, 963. [Google Scholar] [CrossRef] [Green Version]
  30. Senecal, J.J.; Sheppard, J.W.; Shaw, J.A. Efficient Convolutional Neural Networks for Multi-Spectral Image Classification. In Proceedings of the International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary, 14–19 July 2019; pp. 1–8. [Google Scholar]
  31. Zhang, X.; Han, L.; Han, L.; Zhu, L. How Well Do Deep Learning-Based Methods for Land Cover Classification and Object Detection Perform on High Resolution Remote Sensing Imagery? Remote Sens. 2020, 12, 417. [Google Scholar] [CrossRef] [Green Version]
  32. Audebert, N.; Saux, B.L.; Lefevre, S. Semantic Segmentation of Earth Observation Data Using Multimodal and Multi-scale Deep Networks. arXiv 2016, arXiv:1609.06846. [Google Scholar]
  33. Huang, B.; Zhao, B.; Song, Y. Urban land-use mapping using a deep convolutional neural network with high spatial resolution multispectral remote sensing imagery. Remote Sens. Environ. 2018, 214, 73–86. [Google Scholar] [CrossRef]
  34. Kemker, R.; Salvaggio, C.; Kanan, C. Algorithms for Semantic Segmentation of Multispectral Remote Sensing Imagery using Deep Learning. ISPRS J. Photogramm. Remote Sens. 2018, 145, 60–77. [Google Scholar] [CrossRef] [Green Version]
  35. Zheng, C.; Wang, L. Semantic Segmentation of Remote Sensing Imagery Using Object-Based Markov Random Field Model with Regional Penalties. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2015, 8, 1924–1935. [Google Scholar] [CrossRef]
  36. Mura, M.D.; Benediktsson, J.A.; Waske, B.; Bruzzone, L. Extended profiles with morphological attribute filters for the analysis of hyperspectral data. Int. J. Remote Sens. 2010, 31, 5975–5991. [Google Scholar] [CrossRef]
  37. Shu, W.; Liu, P.; He, G.; Wang, G. Hyperspectral Image Classification Using Spectral-Spatial Features with Informative Samples. IEEE Access 2019, 7, 20869–20878. [Google Scholar] [CrossRef]
  38. Zaatour, R.; Bouzidi, S.; Zagrouba, E. Impact of Feature Extraction and Feature Selection Techniques on Extended Attribute Profile-based Hyperspectral Image Classification. In Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2017), Porto, Portugal, 27 February–1 March 2017; pp. 579–586. [Google Scholar]
  39. Bernabe, S.; Marpu, P.R.; Plaza, A.; Benediktsson, J.A. Spectral unmixing of multispectral satellite images with dimensionality expansion using morphological profiles. In Proceedings of the SPIE Satellite Data Compression, Communications, and Processing VIII, San Diego, CA, USA, 19 October 2012; Volume 8514, p. 85140Z. [Google Scholar]
  40. Mura, M.D.; Benediktsson, J.A.; Waske, B.; Bruzzone, L. Morphological attribute profiles for the analysis of very high resolution images. IEEE Trans. Geosci. Remote Sens. 2010, 48, 3747–3762. [Google Scholar] [CrossRef]
  41. Bernabe, S.; Marpu, P.R.; Plaza, A.; Mura, M.D.; Benediktsson, J.A. Spectral–spatial classification of multispectral images using kernel feature space representation. IEEE Geosci. Remote Sens. Lett. 2014, 11, 288–292. [Google Scholar] [CrossRef]
  42. Ghamisi, P.; Benediktsson, J.A.; Cavallaro, G.; Plaza, A. Automatic framework for spectral–spatial classification based on supervised feature extraction and morphological attribute profiles. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2014, 7, 2147–2160. [Google Scholar] [CrossRef]
  43. Dao, M.; Kwan, C.; Bernabe, S.; Plaza, A.; Koperski, K. A Joint Sparsity Approach to Soil Detection Using Expanded Bands of WV-2 Images. IEEE Geosci. Remote Sens. Lett. 2019, 1–5. [Google Scholar] [CrossRef]
  44. Khodadadzadeh, M.; Li, J.; Prasad, S.; Plaza, A. Fusion of Hyperspectral and LiDAR Remote Sensing Data Using Multiple Feature Learning. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2015, 8, 2971–2983. [Google Scholar] [CrossRef]
  45. Li, S.; Qiu, J.; Yang, X.; Liu, H.; Wan, D.; Zhu, Y. A novel approach to hyperspectral band selection based on spectral shape similarity analysis and fast branch and bound search. Eng. Appl. Artif. Intell. 2014, 27, 241–250. [Google Scholar] [CrossRef] [Green Version]
  46. Feng, F.; Li, W.; Du, Q.; Zhang, B. Dimensionality reduction of hyperspectral image with graph-based discriminant analysis considering spectral similarity. Remote Sens. 2017, 9, 323. [Google Scholar] [CrossRef] [Green Version]
  47. Burges, C. A Tutorial on Support Vector Machines for Pattern Recognition. Data Mining and Knowledge Discovery; Kluwer Academic Publishers: Boston, MA, USA, 1998; pp. 121–167. [Google Scholar]
  48. Perez, D.; Banerjee, C.; Kwan, M.; Dao, Y.; Shen, K.; Koperski, G.; Marchisio, G.; Li, J. Deep Learning for Effective Detection of Excavated Soil Related to Illegal Tunnel Activities. In Proceedings of the IEEE Ubiquitous Computing Electronics and Mobile Communication Conference, New York, NY, USA, 19–21 October 2017; pp. 626–632. [Google Scholar]
  49. Srivastava, N.; Hinton, G.; Krizhevsky, A.; Sutskever, I.; Salakhutdinov, R. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 2014, 15, 1929–1958. [Google Scholar]
  50. Deep Learning for HSI Classification. Available online: https://github.com/luozm/Deep-Learning-for-HSI-classification (accessed on 21 April 2020).
  51. Cardillo, G. Cohen’s Kappa: Compute the Cohen’s Kappa Ratio on a 2 × 2 Matrix. Available online: https://www.github.com/dnafinder/Cohen (accessed on 22 April 2020).
  52. Kwan, C.; Gribben, D.; Ayhan, B.; Bernabe, S.; Plaza, A.; Selva, M. Improving Land Cover Classification Using Extended Multi-attribute Profiles (EMAP) Enhanced Color, Near Infrared, and LiDAR Data. Remote Sens. 2020, 26, 1392. [Google Scholar] [CrossRef]
  53. Liao, W.; Pižurica, A.; Bellens, R.; Gautama, S.; Philips, W. Generalized Graph-Based Fusion of Hyperspectral and LiDAR Data Using Morphological Features. IEEE Geosci. Remote Sens. Lett. 2015, 12, 552–556. [Google Scholar] [CrossRef]
  54. Safont, G.; Salazar, A.; Vergara, L. Multiclass alpha integration of scores from multiple classifiers. Neural Comput. 2019, 31, 806–825. [Google Scholar] [CrossRef] [PubMed]
  55. Kwan, C.; Chou, B.; Kwan, L.Y.M.; Larkin, J.; Ayhan, B.; Bell, J.F.; Kerner, H. Demosaicing enhancement using pixel-level fusion. SIViP 2018, 12, 749–756. [Google Scholar] [CrossRef]
Remotesensing 12 02000 g001 550
Figure 1. Our customized conventional neural network (CNN) model.
Figure 1. Our customized conventional neural network (CNN) model.
Remotesensing 12 02000 g001
Remotesensing 12 02000 g002 550
Figure 2. CNN-3D [50].
Figure 2. CNN-3D [50].
Remotesensing 12 02000 g002
Remotesensing 12 02000 g003 550
Figure 3. Color image for the University of Houston area with ground truth land cover annotations of test data.
Figure 3. Color image for the University of Houston area with ground truth land cover annotations of test data.
Remotesensing 12 02000 g003
Remotesensing 12 02000 g004 550
Figure 4. Performance metrics for the investigated band combinations with our customized CNN model.
Figure 4. Performance metrics for the investigated band combinations with our customized CNN model.
Remotesensing 12 02000 g004
Remotesensing 12 02000 g005 550
Figure 5. Classification accuracy for each land cover type with our customized CNN model using the investigated band combinations.
Figure 5. Classification accuracy for each land cover type with our customized CNN model using the investigated band combinations.
Remotesensing 12 02000 g005
Remotesensing 12 02000 g006 550
Figure 6. Estimated land cover maps by our customized CNN method for the whole image.
Figure 6. Estimated land cover maps by our customized CNN method for the whole image.
Remotesensing 12 02000 g006
Remotesensing 12 02000 g007 550
Figure 7. Performance metrics for the investigated band combinations with the CNN-3D model.
Figure 7. Performance metrics for the investigated band combinations with the CNN-3D model.
Remotesensing 12 02000 g007
Remotesensing 12 02000 g008 550
Figure 8. Classification accuracy for each land cover type with CNN-3D model using the investigated band combinations.
Figure 8. Classification accuracy for each land cover type with CNN-3D model using the investigated band combinations.
Remotesensing 12 02000 g008
Remotesensing 12 02000 g009a 550Remotesensing 12 02000 g009b 550
Figure 9. CNN-3D method estimated land cover maps for the whole image.
Figure 9. CNN-3D method estimated land cover maps for the whole image.
Remotesensing 12 02000 g009aRemotesensing 12 02000 g009b
Table
Table 1. Number of pixels per class in the IEEE GRSS data. Total number of unlabeled pixels is 649,816, and total number of pixels is 664,845.
Table 1. Number of pixels per class in the IEEE GRSS data. Total number of unlabeled pixels is 649,816, and total number of pixels is 664,845.
Class
NameNumberColor LegendSamples
TrainTest
Healthy grass1 1981053
Stressed grass2 1901064
Synthetic grass3 192505
Tree4 1881056
Soil5 1861056
Water6 182143
Residential7 1961072
Commercial8 1911053
Road9 1931059
Highway10 1911036
Railway11 1811054
Parking lot 112 1921041
Parking lot 213 184285
Tennis court14 181247
Running track15 181473
1–15 283212,197
Table
Table 2. Classification results obtained by our customized CNN model for the considered test dataset with six different sets of image bands. Bold numbers indicate best performing datasets for each row.
Table 2. Classification results obtained by our customized CNN model for the considered test dataset with six different sets of image bands. Bold numbers indicate best performing datasets for each row.
Bands4 Band4 Band+Lidar44 Band EMAP55 Band EMAP144 Band144 Band+Lidar
Patch Size353535353535
OA (%)0.78440.81080.79900.79900.87920.86020.86020.84290.81480.80500.78780.8097
AA (%)0.81030.83640.82240.82640.89820.88300.88400.86700.84360.83650.82020.8365
K0.76610.79470.78180.78250.86940.84870.84920.83040.80010.78980.77130.7949
(ID)-Class Type (%)
1-Healthy grass0.82240.82340.82050.82240.83100.83100.82340.83100.81580.82620.81670.8234
2-Stressed grass0.84490.84490.84210.84590.83930.82050.84870.71620.84490.84020.83550.8421
3-Synthetic grass0.94260.91090.94850.87921.00001.00000.99410.99800.98610.92870.97620.9822
4-Trees0.92520.91950.89580.92230.98960.92710.95740.92610.91570.92990.92710.9138
5-Soil0.97920.99340.97350.99150.99431.00001.00000.99910.98480.99910.98860.9943
6-Water0.84620.93010.84620.92310.95100.95800.96500.90210.93010.94410.94410.9021
7-Residential0.86470.89180.89090.83860.86470.93470.87500.86750.76870.83490.77240.7817
8-Commercial0.54040.62770.58500.60680.78730.60210.61350.54700.66380.69800.52520.7227
9-Road0.69590.78660.75920.74600.86590.82340.85270.80360.72240.70630.80830.7545
10-Highway0.55310.50100.61200.51540.68440.64000.63710.80700.60910.49520.43730.6004
11-Railway0.68030.70210.67740.68120.86340.89660.86810.86620.76760.70780.72770.7457
12-Parking lot 10.71660.84440.73010.83090.91930.94620.94520.86740.83570.78000.79540.7032
13-Parking lot 20.80350.87370.83160.85960.88420.87020.88770.87370.85960.90880.80350.8807
14-Tennis Court0.97980.99190.96760.98381.00001.00001.00001.00000.98790.95950.99600.9514
15-Running Track0.95980.90490.95560.94930.99790.99580.99151.00000.96190.98940.94930.9493
Table
Table 3. CNN-3D classification results for the test dataset with six different sets of image bands. Bold numbers indicate best performing datasets for each row.
Table 3. CNN-3D classification results for the test dataset with six different sets of image bands. Bold numbers indicate best performing datasets for each row.
Bands4 Band4 Band+Lidar44 Band EMAP55 Band EMAP144 Band144 Band
+Lidar
Patch Size131713171317131777
OA (%)0.75360.61790.78430.70660.87640.86340.87960.87550.80100.8000
AA (%)0.75030.67670.78250.69330.88160.85300.89180.87560.82450.8243
K0.73320.66340.76630.68240.86570.85150.86920.86480.78500.7840
(ID)-Class Type (%)
1-Healthy grass0.80060.82240.82910.79300.82150.84050.83100.83570.84330.8471
2-Stressed grass0.83650.80260.85710.84490.76410.75750.79610.77350.84960.8600
3-Synthetic grass0.57820.43960.62770.49900.99010.99800.99210.96830.78810.7842
4-Trees0.93750.86270.92610.89300.92710.92520.91290.92900.88920.8902
5-Soil0.92990.93750.95740.94030.98670.98770.99340.99620.98580.9886
6-Water0.82520.77620.90210.75520.86010.78320.94410.87410.97900.9441
7-Residential0.84240.83120.85260.82560.91600.89370.92260.89090.86570.8554
8-Commercial0.59540.50900.70560.50240.77870.80530.72550.75780.58120.5793
9-Road0.75730.80830.80740.82150.92920.94240.92070.93960.76770.8017
10-Highway0.57920.38420.62740.41510.68920.66410.68440.70080.59560.4884
11-Railway0.63090.50660.59680.53130.98390.89750.98100.95920.69730.7116
12-Parking lot 10.76660.70030.80020.74930.92220.95770.93850.97120.82710.8204
13-Parking lot 20.80350.75790.84210.74040.85960.84210.86670.78600.92280.9088
14-Tennis Court0.86640.84210.80970.85020.99190.79760.97980.97570.92310.8907
15-Running Track0.50530.16910.59620.23890.80340.70190.88790.77590.85200.9937
Table
Table 4. Classification performance comparison of the two deep learning methods for the test dataset with six different image band sets (best performances are highlighted in bold for each image band set).
Table 4. Classification performance comparison of the two deep learning methods for the test dataset with six different image band sets (best performances are highlighted in bold for each image band set).
CNN
Bands4 Band4 Band+LiDAR44 Band EMAP55 Band EMAP144 Band144 Band+
LiDAR
Patch SizeC3D-13CNN-5C3D-13CNN-5C3D-13CNN-3C3D-13CNN-3C3D-7CNN-3C3D-7CNN-5
OA (%)0.75360.81080.78430.79900.87640.87920.87960.86020.80100.81480.80000.8097
AA (%)0.75030.83640.78250.82640.88160.89820.89180.88400.82450.84360.82430.8365
K0.73320.79470.76630.78250.86570.86940.86920.84920.78500.80010.78400.7949
Table
Table 6. Computational times in minutes for different methods and datasets. CNN and C3D used GPU, whereas JSR and SVM used CPU only.
Table 6. Computational times in minutes for different methods and datasets. CNN and C3D used GPU, whereas JSR and SVM used CPU only.
Computational Times (min)4 Band4 Band+LiDAR44 Band EMAP55 Band EMAP144 Band144 Band+LiDAR
CNN<1<1~1.5~1.5~3~3
C3D<1<1~1~1.5~3~3
JSR629.23891.712248.172198.562210.152310.42
SVM5.323.760.690.471.301.41

Share and Cite

MDPI and ACS Style

Kwan, C.; Ayhan, B.; Budavari, B.; Lu, Y.; Perez, D.; Li, J.; Bernabe, S.; Plaza, A. Deep Learning for Land Cover Classification Using Only a Few Bands. Remote Sens. 2020, 12, 2000. https://doi.org/10.3390/rs12122000

AMA Style

Kwan C, Ayhan B, Budavari B, Lu Y, Perez D, Li J, Bernabe S, Plaza A. Deep Learning for Land Cover Classification Using Only a Few Bands. Remote Sensing. 2020; 12(12):2000. https://doi.org/10.3390/rs12122000

Chicago/Turabian Style

Kwan, Chiman, Bulent Ayhan, Bence Budavari, Yan Lu, Daniel Perez, Jiang Li, Sergio Bernabe, and Antonio Plaza. 2020. "Deep Learning for Land Cover Classification Using Only a Few Bands" Remote Sensing 12, no. 12: 2000. https://doi.org/10.3390/rs12122000

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop