Validation of Multi-Temporal Land-Cover Products Considering Classification Error Propagation
"> Figure 1
<p>Flowchart of time-series sampling.</p> "> Figure 2
<p>Minimum sample size calculated by combining the predicted user precision error rate and significance level.</p> "> Figure 3
<p>Relationship between strata and map classes.</p> "> Figure 4
<p>Misclassification propagation of pixels in the changed strata. The unchanged strata include pixels 1 to 6. The changed strata include pixels 7 to 9. ① indicates that the map and reference data in the changed strata are consistent in the two years; ② is the inconsistency between the map class and the reference in the year (T + 1); Light green means grass; Dark green means forest.</p> "> Figure 5
<p>Interpretation results for the ESA CCI LC data in Google Earth. The red frame refers to the area range of pixel.</p> "> Figure 6
<p>Spatial layout of samples in the base year.</p> "> Figure 7
<p>Proportion of feature categories between the randomly revisited samples and total samples.</p> "> Figure 8
<p>Spatial distribution of the samples. (<b>a</b>) Samples in 2010. (<b>b</b>–<b>f</b>) Samples randomly revisited in the years from 2011 to 2015.</p> "> Figure 8 Cont.
<p>Spatial distribution of the samples. (<b>a</b>) Samples in 2010. (<b>b</b>–<b>f</b>) Samples randomly revisited in the years from 2011 to 2015.</p> "> Figure 9
<p>Single-year precision results for the average of 10 times extracted from the single-temporal samples in 2015. (<b>a</b>) Difference between the accuracy when eliminating the misclassification and including the misclassification. (<b>b</b>) Difference between the single-year accuracy and single-temporal accuracy. WMC represents the difference between the accuracy when eliminating misclassification and the single-temporal accuracy (reference value). IMC represents the difference between the accuracy when including the misclassification and the single-temporal accuracy.</p> "> Figure 10
<p>Single-year precision results of the average of ten times extracted from single-temporal samples in 2010. (<b>a</b>) Difference between the accuracy of eliminating the misclassification and including the misclassification. (<b>b</b>) Difference between the single-year accuracy and single-temporal accuracy.</p> "> Figure 11
<p>Single-year precision results of the average of ten times extracted from single-temporal samples in 2010.</p> "> Figure 12
<p>The accuracy of the <b>ESA CCI LC</b> for 2010–2015.</p> "> Figure 13
<p>User’s accuracy and producer’s accuracy of each class. (<b>a</b>) refers to Crop; (<b>b</b>) refers to Forest; (<b>c</b>) refers to Grass; (<b>d</b>) refers to Shrub; (<b>e</b>) refers to Water; (<b>f</b>) refers to Bareland; (<b>g</b>) refers to Urban; (<b>h</b>) refers to Snow; (<b>i</b>) refers to Sparse Veg.</p> "> Figure 13 Cont.
<p>User’s accuracy and producer’s accuracy of each class. (<b>a</b>) refers to Crop; (<b>b</b>) refers to Forest; (<b>c</b>) refers to Grass; (<b>d</b>) refers to Shrub; (<b>e</b>) refers to Water; (<b>f</b>) refers to Bareland; (<b>g</b>) refers to Urban; (<b>h</b>) refers to Snow; (<b>i</b>) refers to Sparse Veg.</p> "> Figure 14
<p>Stability of the producer’s accuracy for the ESA CCI LC product.</p> "> Figure 15
<p>Stability of the user’s accuracy for the ESA CCI LC product.</p> ">
Abstract
:1. Introduction
2. Methodology
2.1. Sampling Design
2.1.1. Determination of the Total Samples
2.1.2. Selection of Sampling Method
2.1.3. Determination of the Samples for the Changed Strata
2.1.4. Evaluation of the Accuracy
2.2. Process of Updating and Supplementing Time-Series Samples
- (1)
- Starting from the nth year (base year) of the time series, the samples of the nth year are obtained by stratified sampling (see Section 2.1.1 and Section 2.1.2 for the specific sample size and sample distribution of each stratum);
- (2)
- Determining the sample of the changed strata: The changes of the multi-temporal product pixel by pixel according to the sequence of years are detected. The samples from the changed area to supplement the samples in the corresponding “changed stratum” are selected (see Section 2.1.3 for the number of samples to be distributed in each changed stratum);
- (3)
- Supplementing samples: The samples of the changed strata are screened out, and the pixels that spread misclassification are eliminated;
- (4)
- Updating samples: In order to achieve rapid map verification after the release of land cover products and avoid too much time lag between the verification results and the release time of products, the updating of verification data sets needs to be cost-effective without affecting the statistical rigor. Therefore, some revisits are used to update the verification of the data set. The update is divided into targeted and random revisits. Targeted revisit: if the samples in the base year fall within the detected change area, the samples in the base year are removed and added to the corresponding changed stratum. Random revisit: starting from each year after the base year, (100/k)% non-repetitive random sampling inspection on the sample set selected in the base year is conducted, and it is ensured that all the sample sets in the first step are sampled once in year n + k.
3. Materials
3.1. Land-Cover Data
3.2. Reference Data
4. Results
4.1. Analysis of the Results of the Sampling Design
4.1.1. Selection of Samples in the Base Year
4.1.2. Determination of Supplementary Samples
4.2. Evaluation of the Advantages of Sample Revisiting
4.3. Analysis of the Accuracy Evaluation Results
4.4. Stability Evaluation of Land-Cover Products
5. Discussion
6. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
Appendix A
Reference Data | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Crop | Forest | Grass | Shrub | Water | Bare | Urban | Ice | Sparse Veg | Sum | ||
Map | Crop | 110 | 15 | 16 | 7 | 0 | 3 | 0 | 0 | 3 | 154 |
Forest | 0 | 79 | 4 | 9 | 0 | 2 | 0 | 0 | 2 | 96 | |
Grass | 9 | 1 | 14 | 10 | 0 | 36 | 0 | 1 | 3 | 74 | |
Shrub | 2 | 1 | 10 | 46 | 0 | 1 | 0 | 0 | 24 | 84 | |
Water | 2 | 0 | 1 | 0 | 68 | 0 | 0 | 1 | 0 | 72 | |
Bare | 0 | 0 | 0 | 0 | 0 | 69 | 0 | 1 | 17 | 87 | |
Urban | 5 | 1 | 1 | 0 | 0 | 1 | 39 | 0 | 0 | 47 | |
Snow | 0 | 0 | 0 | 0 | 2 | 1 | 0 | 140 | 0 | 143 | |
Sparse Veg | 0 | 0 | 3 | 1 | 1 | 11 | 0 | 4 | 11 | 31 | |
Sum | 128 | 97 | 49 | 73 | 71 | 124 | 39 | 147 | 60 | 788 |
Reference Data | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Crop | Forest | Grass | Shrub | Water | Bare | Urban | Ice | Sparse Veg | Sum | ||
Map | Crop | 235 | 33 | 24 | 32 | 0 | 9 | 1 | 0 | 3 | 337 |
Forest | 5 | 212 | 10 | 27 | 1 | 5 | 1 | 0 | 2 | 263 | |
Grass | 21 | 2 | 26 | 16 | 1 | 75 | 0 | 4 | 3 | 148 | |
Shrub | 5 | 4 | 11 | 98 | 0 | 24 | 0 | 1 | 31 | 174 | |
Water | 2 | 0 | 1 | 0 | 140 | 2 | 0 | 1 | 0 | 146 | |
Bare | 0 | 0 | 0 | 0 | 0 | 151 | 1 | 2 | 23 | 177 | |
Urban | 7 | 2 | 3 | 0 | 0 | 2 | 140 | 0 | 0 | 154 | |
Snow | 0 | 0 | 0 | 0 | 4 | 3 | 0 | 330 | 0 | 337 | |
Sparse Veg | 0 | 1 | 6 | 1 | 1 | 32 | 0 | 10 | 26 | 77 | |
Sum | 275 | 254 | 81 | 174 | 147 | 303 | 143 | 348 | 88 | 1813 |
Reference Data | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Crop | Forest | Grass | Shrub | Water | Bare | Urban | Ice | Sparse Veg | Sum | ||
Map | Crop | 700 | 86 | 60 | 119 | 1 | 26 | 2 | 0 | 0 | 994 |
Forest | 16 | 657 | 46 | 102 | 1 | 12 | 1 | 0 | 1 | 836 | |
Grass | 50 | 11 | 101 | 45 | 2 | 227 | 1 | 11 | 5 | 453 | |
Shrub | 14 | 23 | 14 | 264 | 0 | 87 | 0 | 2 | 33 | 437 | |
Water | 2 | 1 | 0 | 0 | 362 | 8 | 0 | 8 | 0 | 381 | |
Bare | 3 | 1 | 0 | 1 | 0 | 434 | 4 | 11 | 37 | 491 | |
Urban | 29 | 11 | 5 | 0 | 0 | 8 | 586 | 0 | 0 | 639 | |
Snow | 0 | 0 | 0 | 0 | 7 | 20 | 0 | 943 | 0 | 970 | |
Sparse Veg | 10 | 5 | 7 | 9 | 0 | 101 | 0 | 32 | 100 | 264 | |
Sum | 824 | 795 | 233 | 540 | 373 | 923 | 594 | 1007 | 176 | 5465 |
References
- Schewe, J.; Gosling, S.N.; Reyer, C.; Zhao, F.; Ciais, P.; Elliott, J.; Francois, L.; Huber, V.; Lotze, H.K.; Seneviratne, S.I.; et al. State-of-the-art global models underestimate impacts from climate extremes. Nat. Commun. 2019, 10, 1005. [Google Scholar] [CrossRef] [PubMed]
- Gómez, C.; White, J.C.; Wulder, M.A. Optical remotely sensed time series data for land cover classification: A review. ISPRS-J. Photogramm. Remote Sens. 2016, 116, 55–72. [Google Scholar] [CrossRef]
- Houghton, R.A.; House, J.I.; Pongratz, J.; van der Werf, G.R.; DeFries, R.S.; Hansen, M.C.; Le Quere, C.; Ramankutty, N. Carbon emissions from land use and land-cover change. Biogeosciences 2012, 9, 5125–5142. [Google Scholar] [CrossRef]
- Wulder, M.A.; Coops, N.C.; Roy, D.P.; White, J.C.; Hermosilla, T. Land cover 2.0. Int. J. Remote Sens. 2018, 39, 4254–4284. [Google Scholar] [CrossRef]
- Foley, J.A.; DeFries, R.; Asner, G.P.; Barford, C.; Bonan, G.; Carpenter, S.R.; Chapin, F.S.; Coe, M.T.; Daily, G.C.; Gibbs, H.K.; et al. Global consequences of land use. Science 2005, 309, 570–574. [Google Scholar] [CrossRef] [PubMed]
- Li, J.Y.; Gao, Y.; Huang, X. The impact of urban agglomeration on ozone precursor conditions: A systematic investigation across global agglomerations utilizing multisource geospatial datasets. Sci. Total Environ. 2020, 704, 135458. [Google Scholar] [CrossRef] [PubMed]
- Xiao, R.; Liu, Y.; Huang, X.; Shi, R.; Yu, W.; Zhang, T. Exploring the driving forces of farmland loss under rapid urbanization using binary logistic regression and spatial regression: A case study of Shanghai and Hangzhou Bay. Ecol. Indic. 2018, 95, 455–467. [Google Scholar] [CrossRef]
- Yang, Q.; Huang, X.; Tang, Q. The footprint of urban heat island effect in 302 Chinese cities: Temporal trends and associated factors. Sci. Total Environ. 2019, 655, 652–662. [Google Scholar] [CrossRef] [PubMed]
- Chapin, F.S.; Zavaleta, E.S.; Eviner, V.T.; Naylor, R.L.; Vitousek, P.M.; Reynolds, H.L.; Hooper, D.U.; Lavorel, S.; Sala, O.E.; Hobbie, S.E.; et al. Consequences of changing biodiversity. Nature 2020, 405, 234–242. [Google Scholar] [CrossRef] [PubMed]
- Turner, B.L.; Lambin, E.F.; Reenberg, A. The emergence of land change science for global environmental change and sustainability. Proc. Natl. Acad. Sci. USA 2007, 104, 20666–20671. [Google Scholar] [CrossRef] [PubMed]
- Buchhorn, M.; Lesiv, M.; Tsendbazar, N.E.; Herold, M.; Bertels, L.; Smets, B. Copernicus global Land Cover layers—Collection 2. Remote Sens. 2020, 12, 1044. [Google Scholar] [CrossRef]
- Liu, H.; Gong, P.; Wang, J.; Clinton, N.; Bai, Y.; Liang, S. Annual dynamics of global land cover and its long-term changes from 1982 to 2015. Earth Syst. Sci. Data 2020, 12, 1217–1243. [Google Scholar] [CrossRef]
- Justice, C.; Gutman, G.; Vadrevu, K.P. NASA Land Cover and Land Use Change (LCLUC): An interdisciplinary research program. J. Environ. Manag. 2015, 148, 4–9. [Google Scholar] [CrossRef] [PubMed]
- Brown, J.F.; Tollerud, H.J.; Barber, C.P.; Zhou, Q.; Dwyer, J.L.; Vogelmann, J.E.; Loveland, T.R.; Woodcock, C.E.; Stehman, S.V.; Zhu, Z.; et al. Lessons learned implementing an operational continuous United States national land change monitoring capability: The Land Change Monitoring, Assessment, and Projection (LCMAP) approach. Remote Sens. Environ. 2020, 238, 111356. [Google Scholar] [CrossRef]
- Zhu, Z.; Qiu, S.; Ye, S. Remote sensing of land change: A multifaceted perspective. Remote Sens. Environ. 2022, 282, 113266. [Google Scholar] [CrossRef]
- Townshend, J.; Justice, C.; Li, W.; Gurney, C.; Mcmanus, J. Global land cover classification by remote sensing: Present capabilities and future possibilities. Remote Sens. Environ. 1991, 35, 243–255. [Google Scholar] [CrossRef]
- Olofsson, P.; Stehman, S.V.; Woodcock, C.E.; Sulla-Menashe, D.; Sibley, A.M.; Newell, J.D.; Friedl, M.A.; Herold, M. A global land-cover validation data set, part I: Fundamental design principles. Int. J. Remote Sens. 2012, 33, 5768–5788. [Google Scholar] [CrossRef]
- Findell, K.L.; Berg, A.; Gentine, P.; Krasting, J.P.; Lintner, B.R.; Malyshev, S.; Santanello, J.A.; Shevliakova, E. The impact of anthropogenic land use and land cover change on regional climate extremes. Nat. Commun. 2017, 8, 989. [Google Scholar] [CrossRef] [PubMed]
- Stehman, S.V.; Czaplewski, R.L. Design and Analysis for Thematic Map Accuracy Assessment: Fundamental Principles. Remote Sens. Environ. 1998, 64, 331–344. [Google Scholar] [CrossRef]
- Stehman, S.V. Statistical Rigor and Practical Utility in Thematic Map Accuracy Assessment. Photogramm. Eng. Remote Sens. 2001, 67, 727–734. [Google Scholar]
- Grekousis, G.; Mountrakis, G.; Kavouras, M. An overview of 21 global and 43 regional land-cover mapping products. Int. J. Remote Sens. 2015, 36, 5309–5335. [Google Scholar] [CrossRef]
- Loveland, T.R.; Reed, B.C.; Brown, J.F.; Ohlen, D.O.; Zhu, Z.; Yang, L.; Merchant, J.W. Development of a global land cover characteristics database and IGBP DISCover from 1 km AVHRR data. Int. J. Remote Sens. 2000, 21, 1303–1330. [Google Scholar] [CrossRef]
- Bartholomé, E.; Belward, A.S. GLC2000: A new approach to global land cover mapping from Earth Observation data. Int. J. Remote Sens. 2005, 26, 1959–1977. [Google Scholar] [CrossRef]
- Herold, M.; Mayaux, P.; Woodcock, C.E.; Baccini, A.; Schmullius, C. Some challenges in global land cover mapping: An assessment of agreement and accuracy in existing 1 km datasets. Remote Sens. Environ. 2008, 112, 2538–2556. [Google Scholar] [CrossRef]
- Latham, J.; Cumani, R.; Rosati, I.; Bloise, M. Global Land Cover SHARE (GLC-SHARE) Database Beta-Release Version 1.0; Land and Water Division: Rome, Italy, 2014. [Google Scholar]
- Kaptue Tchuenté, A.T.; Roujean, J.L.; De Jong, S.M. Comparison and relative quality assessment of the GLC2000, GLOBCOVER, MODIS and ECOCLIMAP land cover data sets at the African continental scale. Int. J. Appl. Earth Obs. Geoinf. 2011, 13, 207–219. [Google Scholar] [CrossRef]
- Gong, P.; Liu, H.; Zhang, M.N.; Li, C.C.; Wang, J.; Huang, H.B.; Clinton, N.; Ji, L.Y.; Li, W.Y.; Bai, Y.Q.; et al. Stable classification with limited sample: Transferring a 30-m resolution sample set collected in 2015 to mapping 10-m resolution global land cover in 2017. Sci. Bull. 2019, 64, 370–373. [Google Scholar] [CrossRef] [PubMed]
- Chen, J.; Chen, J.; Liao, A.P.; Cao, X.; Chen, L.J.; Chen, X.H.; He, C.Y.; Han, G.; Peng, S.; Lu, M.; et al. Global land cover mapping at 30 m resolution: A POK-based operational approach. ISPRS-J. Photogramm. Remote Sens. 2015, 103, 7–27. [Google Scholar] [CrossRef]
- Chen, J.; Chen, L.J.; Chen, F.; Ban, Y.F.; Li, S.N.A.; Han, G.; Tong, X.H.; Liu, C.; Stamenova, V.; Stamenov, S. Collaborative validation of GlobeLand30: Methodology and practices. Geo-Spat. Inf. Sci. 2021, 24, 134–144. [Google Scholar] [CrossRef]
- Woodcock, C.E.; Loveland, T.R.; Herold, M.; Bauer, M.E. Transitioning from change detection to monitoring with remote sensing: A paradigm shift. Remote Sens. Environ. 2020, 238, 111558. [Google Scholar] [CrossRef]
- Wulder, M.A.; Roy, D.P.; Radeloff, V.C.; Loveland, T.R.; Anderson, M.C.; Johnson, D.M.; Healey, S.; Zhu, Z.; Scambos, T.A.; Pahlevan, N.; et al. Fifty years of Landsat science and impacts. Remote Sens. Environ. 2022, 280, 113195. [Google Scholar] [CrossRef]
- Wickham, J.; Stehman, S.V.; Gass, L.; Dewitz, J.; Fry, J.A.; Wade, T.G. Accuracy assessment of NLCD 2006 land cover and impervious surface. Remote Sens. Environ. 2013, 130, 294–304. [Google Scholar] [CrossRef]
- Wickham, J.; Stehman, S.V.; Gass, L.; Dewitz, J.A.; Sorenson, D.G.; Granneman, B.J.; Poss, R.V.; Baer, L.A. The accuracy assessment of the 2011 National Land Cover Database (NLCD). Remote Sens. Environ. 2017, 191, 328–341. [Google Scholar] [CrossRef]
- Wickham, J.; Stehman, S.V.; Sorenson, D.G.; Gass, L.; Dewitz, J.A. Thematic accuracy assessment of the NLCD 2016 land cover for the conterminous United States. Remote Sens. Environ. 2021, 257, 112357. [Google Scholar] [CrossRef]
- Wickham, J.; Stehman, S.V.; Sorenson, D.G.; Gass, L.; Dewitz, J.A. Thematic accuracy assessment of the NLCD 2019 land cover for the conterminous United States. GISci. Remote Sens. 2023, 60, 2181143. [Google Scholar] [CrossRef]
- Tsendbazar, N.; Herold, M.; Li, L.; Tarko, A.; de Bruin, S.; Masiliunas, D.; Lesiv, M.; Fritz, S.; Buchhorn, M.; Smets, B.; et al. Towards operational validation of annual global land cover maps. Remote Sens. Environ. 2021, 266, 112686. [Google Scholar] [CrossRef]
- Tang, X.J.; Bullock, E.L.; Olofssonm, P.; Estel, S.; Woodcock, C.E. Near real-time monitoring of tropical forest disturbance: New algorithms and assessment framework. Remote Sens. Environ. 2019, 224, 202–218. [Google Scholar] [CrossRef]
- Potapov, P.; Turubanova, S.; Hansen, M.C.; Tyukavina, A.; Zalles, V.; Khan, A.; Song, X.P.; Pickens, A.; Shen, Q.; Cortez, J. Global maps of cropland extent and change show accelerated cropland expansion in the twenty-first century. Nat. Food 2021, 3, 19–28. [Google Scholar] [CrossRef] [PubMed]
- Druce, D.; Tong, X.Y.; Lei, X.; Guo, T.; Kittel, C.M.M.; Grogan, K.; Tottrup, C. An Optical and SAR Based Fusion Approach for Mapping Surface Water Dynamics over Mainland China. Remote Sens. 2021, 13, 1663. [Google Scholar] [CrossRef]
- Song, X.P.; Hansen, M.C.; Stehman, S.V.; Potapov, P.V.; Tyukavina, A.; Vermote, E.F.; Townshend, J.R. Global land change from 1982 to 2016. Nature 2018, 560, 639–643. [Google Scholar] [CrossRef] [PubMed]
- Potapov, P.; Hansen, M.C.; Pickens, A.; Hernandez-Serna, A.; Tyukavina, A.; Turubanova, S.; Zalles, V.; Li, X.Y.; Khan, A.; Stolle, F.; et al. The Global 2000-2020 Land Cover and Land Use Change Dataset Derived from the Landsat Archive: First Results. Front. Remote Sens. 2022, 3, 856903. [Google Scholar] [CrossRef]
- Arévalo, P.; Olofsson, P.; Woodcock, C.E. Continuous monitoring of land change activities and post-disturbance dynamics from Landsat time series: A test methodology for REDD+ reporting. Remote Sens. Environ. 2020, 238, 111051. [Google Scholar] [CrossRef]
- Gong, Y.L.; Xie, H.; Liao, S.C.; Lu, Y.; Jin, Y.M.; Wei, C.; Tong, X.H. Assessing the Accuracy of Multi-Temporal GlobeLand30 Products in China Using a Spatiotemporal Stratified Sampling Method. Remote Sens. 2023, 15, 4593. [Google Scholar] [CrossRef]
- Strahler, A.H.; Boschetti, L.; Foody, G.M.; Friedl, M.A.; Hansen, M.C.; Herold, M.; Mayaux, P.; Morisette, J.T.; Stehman, S.V.; Woodcock, C.E. Global Land Cover Validation: Recommendations for Evaluation and Accuracy Assessment of Global Land Cover Maps; Office for Official Publications of the European Communities: Luxembourg, 2006. [Google Scholar]
- Stehman, S.V.; Foody, G.M. Key issues in rigorous accuracy assessment of land cover products. Remote Sens. Environ. 2019, 231, 111199. [Google Scholar] [CrossRef]
- Ye, S.; Pontius, R.G.; Rakshit, R. A review of accuracy assessment for object-based image analysis: From per-pixel to per-polygon approaches. ISPRS-J. Photogramm. Remote Sens. 2018, 141, 137–147. [Google Scholar] [CrossRef]
- Van Oort, P.A.J. Improving land cover change estimates by accounting for classification errors. Int. J. Remote Sens. 2005, 26, 3009–3024. [Google Scholar] [CrossRef]
- Pouliot, D.; Latifovic, R.; Zabcic, N.; Guindon, L.; Olthof, I. Development and assessment of a 250 m spatial resolution MODIS annual land cover time series (2000–2011) for the forest region of Canada derived from change-based updating. Remote Sens. Environ. 2014, 140, 731–743. [Google Scholar] [CrossRef]
- Asiamah, N.; Mensah, H.K.; Oteng-abayie, E.F. Do Larger Samples Really Lead to More Precise Estimates? A Simulation Study. Am. J. Educ. Res. 2017, 5, 9–17. [Google Scholar]
- Milligan, G.W. Is sampling really dead? Qual. Prog. 1991, 24, 77–81. [Google Scholar]
- Xie, H.; Tong, X.H.; Meng, W.; Liang, D.; Wang, Z.H.; Shi, W.Z. A Multilevel Stratified Spatial Sampling Approach for the Quality Assessment of Remote-Sensing-Derived Products. IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens. 2015, 8, 4699–4713. [Google Scholar] [CrossRef]
- Czaplewski, R.L.; and Patterson, P.L. Classification accuracy for stratification with remotely sensed data. For. Sci. 2003, 49, 402–408. [Google Scholar] [CrossRef]
- Szantoi, Z.; Brink, A.; Lupi, A.; Mammone, C.; Jaffrain, G. Key landscapes for conservation land cover and change monitoring, thematic and validation datasets for sub-Saharan Africa. Earth Syst. Sci. Data 2020, 12, 3001–3019. [Google Scholar] [CrossRef]
- Gallaun, H.; Steinegger, M.; Wack, R.; Schardt, M.; Kornberger, B.; Schmitt, U. Remote Sensing Based Two-Stage Sampling for Accuracy Assessment and Area Estimation of Land Cover Changes. Remote Sens. 2015, 7, 11992–12008. [Google Scholar] [CrossRef]
- Bossard, M.; Feranec, J.; Otahel, J. CORINE Land Cover Technical Guide: Addendum 2000; European Environment Agency Copenhagen: Copenhagen, Denmark, 2000. [Google Scholar]
- Stehman, S.V. Estimating area and map accuracy for stratified random sampling when the strata are different from the map classes. Int. J. Remote Sens. 2014, 35, 4923–4939. [Google Scholar] [CrossRef]
- Karra, K.; Kontgis, C.; Statman-Weil, Z.; Mazzariello, J.C.; Mathis, M.; Brumby, S.P. Global Land Use/Land Cover with Sentinel 2 and Deep Learning; IEEE: Manhattan, NY, USA, 2021. [Google Scholar]
- Venter, Z.S.; Barton, D.N.; Chakraborty, T.; Simensen, T.; Singh, G. Global 10 m Land Use Land Cover Datasets: A Comparison of Dynamic World, World Cover and Esri Land Cover. Remote Sens. 2022, 14, 4101. [Google Scholar] [CrossRef]
- Defourny, P.; Lamarche, C.; Marissiaux, Q.; Carsten, B.; Martin, B.; Grit, K. Product User Guide and Specification: ICDR Land Cover 2016–2020; European Centre for Medium-Range Weather Forecasts (ECMWF): Reading, UK, 2021. [Google Scholar]
- Defourny, P.; Lamarche, C.; Bontemps, S.; De Maet, T.; Van Bogaert, E.; Moreau, I.; Brockmann, C.; Boettcher, M.; Kirches, G.; Wevers, J.; et al. Land Cover CCI Product User Guide–Version 2.0; UCL-Geomatics: Louvain-la-Neuve, Belgium, 2017. [Google Scholar]
- Sun, W.Y.; Ding, X.T.; Su, J.B. Land use and cover changes on the Loess Plateau: A comparison of six global or national land use and cover datasets. Land Use Pol. 2022, 119, 106165. [Google Scholar] [CrossRef]
- Xu, Y.D.; Yu, L.; Feng, D.L.; Peng, D.L.; Li, C.C.; Huang, X.M.; Lu, H.; Gong, P. Comparisons of three recent moderate resolution African land cover datasets: CGLS-LC100, ESA-S2-LC20, and FROM-GLC-Africa30. Int. J. Remote Sens. 2019, 40, 6185–6202. [Google Scholar] [CrossRef]
- Liu, X.X.; Yu, L.; Si, Y.L.; Zhang, C.; Lu, H.; Yu, C.Q.; Gong, P. Identifying patterns and hotspots of global land cover transitions using the ESA CCI Land Cover dataset. Remote Sens. Lett. 2018, 9, 972–981. [Google Scholar] [CrossRef]
- Mousivanda, A.; Arsanjani, J.J. Insights on the historical and emerging global land cover changes: The case of ESA-CCI-LC datasets. Appl. Geogr. 2019, 106, 82–92. [Google Scholar] [CrossRef]
- Jiang, L.; Yu, L. Analyzing land use intensity changes within and outside protected areas using ESA CCI-LC datasets. Glob. Ecol. Conserv. 2019, 20, e00789. [Google Scholar] [CrossRef]
- Bontemps, S.; Defourny, P.; Van Bogaert, E.; Arino, O.; Kalogirou, V.; Perez, J.R. GLOBCOVER 2009 Products Description and Validation Report; UCLouvain & ESA Team: Louvain-la-Neuve, Belgium, 2011. [Google Scholar]
- Liu, P.Y.; Pei, J.; Guo, H.; Tian, H.F.; Fang, H.J.; Wang, L. Evaluating the Accuracy and Spatial Agreement of Five Global Land Cover Datasets in the Ecologically Vulnerable South China Karst. Remote Sens. 2022, 14, 3090. [Google Scholar] [CrossRef]
- Liu, S.; Liu, X.X.; Yu, L.; Wang, Y.; Zhang, G.J.; Gong, P.; Huang, W.Y.; Wang, B.; Yang, M.M.; Cheng, Y.Q. Climate response to introduction of the ESA CCI land cover data to the NCAR CESM. Clim. Dyn. 2021, 56, 4109–4127. [Google Scholar] [CrossRef]
- Liu, S.S.; Su, H.; Cao, G.F.; Wang, S.Q.; Guan, Q.F. Learning from data: A post classification method for annual land cover analysis in urban areas. ISPRS-J. Photogramm. Remote Sens. 2019, 154, 202–215. [Google Scholar] [CrossRef]
Methods | Advantages | Sampling Parameters | Considering Classification Error |
---|---|---|---|
Wickham et al. (2013) [32] | Reducing the evaluation lag in multi-temporal data of land-cover | Empirically determined sample size | No |
Tsendbazar et al. (2021) [36] | Empirically determined sampling parameters | No | |
Tang et al. (2019) [37] | Monitoring the change of a single class in different periods | Cochran sampling model; fixed sample size for the rare land type | No |
Potapov et al. (2021) [38] | Empirically determined sample size | No | |
Druce et al. (2021) [39] | Empirically determined sample size | No | |
Arévalo et al. (2020) [42] | Improving the stratification efficiency of land-cover change | Cochran sampling model; fixed sample size for the rare land type | No |
Gong et al. (2023) [43] | Probabilistic statistical sampling model; proportional distribution; fixed sample size for the rare land type | No |
Classes Considered in This Study | Land-Use Types Used in the CCI LC Maps | Description |
---|---|---|
Agriculture | 10, 11, 12 | Rainfed cropland |
20 | Irrigated cropland | |
30 | Mosaic cropland (>50%)/natural vegetation (tree, shrub, herbaceous cover) (<50%) | |
40 | Mosaic natural vegetation (tree, shrub, herbaceous cover) (>50%)/cropland (<50%) | |
Forest | 50 | Tree cover, broad-leaved, evergreen, closed to open (>15%) |
60, 61, 62 | Tree cover, broad-leaved, deciduous, closed to open (>15%) | |
70, 71, 72 | Tree cover, needle-leaved, evergreen, closed to open (>15%) | |
80, 81, 82 | Tree cover, needle-leaved, deciduous, closed to open (>15%) | |
90 | Tree cover, mixed leaf type (broad-leaved and needle-leaved) | |
100 | Mosaic tree and shrub (>50%)/herbaceous cover (<50%) | |
110 | Mosaic herbaceous cover (>50%)/tree and shrub (<50%) | |
160 | Tree cover, flooded, fresh, or brackish water | |
170 | Tree cover, flooded, saline water | |
Grass | 130 | Grassland |
140 | Lichens and mosses | |
180 | Shrub or herbaceous cover, flooded, fresh-saline, or brackish water | |
Shrub | 120, 121, 122 | Shrubland |
Water | 210 | Water |
Bare land | 200, 201, 202 | Bare areas |
Urban | 190 | Urban areas |
Ice/snow | 220 | Permanent snow and ice |
Sparse vegetation | 150, 151, 152, 153 | Sparse vegetation (tree, shrub, herbaceous cover) |
No. | Surface Features | Number of Pixels | Predicted Classification Accuracy | Area Ratio (Number of Pixels of Surface Feature/Total Pixels) | Total Sample Size |
---|---|---|---|---|---|
1 | Crop | 332019546 | 80% | 3.95% | 14,645 |
2 | Forest | 668301893 | 85% | 7.96% | |
3 | Grass | 253197417 | 50% | 3.01% | |
4 | Shrub | 175707096 | 60% | 2.09% | |
5 | Water | 5674946201 | 90% | 67.57% | |
6 | Bare | 253115927 | 80% | 3.01% | |
7 | Urban | 8842764 | 85% | 0.11% | |
8 | Ice | 871449834 | 90% | 10.38% | |
9 | Sparse Veg | 160499322 | 30% | 1.91% |
No. | Surface Features | Number of Pixels | Allocation | Area Ratio |
---|---|---|---|---|
1 | Crop | 332019546 | 2082 | 16.57% |
2 | Forest | 668301893 | 3345 | 33.35% |
3 | Grass | 253197417 | 1786 | 12.64% |
4 | Shrub | 175707096 | 1496 | 8.77% |
5 | Water | 66389169 | 1085 | 3.31% |
6 | Bare | 253115927 | 1786 | 12.63% |
7 | Urban | 8842764 | 870 | 0.44% |
8 | Ice | 85822331 | 1158 | 4.28% |
9 | Sparse Veg | 160499322 | 1438 | 8.01% |
Adjacent Years | The Total Samples in the Changed Strata |
---|---|
2010–2011 | 65 |
2011–2012 | 49 |
2012–2013 | 41 |
2013–2014 | 72 |
2014–2015 | 10 |
2010–2011 | 2011–2012 | 2012–2013 | 2013–2014 | 2014–2015 | |
---|---|---|---|---|---|
Urban gain | 100 | 100 | 109 | 100 | 111 |
Forest loss | 100 | 100 | 112 | 161 | 138 |
Forest gain | 167 | 160 | 133 | 140 | - |
Agriculture loss | 100 | 100 | 100 | 100 | - |
Agriculture gain | 100 | 100 | 100 | 100 | - |
Catch-all | 135 | 136 | 108 | 100 | 100 |
Year 2010 | Year 2011 | Year 2012 | ||||||
---|---|---|---|---|---|---|---|---|
Strata | CIM | CEM | Strata | CIM | CEM | Strata | CIM | CEM |
Crop loss | 0.2750 | 0 | Crop gain | 0.6154 | 0 | Crop gain | 0.5833 | 0.1111 |
Forest loss | 0.6667 | 0 | Forest gain | 0.7692 | 0 | Forest gain | 0.5417 | 0.1667 |
Grass loss | 0 | 0 | Grass gain | 0 | 0 | Grass gain | 0 | 0 |
Shrub loss | 0 | - | Shrub gain | - | - | Shrub gain | - | - |
Water loss | 1 | - | Water gain | 0.5 | 0 | Water gain | - | - |
Bare loss | 0.6842 | 0 | Bare gain | 0.5 | 0 | Bare gain | 0.6667 | 0 |
Urban loss | - | - | Urban gain | 0.9216 | 0.2 | Urban gain | 0.9333 | 0.4 |
Snow loss | - | - | Snow gain | - | - | Snow gain | - | - |
Sparse loss | 0.4286 | 0 | Sparse gain | 0.3750 | 0 | Sparse gain | 0.1429 | 0 |
Year 2013 | Year 2014 | Year 2015 | ||||||
Strata | CIM | CEM | Strata | CIM | CEM | Strata | CIM | CEM |
Crop gain | 0.9048 | - | Crop gain | 0.4667 | 0 | Crop gain | - | - |
Forest gain | 0.3684 | 0 | Forest gain | 0.5750 | 0 | Forest gain | - | - |
Grass gain | 0 | 0 | Grass gain | 0 | 0 | Grass gain | - | - |
Shrub gain | - | - | Shrub gain | - | - | Shrub gain | - | - |
Water gain | - | - | Water gain | 1 | - | Water gain | - | - |
Bare gain | 0 | - | Bare gain | 0.4 | 0 | Bare gain | 0 | 0 |
Urban gain | 0.9412 | 0 | Urban gain | 0.8966 | 0 | Urban gain | 0.9079 | 0 |
Snow gain | - | - | Snow gain | - | - | Snow gain | - | - |
Sparse gain | 0.25 | 0 | Sparse gain | 0 | 0 | Sparse gain | - | - |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Liao, S.; Xie, H.; Gong, Y.; Jin, Y.; Xu, X.; Chen, P.; Tong, X. Validation of Multi-Temporal Land-Cover Products Considering Classification Error Propagation. Remote Sens. 2024, 16, 2968. https://doi.org/10.3390/rs16162968
Liao S, Xie H, Gong Y, Jin Y, Xu X, Chen P, Tong X. Validation of Multi-Temporal Land-Cover Products Considering Classification Error Propagation. Remote Sensing. 2024; 16(16):2968. https://doi.org/10.3390/rs16162968
Chicago/Turabian StyleLiao, Shicheng, Huan Xie, Yali Gong, Yanmin Jin, Xiong Xu, Peng Chen, and Xiaohua Tong. 2024. "Validation of Multi-Temporal Land-Cover Products Considering Classification Error Propagation" Remote Sensing 16, no. 16: 2968. https://doi.org/10.3390/rs16162968