[go: up one dir, main page]

Skip to main content

Techniques for the Validation of LUCC Modeling Outputs

  • Chapter
  • First Online:
Geomatic Approaches for Modeling Land Change Scenarios

Abstract

Validation is the third stage in the modeling process, after calibration and simulation, and also applies to scenarios. It is an essential part of the process in that the credibility of a model depends on the accuracy of its output. A large range of validation approaches and tools exist, many of which can also be used during the calibration stage. In this chapter we distinguish between purely quantitative validation techniques and those that also consider the spatial allocation of simulated land use/cover changes (LUCC). According to model outputs and objectives, simulation maps can be either hard-classified or soft-classified. While some validation techniques apply to both types of map (cross tabulation matrices and indices, congruence of model outputs), others are specific to one. Techniques such as LUCC indicators, feature and pattern recognition and error analysis are used to validate hard-classified simulation maps, while ROC is used to test soft-classified maps. We then look at a second validation approach based on LUCC dynamics such as LUCC components, intensity analysis, data uncertainty and the impact of spatial and temporal scales. Finally, we compare a group of the most common model software programs (those used by the contributors to parts II and III of this book), in order to list their validation capabilities.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    See the short presentations in Part V of this book about (in alphabetical order) APoLUS, CA_MARKOV, CLUMondo, Dinamica EGO, Land Change Modeler (LCM), LucSim, Metronamica and SLEUTH. The authors are also grateful to all contributors who helped us understand the different software packages.

References

  • Abdou M, Hamill L, Gilbert N (2012) Designing and Building an Agent-Based Model. In: Heppenstall et al (eds) Agent-based models of geographical systems. Springer, pp 141–166

    Google Scholar 

  • Ahlqvist O (2008) Extending post-classification change detection using semantic similarity metrics to overcome class heterogeneity: a study of 1992 and 2001 US National Land Cover Database changes. Remote Sens Environ 112(3):1226–1241

    Article  Google Scholar 

  • Aldwaik SZ, Pontius RG Jr (2012) Intensity analysis to unify measurements of size and stationaity of land changes by interval, category, and transition. Landsc Urban Plann 106:103–114

    Article  Google Scholar 

  • Alcamo J (2008) Environmental futures: the practice of environmental scenario analysis. Developments in integrated environmental assessment, vol 2. Elsevier, Amsterdam

    Google Scholar 

  • Alcamo J, Schaldach R, Koch J, Kölking C, Lapola D, Priess J (2011) Evaluation of an integrated land use change model including a scenario analysis of land use change for continental Africa. Environ Model Softw 26(8):1017–1027. doi:10.1016/j.envsoft.2011.03.002

    Article  Google Scholar 

  • Allen THF, Starr B (1982) Hierarchy: Perspectives for ecological complexity. University of Chicago Press, Chicago 310 p

    Google Scholar 

  • Burnicki AC, Brown DG, Goovaerts P (2007) Simulating error propagation in land-cover change analysis: The implications of temporal dependence. Comput Environ Urban Syst 31:282–302

    Article  Google Scholar 

  • Bone C, Dragicevic S, White R (2011) Modeling-in-the middle: bridging the gap between agent-based modeling and multi-objective decision-making for land use change. Int J Geogr Inf Sci 25(5):717–737. doi:10.1080/13658816.2010.495076

    Article  Google Scholar 

  • Botequilha A, Miller J, Ahern J, McGarigal K (2006) Measuring landscapes. A planner´s handbook. Island Press, Washington

    Google Scholar 

  • Brown D et al (2005) Path dependence and the validation of agent-based spatial models of land use. Int J Geogr Inf Sci 19:153–174

    Article  Google Scholar 

  • Camacho Olmedo MT, Paegelow M, Mas JF (2013) Interest in intermediate soft-classified maps in land change model validation: suitability versus transition potential. Int J Geogr Inf Sci 27(12):2343–2361

    Article  Google Scholar 

  • Camacho Olmedo MT, Pontius RG Jr, Paegelow M, Mas JF (2015) Comparison of simulation models in terms of quantity and allocation of land change. Environ Modell Softw 69:214–221 (Elsevier)

    Google Scholar 

  • Colas R (2016) Comprendre les changements d’utilisation des terres en France pour mieux estimer leurs impacts sur les émissions à effet de serre. De l’observation à la modélisation. PhD thesis, University Paris 7, 530 p

    Google Scholar 

  • Conway TM, Wellen CC (2011) Not developed yet? Alternative ways to include locations without changes in land use change models. Int J Geogr Inf Sci 25(10):1613–1631. doi:10.1080/13658816.2010.534738

    Article  Google Scholar 

  • Coquillard P, Hill DRC (1997) Modélisation et simulation d’écosystèmes. Des modèles déterministes aux simulations à événements discrets, Paris, Masson

    Google Scholar 

  • Croks AT, Heppenstall AJ (2012) Introduction to Agent-based modelling. In: Heppenstall et al (eds) Agent-based models of geographical systems. Springer, pp 85–108

    Google Scholar 

  • Eastman JR, Jin W, Kyem PAK, Toleando J (1995) Raster procedures for multi-criteria/multi-objective decisions. Photogramm Eng Remote Sens 61(5):539–547

    Google Scholar 

  • Eastman JR, Solorzano LA, Van Fossen ME (2005) Transition potential modeling for landcover change. In: Maguire DJ, Batty M, Goodchild MF (eds) GIS, spatial analysis, and modeling. ESRI, Redland CA, pp 357–385

    Google Scholar 

  • Eastman JR (2009) Idrisi Taiga, Guide to GIS and Image Processing, manual version 16.02, Clark University 342 pp

    Google Scholar 

  • Engelen G, White R (2007) Validating and calibrating integrated cellular automata based models of land use change. In: Albeverio S et al (eds) The dynamics of complex urban systems. An interdisciplinary approach, Physica, pp 185–211

    Google Scholar 

  • Forman RTT (1995) Land Mosaics: the ecology of landscapes and regions. EEUU, Cambridge

    Google Scholar 

  • Gaucherel C (2007) Multiscale heterogeneity map and associated scaling profile for landscape analysis. Landsc Urban Plann 82(3):95–102

    Article  Google Scholar 

  • Gaucherel C, Alleaume S, Hély C (2008) The Comparison Map Profile method: a strategy for multiscale comparison of quantitative and qualitative images. IEEE Trans Geosci Remote Sens 46(9):2708–2719

    Article  Google Scholar 

  • Gómez Delgado M, Barredo JI (2005) Sistemas de Información Geográfica y evaluación multicriterio en la ordenación del territorio (GIS and multicriteria evaluation for urban and regional planning). Ra-Ma, Madrid

    Google Scholar 

  • Gómez Delgado M, Tarantola S (2006) Global sensitivity analysis, GIS and multi-criteria evaluation for a sustainable planning of hazardous waste disposal site in Spain. Int J Geogr Inf Sci 20:449–466

    Article  Google Scholar 

  • Hagen A (2003) Fuzzy set approach to assessing similarity of categorical maps. Int J Geogr Inf Sci 17(3):235–249

    Article  Google Scholar 

  • Hagen-Zanker A, Straatman B, Uljee I (2005) Further developments of a fuzzy set map comparison approach. Int J Geogr Inf Sci 19:769–785

    Article  Google Scholar 

  • Hagen-Zanker A (2009) An improved Fuzzy Kappa statistic that accounts for spatial autocorrelation. Int J Geogr Inf Sci 23(1):61–73. doi:10.1080/13658810802570317

    Article  Google Scholar 

  • Houet T, Gourmelon F (2014) La géoprospective—Apport de la dimension spatiale aux approches prospectives. Cybergéo 8 pp. http://cybergeo.revues.org/26194

  • Houet T, Aguejdad R, Doukari O, Battaia G, Clarke K (2016) Description and validation of a “non path-dependent” model for projecting contrasting urban growth futures. Cybergeo, document 759, 6 Jan 2016. http://cybergeo.revues.org/27397

  • Huang J, Pontius RG Jr, Li Q, Zhang Y (2012) Use of intensity analysis to link patterns with process of land change from 1986 to 2007 in a coastal watershed of southeast China. Appl Geogr 34:371–384

    Article  Google Scholar 

  • Jansen LJM (2006) Harmonisation of land-use class sets to facilitate compatibility and comparability of data across space and time. In: 12th CEReS international symposium, 13–14 Dec 2005, Japan 29 pp

    Google Scholar 

  • Jackson DA (1995) PROTEST: a procrustean randomization test of community environment concordance. Ecoscience 2:297–303

    Article  Google Scholar 

  • Jokar Arsanjani J (2012) Dynamic land-use/cover change simulation: geosimulation and multi agent-based modelling. Springer, Springer Theses

    Book  Google Scholar 

  • Jansen LJM, Veldkamp TA (2011) Evaluation of the variation in semantic contents of class sets on modelling dynamics of land-use changes. Int J Geogr Inf Sci. doi:10.1080/13658816.2011.609989

    Google Scholar 

  • Kim JH (2013) Spatiotemporal scale dependency and other sensitivities in dynamic land-use change simulations. Int J Geogr Inf Sci 27:1782–1803

    Article  Google Scholar 

  • Klug W, Graziani G, Grippa G, Pierce D, Tassone C (eds) (1992) Evaluation of long range atmospheric transport models using environmental radioactivity data from the Chernobyl accident: the ATMES report. Elsevier, London

    Google Scholar 

  • Krüger C, Lakes T (2015) Revealing uncertainties in land change modeling using probabilities. Trans GIS 20(4):526–546

    Google Scholar 

  • Kuhnert M, Voinov A, Seppelt R (2005) Comparing raster map comparison algorithms for spatial modelling and analysis. Photogramm Eng Remote Sens 71(8):975–984

    Article  Google Scholar 

  • Kamusoko C, Aniya M, Adi B, Manjoro M (2009) Rural sustainability under threat in Zimbabwe-Simulation of future land use/cover changes in the Bindura district based on the Markov-cellular automata model. Appl Geogr 29:435–447. doi:10.1016/j.apgeog.2008.10.002

    Article  Google Scholar 

  • Li X, Chen Y, Liu X, Li D, He J (2011) Concepts, methodologies and tools of an integrated geographical simulation and optimization system. Int J GIS 25(4):633–655

    Google Scholar 

  • Lee YJ, Lee JW, Chai DJ, Hwang BH, Ryu KH (2009) Mining temporal interval relational rules from temporal data. J Syst Softw 82:155–167

    Article  Google Scholar 

  • Lin YP, Chu HJ, Wu CF, Verburg PH (2011) Predictive ability of logistic regression, auto-logistic regression and neural network models in empirical land-use change modeling—a case study. Int J Geogr Inf Sci 25(1):65–87. doi:10.1080/13658811003752332

    Article  Google Scholar 

  • Liu J, Deng X (2010) Progress of the research methodologies on the temporal and spatial process of LUCC. Chin Sci Bull 55:1354–1362

    Article  Google Scholar 

  • McGarigal K, Marks BJ (1995) FRAGSTATS: spatial pattern analysis program for quantifying landscape structure. USDA For Serv Gen Tech Rep

    Google Scholar 

  • Mas JF, Kolb M, Houet T, Paegelow M, Camacho Olmedo MT (2011) Eclairer le choix de modèles de simulation des changements des modes d’occupation et d’usages des sols. Revue Internationale de Géomatique 21(3):405–430

    Article  Google Scholar 

  • Mas JF, Soares Filho B, Pontius RG Jr, Farfan Gutierrez M, Rodrigues H (2013) A suite of tools for ROC analysis of spatial models. ISPRS Int J Geo-Inf 2(3):869–887. doi:10.3390/ijgi2030869

    Article  Google Scholar 

  • Mas JF, Kolb M, Paegelow M, Camacho Olmedo MT, Houet T (2014) Inductive pattern-based land use/cover change models: a comparison of four software packages. Environ Model Softw 51:94–111

    Article  Google Scholar 

  • Mobaied S, Riera B, Lalanne A, Baguette M, Machon N (2011) The use of diachronic spatial approaches and predictive modelling to study the vegetation dynamics of a managed heathland. Biodivers Conserv 20:73–88. doi:10.1007/s10531-010-9947-1

    Article  Google Scholar 

  • Ngo TA, See L (2012) Calibration and validation of agent-based models of land cover change. In: Heppenstall et al (eds) Agent-based models of geographical systems. Springer, pp 181–198

    Google Scholar 

  • North MJ, Macal CM (2007) Managing business complexity: Discovering strategic solutions with agent-based modelling and simulation. Oxford University Press, New York

    Book  Google Scholar 

  • Oreskes N, Shrader-Frechette K, Belitz K (1994) Verification, validation and confirmation of numerical-models in the earth-sciences. Science 263:641–646

    Article  Google Scholar 

  • Paegelow M, Camacho Olmedo MT (2005) Possibilities and limits of prospective GIS land cover modelling—a compared case study: Garrotxes (France) and Alta Alpujarra Granadina (Spain). Int J Geogr Inf Sci 19(6):697–722

    Article  Google Scholar 

  • Paegelow M, Camacho Olmedo MT (eds) (2008) Modelling environmental dynamics. Advances in geomatic solutions. Springer, series: Environmental science and engineering

    Google Scholar 

  • Paegelow M, Camacho Olmedo MT, Mas JF, Houet T, Pontius RG Jr (2013) Land change modelling: moving beyond projections. Int J Geogr Inf Sci 27(9):1691–1695

    Article  Google Scholar 

  • Paegelow M, Camacho Olmedo MT, Mas JF, Houet T (2014) Benchmarking of LUCC modelling tools by various validation techniques and error analysis. Cybergeo 701, 22 Dec 2014. http://cybergeo.revues.org

  • Perica S, Foufoula-Georgiou E (1996) Model for multiscale disaggregation of spatial rainfall based on coupling meteorological and scaling descriptions. J Geophys Res 101(D21):26347–26361

    Article  Google Scholar 

  • Pérez-Vega A, Mas JF, Ligmann-Zielinska A (2012) Comparing two approaches to land use/cover change modelling and their implications for the assessment of biodiversity loss in a deciduous tropical forest. Environ Model Softw 29(1):11–23

    Article  Google Scholar 

  • Pontius RG Jr (2000) Quantification error versus location error in comparison of categorical maps. Photogramm Eng Remote Sens 66(8):1011–1016

    Google Scholar 

  • Pontius RG Jr (2002) Statistical methods to partition effects of quantity and location during comparison of categorical maps at multiple resolutions. Photogramm Eng Remote Sens 68:1041–1049

    Google Scholar 

  • Pontius RG Jr, Pacheco P (2004) Calibration and validation of a model of forest disturbance in the Western Ghats, India 1920–1990. GeoJournal 61: 325–334

    Google Scholar 

  • Pontius RG Jr, Schneider LC (2001) Land-cover change model validation by an ROC method for the Ipswich watershed, Massachusetts, USA. Agric Ecosyst Environ 85:239–248

    Article  Google Scholar 

  • Pontius RG Jr, Huffaker D, Denman K (2004a) Useful techniques of validation for spatially explicit land-change models. Ecol Model 179(4):445–461

    Article  Google Scholar 

  • Pontius RG Jr, Shusas E, McEachern M (2004b) Detecting important categorical land changes while accounting for persistence. Agric Ecosyst Environ 101:251–268

    Article  Google Scholar 

  • Pontius RG Jr, Malanson J (2005) Comparison of the structure and accuracy of two land change models. Int J Geogr Inf Sci 19:243–265

    Article  Google Scholar 

  • Pontius RG Jr, Cheuk ML (2006) A generalized cross-tabulation matrix to compare soft-classified maps at multiple resolutions. Int J Geogr Inf Sci 20(1):1–30. doi:10.1080/13658810500391024

    Article  Google Scholar 

  • Pontius RG Jr, Lippitt CD (2006) Can error explain map differences over time? Cartogr Geogr Inf Sci 33(2):159–171

    Article  Google Scholar 

  • Pontius RG Jr, Verluis AJ, Malizia NR (2006) Visualizing certainty of extrapolations from models of land change. Landsc Ecol 21(7):1151–1166

    Article  Google Scholar 

  • Pontius RG Jr, Walker RT, Yao-Kumar R, Arima E, Aldrich S, Caldas M, Vergara D (2007) Accuracy assessment for a simulation model of Amazonian deforestation. Ann Assoc Am Geogr 97:677–695

    Article  Google Scholar 

  • Pontius Jr RG, LI X (2010) Land transition estimates from erroneous maps. J Land Use Sci 5(1):31–44

    Google Scholar 

  • Pontius RG Jr, Petrova SH (2010) Assessing a predictive model on land change using uncertain data. Environ Model Softw 25:299–309

    Article  Google Scholar 

  • Pontius RG Jr, Boersma W, Castella JC, Clarke K, de Nijs T, Dietzel C, Duan Z, Fotsing E, Goldstein N, Kok K, Koomen E, Lippitt CD, McConnell W, Sood AM, Pijankowski B, Pidhadia S, Sweeney S, Trung TN, Veldkamp AT, Verburg PH (2008) Comparing the input, output, and validation maps for several models of land change. Anna Region Sci 42(1):11–27

    Article  Google Scholar 

  • Pontius RG Jr, Millones M (2011) Death to kappa and to some of my previous work: a better alternative. Int J Remote Sens 32(15):4407–4429

    Article  Google Scholar 

  • Pontius RG Jr, Gao Y, Giner NM, Kohyama T, Osaki M, Hirose K (2013) Design and interpretation of intensity analysis illustrated by land change in Central Kalimantan, Indonesia. Land 2(3):351–369. doi:10.3390/land2030351

    Article  Google Scholar 

  • Pontius Jr RG, Si K (2013) The total operating characteristic to measure diagnostic ability for multiple thresholds. Int J Geogr Inf Sci

    Google Scholar 

  • Rodrigues HO, Soares Filho BS, de Souza Costa WL (2007) Dinamica EGO, uma plataforma para modelagem de sistemas ambientais. Anais XIII Simposio Brasileiro de Sensoriamento Remoto, INPE 3089–3096

    Google Scholar 

  • Runfola DM, Pontius RG Jr (2013) Measuring the temporal instability of land change using the flow matrix. Int J Geogr Inf Sci 26(9):1696–1716

    Article  Google Scholar 

  • Rykiel EJJ (1996) Testing ecological models: the meaning of validation. Ecol Model 90:229–244

    Article  Google Scholar 

  • Sargent RG (2009) Verification and validation of simulation models. In: Rosetti MDD, Hill RR, Johansson B, Dunkin A, Ingalls RG (eds) Proceedings of the 2009 winter simulation conference. IEEE, Syracuse, NY, pp 162–176

    Google Scholar 

  • Schneider LC, Pontius RG Jr (2001) Modeling land-use change in the Ipswich watershed, Massachusetts, USA. Agric Ecosyst Environ 85:83–94

    Article  Google Scholar 

  • Shirley LJ, Battaglia LL (2008) Projecting fine resolution land cover dynamics for a rapidly changing terrestrial-aquatic transition in Terrebone basin, Louisiana, USA. J Coastal Res 246:1545–1554

    Article  Google Scholar 

  • Silva TS, Tagliani PRA (2012) Environmental planning in the medium littoral of the Rio Grande do Sul coastal plain—Southern Brazil: elements for coastal management. Ocean Coast Manag 59:20–30

    Article  Google Scholar 

  • Soares-Filho BS, Pennachin CL, Cerqueira G (2002) DINAMICA—a stochastic cellular automata model designed to simulate the landscape dynamics in an Amazonian colonization frontier. Ecol Model 154(3):217–235

    Article  Google Scholar 

  • Sohl TL et al (2012) Spatially explicit land-use and land-cover scenarios for the Great Plains of the United States. Agric Ecosyst Environ 153:1–15. doi:10.1016/j.agee.2012.02.019

    Article  Google Scholar 

  • Steiner F, McSherry L, Cohen J (2000) Land suitability analysis for the upper Gila River watershed. Landsc Urban Plann 50(4):199–214

    Article  Google Scholar 

  • Syphard AD, Clarke KC, Franklin J, Regan HM, Mcginnis M (2011) Forecasts of habitat loss and fragmentation due to urban growth are sensitive to source of input data. J Environ Manage 92:1882–1893

    Article  Google Scholar 

  • Takada T, Miyamoto A, Hasegawa SF (2010) Derivation of a yearly transition probability matrix for land-use dynamics and its applications. Landsc Ecol 25(4):561–572

    Article  Google Scholar 

  • Torrens PM (2011) Calibrating and validating cellular automata models of urbanization. In: Yang X (ed) Urban remote sensing. Monitoring, synthesis and modelling in the urban environment. Wiley, New York, pp 335–345

    Chapter  Google Scholar 

  • Turner MG, Constanza R, Sklar FH (1989) Methods to evaluate the performance of spatial simulation models. Ecol Model 48:1–18

    Article  Google Scholar 

  • Villa N, Paegelow M, Camacho Olmedo MT, Cornez L, Ferraty F, Ferré L, Sarda P (2006) Various approaches for predicting land cover in Mediterranean mountains. Commun Stat 36(1):73–86

    Article  Google Scholar 

  • Visser H, de Nijs T (2006) The map comparison kit. Environ Model Softw 21:346–358

    Article  Google Scholar 

  • van Vliet J, Bregt AK, Brown DG, van Delden H, Heckbert S, Verburg PH (2016) A review of current calibration and validation practices in land change modelling. EMS 82:174–182

    Google Scholar 

  • Wang J, Mountrakis G (2011) Developing a multi-network urbanization model: A case study of urban growth in Denver, Colorado. Int J Geogr Inf Sci 25(2):229–253. doi:10.1080/13658810903473213

    Article  Google Scholar 

  • White R, Uljee I, Engelen G (2012) Integrated modelling of population, employment, and land use change with a multiple activity based variable grid cellular automaton. Int J GIS 26:1251–1280

    Google Scholar 

  • White R, Engelen G, Uljee I (1997) The use of constrained cellular automata for high resolution modelling of urban land use dynamics. Environ Plan 24:323–343

    Article  Google Scholar 

  • Yu J, Chen Y, Wu J, Khan S (2011) Cellular automata-based spatial multi-criteria land suitability simulation for irrigated agriculture. Int J Geogr Inf Sci 25(1):131–148. doi:10.1080/13658811003785571

    Article  Google Scholar 

  • Krüger C and Lakes T (2015) Revealing Uncertainties in Land Change Modeling using probabilities. Transactions in GIS, v20-4, 526-546

    Google Scholar 

Download references

Acknowledgements

This work was supported by the BIA2013-43462-P project funded by the Spanish Ministry of Economy and Competitiveness and by the Regional European Fund FEDER. This study was also supported by the Consejo Nacional de Ciencia y Tecnología (CONACYT) and the Secretaría de Educación Pública through the project ¿Puede la modelación espacial ayudarnos a entender los procesos de cambio de cobertura/uso del suelo y de degradación ambiental? Fondos SEP-CONACyT 178816.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to M. Paegelow .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this chapter

Cite this chapter

Paegelow, M., Camacho Olmedo, M.T., Mas, J.F. (2018). Techniques for the Validation of LUCC Modeling Outputs. In: Camacho Olmedo, M., Paegelow, M., Mas, JF., Escobar, F. (eds) Geomatic Approaches for Modeling Land Change Scenarios. Lecture Notes in Geoinformation and Cartography. Springer, Cham. https://doi.org/10.1007/978-3-319-60801-3_4

Download citation

Publish with us

Policies and ethics