Abstract
Validation is the third stage in the modeling process, after calibration and simulation, and also applies to scenarios. It is an essential part of the process in that the credibility of a model depends on the accuracy of its output. A large range of validation approaches and tools exist, many of which can also be used during the calibration stage. In this chapter we distinguish between purely quantitative validation techniques and those that also consider the spatial allocation of simulated land use/cover changes (LUCC). According to model outputs and objectives, simulation maps can be either hard-classified or soft-classified. While some validation techniques apply to both types of map (cross tabulation matrices and indices, congruence of model outputs), others are specific to one. Techniques such as LUCC indicators, feature and pattern recognition and error analysis are used to validate hard-classified simulation maps, while ROC is used to test soft-classified maps. We then look at a second validation approach based on LUCC dynamics such as LUCC components, intensity analysis, data uncertainty and the impact of spatial and temporal scales. Finally, we compare a group of the most common model software programs (those used by the contributors to parts II and III of this book), in order to list their validation capabilities.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
See the short presentations in Part V of this book about (in alphabetical order) APoLUS, CA_MARKOV, CLUMondo, Dinamica EGO, Land Change Modeler (LCM), LucSim, Metronamica and SLEUTH. The authors are also grateful to all contributors who helped us understand the different software packages.
References
Abdou M, Hamill L, Gilbert N (2012) Designing and Building an Agent-Based Model. In: Heppenstall et al (eds) Agent-based models of geographical systems. Springer, pp 141–166
Ahlqvist O (2008) Extending post-classification change detection using semantic similarity metrics to overcome class heterogeneity: a study of 1992 and 2001 US National Land Cover Database changes. Remote Sens Environ 112(3):1226–1241
Aldwaik SZ, Pontius RG Jr (2012) Intensity analysis to unify measurements of size and stationaity of land changes by interval, category, and transition. Landsc Urban Plann 106:103–114
Alcamo J (2008) Environmental futures: the practice of environmental scenario analysis. Developments in integrated environmental assessment, vol 2. Elsevier, Amsterdam
Alcamo J, Schaldach R, Koch J, Kölking C, Lapola D, Priess J (2011) Evaluation of an integrated land use change model including a scenario analysis of land use change for continental Africa. Environ Model Softw 26(8):1017–1027. doi:10.1016/j.envsoft.2011.03.002
Allen THF, Starr B (1982) Hierarchy: Perspectives for ecological complexity. University of Chicago Press, Chicago 310 p
Burnicki AC, Brown DG, Goovaerts P (2007) Simulating error propagation in land-cover change analysis: The implications of temporal dependence. Comput Environ Urban Syst 31:282–302
Bone C, Dragicevic S, White R (2011) Modeling-in-the middle: bridging the gap between agent-based modeling and multi-objective decision-making for land use change. Int J Geogr Inf Sci 25(5):717–737. doi:10.1080/13658816.2010.495076
Botequilha A, Miller J, Ahern J, McGarigal K (2006) Measuring landscapes. A planner´s handbook. Island Press, Washington
Brown D et al (2005) Path dependence and the validation of agent-based spatial models of land use. Int J Geogr Inf Sci 19:153–174
Camacho Olmedo MT, Paegelow M, Mas JF (2013) Interest in intermediate soft-classified maps in land change model validation: suitability versus transition potential. Int J Geogr Inf Sci 27(12):2343–2361
Camacho Olmedo MT, Pontius RG Jr, Paegelow M, Mas JF (2015) Comparison of simulation models in terms of quantity and allocation of land change. Environ Modell Softw 69:214–221 (Elsevier)
Colas R (2016) Comprendre les changements d’utilisation des terres en France pour mieux estimer leurs impacts sur les émissions à effet de serre. De l’observation à la modélisation. PhD thesis, University Paris 7, 530 p
Conway TM, Wellen CC (2011) Not developed yet? Alternative ways to include locations without changes in land use change models. Int J Geogr Inf Sci 25(10):1613–1631. doi:10.1080/13658816.2010.534738
Coquillard P, Hill DRC (1997) Modélisation et simulation d’écosystèmes. Des modèles déterministes aux simulations à événements discrets, Paris, Masson
Croks AT, Heppenstall AJ (2012) Introduction to Agent-based modelling. In: Heppenstall et al (eds) Agent-based models of geographical systems. Springer, pp 85–108
Eastman JR, Jin W, Kyem PAK, Toleando J (1995) Raster procedures for multi-criteria/multi-objective decisions. Photogramm Eng Remote Sens 61(5):539–547
Eastman JR, Solorzano LA, Van Fossen ME (2005) Transition potential modeling for landcover change. In: Maguire DJ, Batty M, Goodchild MF (eds) GIS, spatial analysis, and modeling. ESRI, Redland CA, pp 357–385
Eastman JR (2009) Idrisi Taiga, Guide to GIS and Image Processing, manual version 16.02, Clark University 342 pp
Engelen G, White R (2007) Validating and calibrating integrated cellular automata based models of land use change. In: Albeverio S et al (eds) The dynamics of complex urban systems. An interdisciplinary approach, Physica, pp 185–211
Forman RTT (1995) Land Mosaics: the ecology of landscapes and regions. EEUU, Cambridge
Gaucherel C (2007) Multiscale heterogeneity map and associated scaling profile for landscape analysis. Landsc Urban Plann 82(3):95–102
Gaucherel C, Alleaume S, Hély C (2008) The Comparison Map Profile method: a strategy for multiscale comparison of quantitative and qualitative images. IEEE Trans Geosci Remote Sens 46(9):2708–2719
Gómez Delgado M, Barredo JI (2005) Sistemas de Información Geográfica y evaluación multicriterio en la ordenación del territorio (GIS and multicriteria evaluation for urban and regional planning). Ra-Ma, Madrid
Gómez Delgado M, Tarantola S (2006) Global sensitivity analysis, GIS and multi-criteria evaluation for a sustainable planning of hazardous waste disposal site in Spain. Int J Geogr Inf Sci 20:449–466
Hagen A (2003) Fuzzy set approach to assessing similarity of categorical maps. Int J Geogr Inf Sci 17(3):235–249
Hagen-Zanker A, Straatman B, Uljee I (2005) Further developments of a fuzzy set map comparison approach. Int J Geogr Inf Sci 19:769–785
Hagen-Zanker A (2009) An improved Fuzzy Kappa statistic that accounts for spatial autocorrelation. Int J Geogr Inf Sci 23(1):61–73. doi:10.1080/13658810802570317
Houet T, Gourmelon F (2014) La géoprospective—Apport de la dimension spatiale aux approches prospectives. Cybergéo 8 pp. http://cybergeo.revues.org/26194
Houet T, Aguejdad R, Doukari O, Battaia G, Clarke K (2016) Description and validation of a “non path-dependent” model for projecting contrasting urban growth futures. Cybergeo, document 759, 6 Jan 2016. http://cybergeo.revues.org/27397
Huang J, Pontius RG Jr, Li Q, Zhang Y (2012) Use of intensity analysis to link patterns with process of land change from 1986 to 2007 in a coastal watershed of southeast China. Appl Geogr 34:371–384
Jansen LJM (2006) Harmonisation of land-use class sets to facilitate compatibility and comparability of data across space and time. In: 12th CEReS international symposium, 13–14 Dec 2005, Japan 29 pp
Jackson DA (1995) PROTEST: a procrustean randomization test of community environment concordance. Ecoscience 2:297–303
Jokar Arsanjani J (2012) Dynamic land-use/cover change simulation: geosimulation and multi agent-based modelling. Springer, Springer Theses
Jansen LJM, Veldkamp TA (2011) Evaluation of the variation in semantic contents of class sets on modelling dynamics of land-use changes. Int J Geogr Inf Sci. doi:10.1080/13658816.2011.609989
Kim JH (2013) Spatiotemporal scale dependency and other sensitivities in dynamic land-use change simulations. Int J Geogr Inf Sci 27:1782–1803
Klug W, Graziani G, Grippa G, Pierce D, Tassone C (eds) (1992) Evaluation of long range atmospheric transport models using environmental radioactivity data from the Chernobyl accident: the ATMES report. Elsevier, London
Krüger C, Lakes T (2015) Revealing uncertainties in land change modeling using probabilities. Trans GIS 20(4):526–546
Kuhnert M, Voinov A, Seppelt R (2005) Comparing raster map comparison algorithms for spatial modelling and analysis. Photogramm Eng Remote Sens 71(8):975–984
Kamusoko C, Aniya M, Adi B, Manjoro M (2009) Rural sustainability under threat in Zimbabwe-Simulation of future land use/cover changes in the Bindura district based on the Markov-cellular automata model. Appl Geogr 29:435–447. doi:10.1016/j.apgeog.2008.10.002
Li X, Chen Y, Liu X, Li D, He J (2011) Concepts, methodologies and tools of an integrated geographical simulation and optimization system. Int J GIS 25(4):633–655
Lee YJ, Lee JW, Chai DJ, Hwang BH, Ryu KH (2009) Mining temporal interval relational rules from temporal data. J Syst Softw 82:155–167
Lin YP, Chu HJ, Wu CF, Verburg PH (2011) Predictive ability of logistic regression, auto-logistic regression and neural network models in empirical land-use change modeling—a case study. Int J Geogr Inf Sci 25(1):65–87. doi:10.1080/13658811003752332
Liu J, Deng X (2010) Progress of the research methodologies on the temporal and spatial process of LUCC. Chin Sci Bull 55:1354–1362
McGarigal K, Marks BJ (1995) FRAGSTATS: spatial pattern analysis program for quantifying landscape structure. USDA For Serv Gen Tech Rep
Mas JF, Kolb M, Houet T, Paegelow M, Camacho Olmedo MT (2011) Eclairer le choix de modèles de simulation des changements des modes d’occupation et d’usages des sols. Revue Internationale de Géomatique 21(3):405–430
Mas JF, Soares Filho B, Pontius RG Jr, Farfan Gutierrez M, Rodrigues H (2013) A suite of tools for ROC analysis of spatial models. ISPRS Int J Geo-Inf 2(3):869–887. doi:10.3390/ijgi2030869
Mas JF, Kolb M, Paegelow M, Camacho Olmedo MT, Houet T (2014) Inductive pattern-based land use/cover change models: a comparison of four software packages. Environ Model Softw 51:94–111
Mobaied S, Riera B, Lalanne A, Baguette M, Machon N (2011) The use of diachronic spatial approaches and predictive modelling to study the vegetation dynamics of a managed heathland. Biodivers Conserv 20:73–88. doi:10.1007/s10531-010-9947-1
Ngo TA, See L (2012) Calibration and validation of agent-based models of land cover change. In: Heppenstall et al (eds) Agent-based models of geographical systems. Springer, pp 181–198
North MJ, Macal CM (2007) Managing business complexity: Discovering strategic solutions with agent-based modelling and simulation. Oxford University Press, New York
Oreskes N, Shrader-Frechette K, Belitz K (1994) Verification, validation and confirmation of numerical-models in the earth-sciences. Science 263:641–646
Paegelow M, Camacho Olmedo MT (2005) Possibilities and limits of prospective GIS land cover modelling—a compared case study: Garrotxes (France) and Alta Alpujarra Granadina (Spain). Int J Geogr Inf Sci 19(6):697–722
Paegelow M, Camacho Olmedo MT (eds) (2008) Modelling environmental dynamics. Advances in geomatic solutions. Springer, series: Environmental science and engineering
Paegelow M, Camacho Olmedo MT, Mas JF, Houet T, Pontius RG Jr (2013) Land change modelling: moving beyond projections. Int J Geogr Inf Sci 27(9):1691–1695
Paegelow M, Camacho Olmedo MT, Mas JF, Houet T (2014) Benchmarking of LUCC modelling tools by various validation techniques and error analysis. Cybergeo 701, 22 Dec 2014. http://cybergeo.revues.org
Perica S, Foufoula-Georgiou E (1996) Model for multiscale disaggregation of spatial rainfall based on coupling meteorological and scaling descriptions. J Geophys Res 101(D21):26347–26361
Pérez-Vega A, Mas JF, Ligmann-Zielinska A (2012) Comparing two approaches to land use/cover change modelling and their implications for the assessment of biodiversity loss in a deciduous tropical forest. Environ Model Softw 29(1):11–23
Pontius RG Jr (2000) Quantification error versus location error in comparison of categorical maps. Photogramm Eng Remote Sens 66(8):1011–1016
Pontius RG Jr (2002) Statistical methods to partition effects of quantity and location during comparison of categorical maps at multiple resolutions. Photogramm Eng Remote Sens 68:1041–1049
Pontius RG Jr, Pacheco P (2004) Calibration and validation of a model of forest disturbance in the Western Ghats, India 1920–1990. GeoJournal 61: 325–334
Pontius RG Jr, Schneider LC (2001) Land-cover change model validation by an ROC method for the Ipswich watershed, Massachusetts, USA. Agric Ecosyst Environ 85:239–248
Pontius RG Jr, Huffaker D, Denman K (2004a) Useful techniques of validation for spatially explicit land-change models. Ecol Model 179(4):445–461
Pontius RG Jr, Shusas E, McEachern M (2004b) Detecting important categorical land changes while accounting for persistence. Agric Ecosyst Environ 101:251–268
Pontius RG Jr, Malanson J (2005) Comparison of the structure and accuracy of two land change models. Int J Geogr Inf Sci 19:243–265
Pontius RG Jr, Cheuk ML (2006) A generalized cross-tabulation matrix to compare soft-classified maps at multiple resolutions. Int J Geogr Inf Sci 20(1):1–30. doi:10.1080/13658810500391024
Pontius RG Jr, Lippitt CD (2006) Can error explain map differences over time? Cartogr Geogr Inf Sci 33(2):159–171
Pontius RG Jr, Verluis AJ, Malizia NR (2006) Visualizing certainty of extrapolations from models of land change. Landsc Ecol 21(7):1151–1166
Pontius RG Jr, Walker RT, Yao-Kumar R, Arima E, Aldrich S, Caldas M, Vergara D (2007) Accuracy assessment for a simulation model of Amazonian deforestation. Ann Assoc Am Geogr 97:677–695
Pontius Jr RG, LI X (2010) Land transition estimates from erroneous maps. J Land Use Sci 5(1):31–44
Pontius RG Jr, Petrova SH (2010) Assessing a predictive model on land change using uncertain data. Environ Model Softw 25:299–309
Pontius RG Jr, Boersma W, Castella JC, Clarke K, de Nijs T, Dietzel C, Duan Z, Fotsing E, Goldstein N, Kok K, Koomen E, Lippitt CD, McConnell W, Sood AM, Pijankowski B, Pidhadia S, Sweeney S, Trung TN, Veldkamp AT, Verburg PH (2008) Comparing the input, output, and validation maps for several models of land change. Anna Region Sci 42(1):11–27
Pontius RG Jr, Millones M (2011) Death to kappa and to some of my previous work: a better alternative. Int J Remote Sens 32(15):4407–4429
Pontius RG Jr, Gao Y, Giner NM, Kohyama T, Osaki M, Hirose K (2013) Design and interpretation of intensity analysis illustrated by land change in Central Kalimantan, Indonesia. Land 2(3):351–369. doi:10.3390/land2030351
Pontius Jr RG, Si K (2013) The total operating characteristic to measure diagnostic ability for multiple thresholds. Int J Geogr Inf Sci
Rodrigues HO, Soares Filho BS, de Souza Costa WL (2007) Dinamica EGO, uma plataforma para modelagem de sistemas ambientais. Anais XIII Simposio Brasileiro de Sensoriamento Remoto, INPE 3089–3096
Runfola DM, Pontius RG Jr (2013) Measuring the temporal instability of land change using the flow matrix. Int J Geogr Inf Sci 26(9):1696–1716
Rykiel EJJ (1996) Testing ecological models: the meaning of validation. Ecol Model 90:229–244
Sargent RG (2009) Verification and validation of simulation models. In: Rosetti MDD, Hill RR, Johansson B, Dunkin A, Ingalls RG (eds) Proceedings of the 2009 winter simulation conference. IEEE, Syracuse, NY, pp 162–176
Schneider LC, Pontius RG Jr (2001) Modeling land-use change in the Ipswich watershed, Massachusetts, USA. Agric Ecosyst Environ 85:83–94
Shirley LJ, Battaglia LL (2008) Projecting fine resolution land cover dynamics for a rapidly changing terrestrial-aquatic transition in Terrebone basin, Louisiana, USA. J Coastal Res 246:1545–1554
Silva TS, Tagliani PRA (2012) Environmental planning in the medium littoral of the Rio Grande do Sul coastal plain—Southern Brazil: elements for coastal management. Ocean Coast Manag 59:20–30
Soares-Filho BS, Pennachin CL, Cerqueira G (2002) DINAMICA—a stochastic cellular automata model designed to simulate the landscape dynamics in an Amazonian colonization frontier. Ecol Model 154(3):217–235
Sohl TL et al (2012) Spatially explicit land-use and land-cover scenarios for the Great Plains of the United States. Agric Ecosyst Environ 153:1–15. doi:10.1016/j.agee.2012.02.019
Steiner F, McSherry L, Cohen J (2000) Land suitability analysis for the upper Gila River watershed. Landsc Urban Plann 50(4):199–214
Syphard AD, Clarke KC, Franklin J, Regan HM, Mcginnis M (2011) Forecasts of habitat loss and fragmentation due to urban growth are sensitive to source of input data. J Environ Manage 92:1882–1893
Takada T, Miyamoto A, Hasegawa SF (2010) Derivation of a yearly transition probability matrix for land-use dynamics and its applications. Landsc Ecol 25(4):561–572
Torrens PM (2011) Calibrating and validating cellular automata models of urbanization. In: Yang X (ed) Urban remote sensing. Monitoring, synthesis and modelling in the urban environment. Wiley, New York, pp 335–345
Turner MG, Constanza R, Sklar FH (1989) Methods to evaluate the performance of spatial simulation models. Ecol Model 48:1–18
Villa N, Paegelow M, Camacho Olmedo MT, Cornez L, Ferraty F, Ferré L, Sarda P (2006) Various approaches for predicting land cover in Mediterranean mountains. Commun Stat 36(1):73–86
Visser H, de Nijs T (2006) The map comparison kit. Environ Model Softw 21:346–358
van Vliet J, Bregt AK, Brown DG, van Delden H, Heckbert S, Verburg PH (2016) A review of current calibration and validation practices in land change modelling. EMS 82:174–182
Wang J, Mountrakis G (2011) Developing a multi-network urbanization model: A case study of urban growth in Denver, Colorado. Int J Geogr Inf Sci 25(2):229–253. doi:10.1080/13658810903473213
White R, Uljee I, Engelen G (2012) Integrated modelling of population, employment, and land use change with a multiple activity based variable grid cellular automaton. Int J GIS 26:1251–1280
White R, Engelen G, Uljee I (1997) The use of constrained cellular automata for high resolution modelling of urban land use dynamics. Environ Plan 24:323–343
Yu J, Chen Y, Wu J, Khan S (2011) Cellular automata-based spatial multi-criteria land suitability simulation for irrigated agriculture. Int J Geogr Inf Sci 25(1):131–148. doi:10.1080/13658811003785571
Krüger C and Lakes T (2015) Revealing Uncertainties in Land Change Modeling using probabilities. Transactions in GIS, v20-4, 526-546
Acknowledgements
This work was supported by the BIA2013-43462-P project funded by the Spanish Ministry of Economy and Competitiveness and by the Regional European Fund FEDER. This study was also supported by the Consejo Nacional de Ciencia y Tecnología (CONACYT) and the Secretaría de Educación Pública through the project ¿Puede la modelación espacial ayudarnos a entender los procesos de cambio de cobertura/uso del suelo y de degradación ambiental? Fondos SEP-CONACyT 178816.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this chapter
Cite this chapter
Paegelow, M., Camacho Olmedo, M.T., Mas, J.F. (2018). Techniques for the Validation of LUCC Modeling Outputs. In: Camacho Olmedo, M., Paegelow, M., Mas, JF., Escobar, F. (eds) Geomatic Approaches for Modeling Land Change Scenarios. Lecture Notes in Geoinformation and Cartography. Springer, Cham. https://doi.org/10.1007/978-3-319-60801-3_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-60801-3_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-60800-6
Online ISBN: 978-3-319-60801-3
eBook Packages: Earth and Environmental ScienceEarth and Environmental Science (R0)