Multi-Source Data Processing Middleware for Land Monitoring within a Web-Based Spatial Data Infrastructure for Siberia
"> Figure 1
<p>MODIS sinusoidal tiling system [<a href="#B31-ijgi-02-00553" class="html-bibr">31</a>].</p> "> Figure 2
<p>Original and new CSV (with shortened header information) text file format for a single meteorological measurement.</p> "> Figure 3
<p>System overview for the framework developed within the Siberian Earth System Science Cluster (SIB-ESS-C). OGC, Open Geospatial Consortium; CSW, Catalogue Service for Web; WMS, Web Map Service; WFS, Web Feature Service; WCS, Web Coverage Service; SOS, Sensor Observation Service; WPS, Web Processing Service.</p> "> Figure 4
<p>Processing chain for the integration of MODIS products.</p> "> Figure 5
<p>Processing chain for the integration of Global Surface Summary of the Day (GSOD) and Integrated Surface Database (ISD) data.</p> "> Figure 6
<p>MapServer configuration for tile-index and raster layer.</p> "> Figure 7
<p>MapCache configuration XML: source and tileset layer properties (e.g., for MOD13C1 normalized vegetation difference index (NDVI)).</p> "> Figure 8
<p>Screenshot of the SIB-ESS-C web portal (<a href="http://www.sibessc.uni-jena.de" target="_blank">http://www.sibessc.uni-jena.de</a>).</p> "> Figure 9
<p>Combination of MODIS NDVI and MODIS burned area datasets (<b>Left</b>: previous-fire NDVI; <b>Middle</b>: post-fire NDVI, <b>Right</b>: burned areas) near Yakutsk, Russia.</p> "> Figure 10
<p>Excerpt of a logfile of on-demand processing (processing of a single time step).</p> "> Figure 11
<p>Forest Dragon 2 Geoportal (<b>Left</b>) shows the processed datasets from the SIB-ESS-C (<b>Right</b>).</p> "> Figure 12
<p>HTML preview of the retrieved data from Integrated Surface Database of the selected station, Ust-Maja (near Yakutsk), for the 20 July 2012.</p> ">
Abstract
:1. Introduction
- establishing a multi-source data processing middleware for land observations,
- implementing additional and individual processing steps for integrated data,
- providing standard-compliant visualization, access and download services for time-series data,
- and fostering near real-time monitoring of land processes.
2. Data Sources
Data | Provider | Available Time Ranges |
---|---|---|
Moderate Resolution Imaging Spectroradiometer | NASA | 2000–present |
Global Surface Summary of the Day | NOAA | 1929–present (depending on station) |
Integrated Surface Database | NOAA | 1929–present (depending on station) |
2.1. Moderate Resolution Imaging Spectroradiometer (MODIS)
Science Discipline | Archive Center | Data Access |
---|---|---|
Land | Land Processes Distributed Active Archive Center (LPDAAC) | http://e4ftl01.cr.usgs.gov/ [32] |
Cryosphere | National Snow & Ice Data Center (NSIDC) | ftp://n4ftl01u.ecs.nasa.gov/SAN/ [33] |
2.2. Global Surface Summary of the Day (GSOD)
Parameter | Original Unit | Converted SI-Based Unit |
---|---|---|
Mean temperature | Degrees Fahrenheit | Degrees Celsius |
Mean dew point | Degrees Fahrenheit | Degrees Celsius |
Mean sea-level pressure | Millibar | Pascal |
Mean station pressure | Millibar | Pascal |
Mean visibility | Miles | Kilometer |
Mean wind speed | Knots | Meter per second |
Max sustained wind speed | Knots | Meter per second |
Max wind gust | Knots | Meter per second |
Max temperature | Degrees Fahrenheit | Degrees Celsius |
Min temperature | Degrees Fahrenheit | Degrees Celsius |
Precipitation amount | Inches | Millimeters |
Snow depth | Inches | Millimeters |
2.3. Integrated Surface Database (ISD)
3. Framework for the Multi-Source Data Processing Middleware
Software | Version | Function | Source |
---|---|---|---|
PostgreSQL | 9.1 | Database | postgresql.org |
PostGIS | 2.0 | Spatial extension for PostgreSQL for raster and vector data | postgis.refractions.net |
Wget | 1.13.4 | FTP data retrieval | gnu.org/software/wget |
Python | 2.7 | Data integration script | python.org |
GDAL | 1.9 | HDF to GeoTIFF, data clipping | gdal.org |
R | 2.14.1 | Time-series plotting | r-project.org |
MapServer | 6.0.3 | Data access services | mapserver.org |
MapCache | 1.0 | Caching WMS services | mapserver.org |
istSOS | 2.0 | OGC SOS services | istgeo.ist.supsi.ch/software/istsos |
PyWPS | 3.2.1 | Data processing services | pywps.wald.intevation.org |
pycsw | 1.4.0 | Metadata services | pycsw.org |
Apache | 2.2.22 | Web server | httpd.apache.org |
Drupal CMS | 7 | Web content management system | drupal.org |
jQuery | 1.8 | Frontend JavaScript library | jquery.org |
OpenLayers | 1.12 | Frontend web mapping library | openlayers.org |
3.1. Data Integration
3.1.1. Integration and Processing of MODIS Products
- data server and directory where files are stored (e.g., FTP)
- raster type of product (Swath, Tile, CMG)
- whether 5 min swaths or tiles are needed (if raster type is equal to swath/tile)
- dataset names to be extracted
- no-data and scale information (if necessary for processing)
Column Name | dataset | date | location | geom | raster |
---|---|---|---|---|---|
Column Type | String | Date | String | MultiPolygon | Raster |
3.1.2. Integration and Processing of the GSOD Dataset
Gid | USAF | WBAN | Call (ICAO) | Country | Fips | US state | Station name |
Lat | Lon | Elevation | Date begin | Date end | Geometry | Location | File identifier |
3.1.3. Integration and Processing of the ISD Dataset
3.2. Data Provision
3.2.1. MODIS Data Visualization and Download Services
- dataset (layer) name
- time extend (start, end, interval)
- default time
- time positions
- styling and legend information
3.2.2. Climate Data Services
3.2.3. Metadata Services
General Metadata | |
---|---|
File Identifier | MODIS_MOD11_C3_LST_Day_Series |
Title | Monthly Daytime Land Surface Temperature from MODIS Terra |
Abstract | Time-series of monthly Terra MODIS daytime land surface temperature in Kelvin at 0.05 degrees spatial resolution. To retrieve actual values in Kelvin, a scale factor of 0.02 has to be applied. The unscaled no-data value is encoded as 0. Original MODIS data retrieved from the Land Processes Distributed Active Archive Center (http://e4ftl01.cr.usgs.gov/MOLT/) |
Keywords | MODIS, Terra, Siberia, Temperature, Global, Monthly, Series, Daytime |
Lineage | MODIS HDF Level 2 product was converted to GeoTIFF with gdal_translate (Version 1.9) |
Data Information | |
Description | Land Surface Temperature |
Data Type | RASTER |
Coverage Content Type | Physical Measurement |
SRS | EPSG:4326 |
BBOX | 57.1301270 81.2734985 179.8292847 42.2901001 |
Columns | 2,454 |
Rows | 780 |
Resolution | 0.05 |
Scale Factor | 0.02 |
No Data Value | 0 |
Time Begin | 2000-03-01 |
Time End | 2012-09-01 |
Time Interval | P1M |
Dates | 2000-03-01, 2000-04-01, 2000-05-01,…, 2012-08-01, 2012-09-01 |
Services | |
WMS URL | http://artemis.geogr.uni-jena.de/sibessc/modis |
WMS Protocol | WebMapService:1.3.0:HTTP |
WMS Description | MODIS Terra LST Day Monthly |
WMS Name | mod11c3_lst_day |
WCS URL | http://artemis.geogr.uni-jena.de/sibessc/modis |
WCS Protocol | WebCoverageService:1.1.0:HTTP |
WCS Description | MODIS Terra LST Day Monthly |
WCS Name | mod11c3_lst_day |
4. SIB-ESS-C Web Portal as Client for Middleware Services
5. Applications for the Multi-Source Data Processing Middleware
Short name | Platform | Resolution | Temporal coverage | Data period |
---|---|---|---|---|
Snow Cover | ||||
MOD10CM | Terra/Aqua | 0.05° | 2000/2002–present | Monthly |
MOD10C2 | Terra/Aqua | 0.05° | 2000/2002–present | 8-day |
Land Surface Temperature | ||||
MOD11C3 | Terra/Aqua | 0.05° | 2000/2002–present | Monthly |
MOD11A1 | Terra | 1,000 m | 2010–present | Daily |
Vegetation Indices (NDVI, EVI) | ||||
MOD13C2 | Terra/Aqua | 0.05° | 2000/2002–present | Monthly |
MOD13C1 | Terra/Aqua | 0.05° | 2000/2002–present | 16-day |
MOD13A1 | Terra | 500 m | 2010–present | 16-day |
Burned Area | ||||
MCD45A1 | Terra + Aqua | 500 m | 2000–present | Monthly |
5.1. Automatic Data Integration
5.2. On-Demand Data Integration
6. Conclusion and Outlook
Acknowledgments
References
- Townshend, J.R.; Brady, M.A.; Csiszar, I.A.; Goldammer, J.G.; Justice, C.O.; Skole, D.L. A Revised Strategy for GOFC-GOLD; GOFC-GOLD Report No. 24; Food and Agriculture Organization of the United Nations: Rome, Italy, 2006. [Google Scholar]
- Groisman, P.Y.; Bartalev, S.A. Northern Eurasia earth science partnership initiative (NEESPI), science plan overview. Global Planet. Change 2007, 56, 215–234. [Google Scholar]
- Lentile, L.B.; Holden, Z.A.; Smith, A.M.S.; Falkowski, M.J.; Hudak, A.T.; Morgan, P.; Lewis, S.A.; Gessler, P.E.; Benson, N.C. Remote sensing techniques to assess active fire characteristics and post-fire effects. Int. J. Wildland Fire 2006, 15, 319–345. [Google Scholar] [CrossRef]
- Franklin, S.E. Remote Sensing for Sustainable Forest Management; Lewis Publishers: Boca Raton, FL, USA, 2001. [Google Scholar]
- Asner, G.P.; Alencar, A. Drought impacts on the Amazon forest: The remote sensing perspective. New Phytol. 2010, 187, 569–578. [Google Scholar] [CrossRef]
- Rogan, J.; Chen, D. Remote sensing technology for mapping and monitoring land-cover and land-use change. Prog. Plann. 2004, 61, 301–325. [Google Scholar] [CrossRef]
- Justice, C.O.; Vermote, E.; Townshend, J.R.G.; Defries, R.; Roy, D.P.; Hall, D.K.; Salomonson, V.V.; Privette, J.L.; Riggs, G.; Strahler, A.; et al. The Moderate Resolution Imaging Spectroradiometer (MODIS): Land remote sensing for global change research. IEEE Trans. Geosci. Remote Sens. 1998, 36, 1228–1249. [Google Scholar] [CrossRef]
- Giri, C.P. Remote Sensing of Land Use and Land Cover: Principles and Applications; CRC Press: Boca Raton, FL, USA, 2012. [Google Scholar]
- Acker, J.G.; Leptoukh, G. Online analysis enhances use of NASA earth science data. Eos Trans. Amer. Geophys. Union 2007, 88. [Google Scholar] [CrossRef]
- Jones, K. An introduction to data warehousing: What are the implications for network? Int. J. Network Manage. 1998, 8, 42–56. [Google Scholar]
- Wang, J.; Li, C. Research on the Framework of Spatio-Temporal Data Warehouse. In Proceedings of the 24th International Cartographic Conference, Santiago de Chile, Chile, 15–21 November 2009.
- Bédard, Y.; Han, J. Fundamentals of Spatial Data Warehousing for Geographic Knowledge Discovery. In Geographic Data Mining and Knowledge Discovery; Miller, H.J., Han, J., Eds.; CRC Press: Boca Raton, FL, USA, 2009; pp. 45–68. [Google Scholar]
- Di, L. GeoBrain—A Web Services Based Geospatial Knowledge Building System. In Proceedings of NASA’s Earth Science Technology Conference, Palo Alto, CA, USA, 22–24 June 2004.
- Jiang, J.; Yang, C.; Ren, Y.; Zhu, X. A Middleware-Based Approach to Multi-Source Spatial Data. In Proceedings of 18th International Conference on Geoinformatics, Beijing, China, 18–20 June 2010; pp. 1–4.
- Tang, J.; Ren, Y.; Yang, C.; Shen, L.; Jiang, J. A WebGIS for Sharing and Integration of Multi-Source Heterogeneous Spatial Data. In Proceedings of 2011 IEEE International Geoscience and Remote Sensing Symposium, Vancouver, Canada, 24–29 July 2011; pp. 2943–2946.
- Mohammadi, H.; Rajabifard, A.; Williamson, I.P. Enabling Spatial Data Sharing through Multi-Source Spatial Data Integration. In Proceedings of GSID 11 World Conference, Rotterdam, The Netherlands, 15–19 June 2009.
- Nativi, S.; Craglia, M.; Pearlman, J. The brokering approach for multidisciplinary interoperability: A position paper. Int. J. Spat. Data Infrastr. Res. 2012, 7, 1–15. [Google Scholar]
- Nativi, S.; Khalsa, S.; Domenico, B.; Craglia, M.; Pearlman, J.; Mazzetti, P.; Rew, R. The Brokering Approach for Earth Science Cyberinfrastructure. Available online: http://semanticommunity.info/@api/deki/files/13798/=010_Domenico.pdf (accessed on 1 March 2013).
- Hüttich, C.; Schmullius, C.; Thiel, C.; Pathe, C.; Bartalev, S.; Emelyanov, K.; Korets, M.; Shvidenko, A.; Schepaschenko, D. ZAPÁS: Assessment and Monitoring of Forest Resources in the Framework of EU-Russia Space Dialogue. In Let’s Embrace Space Vol. 2; Schulte-Braucks, R., Breger, P., Bischoff, H., Borowiecka, S., Sadiq, S., Eds.; European Union: Brussels, Belgium, 2012; Volume II, pp. 164–171. [Google Scholar]
- Cechini, M.F.; Mitchell, A.; Pilone, D. NASA Reverb: Standards-Driven Earth Science Data and Service Discovery. In AGU Fall Meeting Abstracts; American Geophysical Union: San Francisco, CA, USA, 2011; Volume 1, p. 1406. [Google Scholar]
- Aundeen, J.L.F.; Anengieter, R.L.K.; Uswell, M.D.B. U.S. geological survey spatial data access. J. Geospatial Eng. 2002, 4, 145–152. [Google Scholar]
- National Climatic Data Center Climate.gov: Data & Services. Available online: http://www.climate.gov/#dataServices (accessed on 6 March 2013).
- De Freitas, R.M.; Arai, E.; Adami, M.; Ferreira, A.S.; Sato, F.Y.; Shimabukuro, Y.E.; Rosa, R.R.; Anderson, L.O.; Friedrich, B.; Rudorff, T. Virtual laboratory of remote sensing time series: Visualization of MODIS EVI2 data set over South America. J. Comput. Interdiscipl. Sci. 2011, 2, 57–68. [Google Scholar]
- Group on Earth Observations GEO. What is GEOSS? Available online: http://www.earthobservations.org/geoss.shtml (accessed on 1 March 2013).
- Christian, E.J. GEOSS architecture principles and the GEOSS clearinghouse. IEEE Syst. J. 2008, 2, 333–337. [Google Scholar] [CrossRef]
- Group on Earth Observations. The Global Earth Observation System of Systems (GEOSS) 10-Year Implementation Plan. Available online: http://www.earthobservations.org/documents/10-Year Implementation Plan.pdf (accessed on 1 March 2013).
- World Meteorological Organization Resolution 40 (Cg-XII). Available online: http://www.wmo.int/pages/about/ Resolution40.html (accessed on 2 March 2013).
- Justice, C.; Townshend, J.R.; Vermote, E.; Masuoka, E.; Wolfe, R.; Saleous, N.; Roy, D.; Morisette, J. An overview of MODIS Land data processing and product status. Remote Sens. Environ. 2002, 83, 3–15. [Google Scholar] [CrossRef]
- Xiong, X.; Chiang, K.; Sun, J.; Barnes, W.L.; Guenther, B.; Salomonson, V.V. NASA EOS Terra and Aqua MODIS on-orbit performance. Adv. Space Res. 2009, 43, 413–422. [Google Scholar] [CrossRef]
- Wolfe, R.E.; Roy, D.P.; Vermote, E. MODIS land data storage, gridding, and compositing methodology: Level 2 grid. IEEE Trans. Geosci. Remote Sens. 1998, 36, 1324–1338. [Google Scholar] [CrossRef]
- NASA Land Processes Distributed Active Archive Center (LP DAAC) MODIS Overview. Available online: https://lpdaac.usgs.gov/products/modis_overview (accessed on 1 March 2013).
- NASA Land Processes Distributed Active Archive Center (LP DAAC) Data PoolI. Available online: https://lpdaac.usgs.gov/get_data/data_pool (accessed on 1 March 2013).
- National Snow and Ice Data Center (NSIDCs) Order Data. Available online: http://nsidc.org/data/modis/order_data.html (accessed on 1 March 2013).
- Lott, N. Global Surface Summary of Day. Available online: http://www.ncdc.noaa.gov/cgi-bin/res40.pl? page=gsod.html (accessed on 1 March 2013).
- Lott, N.; Vose, R.; Del Greco, S.A.; Ross, T.F.; Worley, S.; Comeaux, J. The Integrated Surface Database: Partnerships and Progress. In Proceedings of the 24th Conference in IIPS, New Orleans, LA, USA, 20–24 January 2008.
- Nebert, D.; Whiteside, A.; Vretanos, P.P. OpenGIS Catalogue Services Specification; Open Geospatial Consortium Inc.: Wayland, USA, 2007. [Google Scholar]
- De Beaujardiere, J. OpenGIS Web Map Server Implementation Specification; Open Geospatial Consortium Inc.: Wayland, MA, USA, 2006. [Google Scholar]
- Vretanos, P.A. Web Feature Service Implementation Specification; Open Geospatial Consortium Inc.: Wayland, MA, USA, 2005. [Google Scholar]
- Baumann, P. OGC WCS 2.0 Interface Standard—Core; Open Geospatial Consortium Inc.: Wayland, MA, USA, 2010. [Google Scholar]
- Bröring, A.; Stasch, C.; Echterhoff, J. OGC Sensor Observation Service Interface Standard; Open Geospatial Consortium Inc.: Wayland, MA, USA, 2012. [Google Scholar]
- Schut, P. OpenGIS Web Processing Service; Open Geospatial Consortium Inc.: Wayland, MA, USA, 2007. [Google Scholar]
- Neteler, M. MODIS Sinusoidal GIS SHAPE Files. Available online: http://gis.cri.fmach.it/modis-sinusoidal-gis-files/ (accessed on 4 March 2013).
- Warmerdam, F. The Geospatial Data Abstraction Library. In Open Source Approaches in Spatial Data Handling; Hall, G.B., Leahy, M., Eds.; Springer: Berlin/Heidelberg, Germany, 2008; Volume 2, pp. 87–104. [Google Scholar]
- Obe, R.O.; Hsu, L.S. PostGIS in Action, 1st ed; Manning Publications Co.: Stamford, CT, USA, 2011. [Google Scholar]
- Cannata, M.; Antonovic, M. istSOS: Investigation of the Sensor Observation Service. In Proceedings of 1st International Workshop on Pervasive Web Mapping, Geo-processing and Services, Como, Italy, 26–27 August 2010.
- Cannata, M.; Molinari, M.; Pozzoni, M. istSOS Sensor Observation Management System: A Real Case Application of Hydro-Meteorological Data for Flood Protection. In Proceedings of International Workshop of “The Role of Geomatics in Hydrogeological Risk”, Padua, Italy, 27–28 February 2013.
- Cannata, M.; Antonovic, M. Tutorial: Using istSOS. Available online: http://istgeo.ist.supsi.ch/software/istsos/ tutorial.html (accessed on 8 March 2013).
- ISO 19115:2003 Geographic Information—Metadata; International Organization for Standardization: Geneva, Switzerland, 2003.
- Maduako, I. PostGIS-Based Heterogeneous Sensor Database Framework for the Sensor Observation Service. In Proceedings of FOSS4G Prague 2012, Prague, Czech Republic, 21–25 May 2012; Čepek, A., Ed.; Czech Technical University in Prague: Prague, Czech Republic, 2012; pp. 55–71. [Google Scholar]
- Misev, D.; Baumann, P.; Seib, J. Towards large-scale meteorological data services: A case study. Datenbank-Spektrum 2012, 12, 183–192. [Google Scholar] [CrossRef]
- Gallaher, D.; Grant, G. Data Rods: High Speed, Time-Series Analysis of Massive Cryospheric Data Sets Using Pure Object Databases. In Proceedings of 2012 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Munich, Germany, 22–27 July 2012; pp. 5297–5300.
- Baumann, P. Web Coverage Processing Service (WCPS) Language Interface Standard. OGC 08-068r2; Open Geospatial Consortium Inc.: Wayland, MA, USA, 2009. [Google Scholar]
- Baumann, P. The OGC web coverage processing service (WCPS) standard. GeoInformatica 2009, 14, 447–479. [Google Scholar] [CrossRef]
- Smets, B.; Lacaze, R.; Pacholczyk, P.; Makmahra, H.; Baret, F.; Camacho, F.; Claes, P. Towards an Operational Delivery of a Time-Series of Essential Climate Variables for Global Land Monitoring. In Proceedings of 31st EARSeL Symposium 2011; Halounová, L., Ed.; Czech Technical University in Prague: Prague, Czech Republic, 2011; pp. 205–212. [Google Scholar]
© 2013 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).
Share and Cite
Eberle, J.; Clausnitzer, S.; Hüttich, C.; Schmullius, C. Multi-Source Data Processing Middleware for Land Monitoring within a Web-Based Spatial Data Infrastructure for Siberia. ISPRS Int. J. Geo-Inf. 2013, 2, 553-576. https://doi.org/10.3390/ijgi2030553
Eberle J, Clausnitzer S, Hüttich C, Schmullius C. Multi-Source Data Processing Middleware for Land Monitoring within a Web-Based Spatial Data Infrastructure for Siberia. ISPRS International Journal of Geo-Information. 2013; 2(3):553-576. https://doi.org/10.3390/ijgi2030553
Chicago/Turabian StyleEberle, Jonas, Siegfried Clausnitzer, Christian Hüttich, and Christiane Schmullius. 2013. "Multi-Source Data Processing Middleware for Land Monitoring within a Web-Based Spatial Data Infrastructure for Siberia" ISPRS International Journal of Geo-Information 2, no. 3: 553-576. https://doi.org/10.3390/ijgi2030553
APA StyleEberle, J., Clausnitzer, S., Hüttich, C., & Schmullius, C. (2013). Multi-Source Data Processing Middleware for Land Monitoring within a Web-Based Spatial Data Infrastructure for Siberia. ISPRS International Journal of Geo-Information, 2(3), 553-576. https://doi.org/10.3390/ijgi2030553