際際滷

際際滷Share a Scribd company logo

Ifgi presentation

N
noho

This document discusses Landcare Research's implementation of a greenhouse gas emissions model using the OGC Web Processing Service (WPS) standard in a science collaboration environment. It describes the inputs and process definition for the emissions algorithm WPS process. Issues discussed include efficiently fetching raster data, value attribute table implementations, and challenges of OGC standards being bulky while also needing to ensure interoperability.

1 of 19
The use of WPS in a Science Collaboration Environment implementation of a Greenhouse Gas Emissions model
Topics Introduce Landcare Research Our Vision Architecture Model implementation Closing thoughts Previous slide: Cloud Gate, Chicago. Photo: Niels Hoffmann
Manaaki Whenua - Manaaki Tangata care for the land - care for the people http://www.landcareresearch.co.nz Sustaining biodiversity & restoration Sustaining land environments Sustaining business & living Climate change Maori sustainable futures Weeds, pests and diseases Capability and collaboration Landcare Research Manaaki Whenua Key outcomes Cross-cutting outcomes Underpinning strengths
LRIS: Land Resource Information System 1978-2008+ NZLRI NZLRI NSD Soil DB S-map Soil DB : National Coverage of Environmental Data at 1:50:000 to 15m resolution : Leading the Implementation of geospatial web-delivery in NZ S-Map soil NZLRI ECOSAT veg Curvature horiz Curvature perp Curvature slope S-map Landform Elements S-map siblings S-map pedo-func S-map variability Height Aspect Slope FSL:Phys FSL:Chem FSL:Wet FSL:Env NZEEM Soil loss tons/ha/yr ECOSAT fPAR ECOSAT Woody ECOSAT Indig. Forest PinRadGrow FertReq StockCCap LUNZ & LENZ Rock Slope Soil Erosion Veg LUC
Observations Data and Computational pressure: NOW 25m national data density NEAR FUTURE sub 5m national data density with significant peri-urban sub 1m LIDAR data density Modelling environment NOW essentially batch oriented & 2.5D DESIRED interactive 4D, with real-time visualisation feedback Managed data NOW preserve the data, memorise the model DESIRED keep the model for on-demand re-use
The Vision: SCENZ-Grid SCE NZ-Grid proposes that we can: Do science research on-line together Share each others data not duplicate it Collaboratively develop & use shared models / workflows Use shared compute resources Connect researchers directly to consumers : policy / managers / educators / public
Phase 1: Regolith Portlet REANNZ funded project (2007-2009): GNS and Landcare Research QMAP WMS NZFSL WMS Lookup WS Portal User WF Engine User Interface View Portlet WebService WorkFlow Engine Model Kepler Mono / Java class Web Services Data: WMS Business Logic: SOAP
Phase 2: Platform FRST Backbone funding and BeSTGRID integration (2009-)
'SDI' Find Koordinates Visualize Geoserver / OL Process 52N WPS
WPS Geoserver Globus / Grisu ~1.16 TFLOPS Air cooled Gb Ethernet nterconnects 4.2kW power 104 Intel Xeon cores 2.8GHz each 386GB RAM 2.6TB storage
WPS Extensions: Computation - Collaboration WPS G Current: Unicore, Gridgain Planning to implement Grisu => waiting for release v3 WPS T Started porting to Apache ODE 80% complete, need to refactor the execute part to use the new(er) architecture
LCR Algorithm Repository LCR (Landcare Research) Specific algorithms Raster implementations: Emissions Modeling Landcare repository Sextante repository
Modularity Algorithms: fully modular
Parsers: problem with xml parser Diff requestHandler (left lcr branch, right trunk)
Emissions Algorithm - inputs Landuse (dairy, sheep/beef, deer) Grid => WCS Geotiff Stocking rate (animals/ha) Table => xml Sheep/Beef Ratio (dependent per district) Grid => WCS Geotiff Methane + Nitrous Oxide => CO2eq/animal/year Table => xml
<ProcessDescription wps:processVersion=&quot;2&quot; storeSupported=&quot;true&quot; statusSupported=&quot;false&quot;> <ows:Identifier>org.n52.wps.server.lcr.algorithm.EmissionsAlgorithm</ows:Identifier> <DataInputs> <ows:Identifier> landuse </ows:Identifier> <MimeType>image/tiff</MimeType> <ows:Identifier> districts </ows:Identifier> <MimeType>image/tiff</MimeType> <ows:Identifier> emissionslookup </ows:Identifier> <MimeType>text/XML</MimeType> <Schema>http://arwen/schemas/EmissionsTable.xsd</Schema> <ows:Identifier> stockingrate </ows:Identifier> <MimeType>text/XML</MimeType> <Schema>http://arwen/schemas/EmissionsTable.xsd</Schema> <ows:Identifier> districtlookup </ows:Identifier> <MimeType>text/XML</MimeType> <Schema> http://arwen/schemas/EmissionsTable.xsd </Schema> <ows:Identifier>result</ows:Identifier> <MimeType>image/tiff</MimeType> </ProcessDescription> Emissions Algorithm link Value Attribute Table for Landuse and Districts XML Key-Value pair implementation Key = raster value, Value = list of attributes
Current Issues Fetching WCS coverages efficiently Crop to smallest extent
No checking for CRS validity Value Attribute Table implementation Custom XML / GeoTiff 'linkup' GMLJP2 ??? Client with raster capabilities
Closing thoughts (1) OGC Standards => Big and Bulky WPS very flexible but 'heavyweight' Like to be small and nimble (rest, custom architecture) Selling point for OGC is Interoperability need good clients => again difficult because of flexibility of standards WPS 2.0 profiles ?

More Related Content

Ifgi presentation

  • 1. The use of WPS in a Science Collaboration Environment implementation of a Greenhouse Gas Emissions model
  • 2. Topics Introduce Landcare Research Our Vision Architecture Model implementation Closing thoughts Previous slide: Cloud Gate, Chicago. Photo: Niels Hoffmann
  • 3. Manaaki Whenua - Manaaki Tangata care for the land - care for the people http://www.landcareresearch.co.nz Sustaining biodiversity & restoration Sustaining land environments Sustaining business & living Climate change Maori sustainable futures Weeds, pests and diseases Capability and collaboration Landcare Research Manaaki Whenua Key outcomes Cross-cutting outcomes Underpinning strengths
  • 4. LRIS: Land Resource Information System 1978-2008+ NZLRI NZLRI NSD Soil DB S-map Soil DB : National Coverage of Environmental Data at 1:50:000 to 15m resolution : Leading the Implementation of geospatial web-delivery in NZ S-Map soil NZLRI ECOSAT veg Curvature horiz Curvature perp Curvature slope S-map Landform Elements S-map siblings S-map pedo-func S-map variability Height Aspect Slope FSL:Phys FSL:Chem FSL:Wet FSL:Env NZEEM Soil loss tons/ha/yr ECOSAT fPAR ECOSAT Woody ECOSAT Indig. Forest PinRadGrow FertReq StockCCap LUNZ & LENZ Rock Slope Soil Erosion Veg LUC
  • 5. Observations Data and Computational pressure: NOW 25m national data density NEAR FUTURE sub 5m national data density with significant peri-urban sub 1m LIDAR data density Modelling environment NOW essentially batch oriented & 2.5D DESIRED interactive 4D, with real-time visualisation feedback Managed data NOW preserve the data, memorise the model DESIRED keep the model for on-demand re-use
  • 6. The Vision: SCENZ-Grid SCE NZ-Grid proposes that we can: Do science research on-line together Share each others data not duplicate it Collaboratively develop & use shared models / workflows Use shared compute resources Connect researchers directly to consumers : policy / managers / educators / public
  • 7. Phase 1: Regolith Portlet REANNZ funded project (2007-2009): GNS and Landcare Research QMAP WMS NZFSL WMS Lookup WS Portal User WF Engine User Interface View Portlet WebService WorkFlow Engine Model Kepler Mono / Java class Web Services Data: WMS Business Logic: SOAP
  • 8. Phase 2: Platform FRST Backbone funding and BeSTGRID integration (2009-)
  • 9. 'SDI' Find Koordinates Visualize Geoserver / OL Process 52N WPS
  • 10. WPS Geoserver Globus / Grisu ~1.16 TFLOPS Air cooled Gb Ethernet nterconnects 4.2kW power 104 Intel Xeon cores 2.8GHz each 386GB RAM 2.6TB storage
  • 11. WPS Extensions: Computation - Collaboration WPS G Current: Unicore, Gridgain Planning to implement Grisu => waiting for release v3 WPS T Started porting to Apache ODE 80% complete, need to refactor the execute part to use the new(er) architecture
  • 12. LCR Algorithm Repository LCR (Landcare Research) Specific algorithms Raster implementations: Emissions Modeling Landcare repository Sextante repository
  • 14. Parsers: problem with xml parser Diff requestHandler (left lcr branch, right trunk)
  • 15. Emissions Algorithm - inputs Landuse (dairy, sheep/beef, deer) Grid => WCS Geotiff Stocking rate (animals/ha) Table => xml Sheep/Beef Ratio (dependent per district) Grid => WCS Geotiff Methane + Nitrous Oxide => CO2eq/animal/year Table => xml
  • 16. <ProcessDescription wps:processVersion=&quot;2&quot; storeSupported=&quot;true&quot; statusSupported=&quot;false&quot;> <ows:Identifier>org.n52.wps.server.lcr.algorithm.EmissionsAlgorithm</ows:Identifier> <DataInputs> <ows:Identifier> landuse </ows:Identifier> <MimeType>image/tiff</MimeType> <ows:Identifier> districts </ows:Identifier> <MimeType>image/tiff</MimeType> <ows:Identifier> emissionslookup </ows:Identifier> <MimeType>text/XML</MimeType> <Schema>http://arwen/schemas/EmissionsTable.xsd</Schema> <ows:Identifier> stockingrate </ows:Identifier> <MimeType>text/XML</MimeType> <Schema>http://arwen/schemas/EmissionsTable.xsd</Schema> <ows:Identifier> districtlookup </ows:Identifier> <MimeType>text/XML</MimeType> <Schema> http://arwen/schemas/EmissionsTable.xsd </Schema> <ows:Identifier>result</ows:Identifier> <MimeType>image/tiff</MimeType> </ProcessDescription> Emissions Algorithm link Value Attribute Table for Landuse and Districts XML Key-Value pair implementation Key = raster value, Value = list of attributes
  • 17. Current Issues Fetching WCS coverages efficiently Crop to smallest extent
  • 18. No checking for CRS validity Value Attribute Table implementation Custom XML / GeoTiff 'linkup' GMLJP2 ??? Client with raster capabilities
  • 19. Closing thoughts (1) OGC Standards => Big and Bulky WPS very flexible but 'heavyweight' Like to be small and nimble (rest, custom architecture) Selling point for OGC is Interoperability need good clients => again difficult because of flexibility of standards WPS 2.0 profiles ?
  • 20. Closing thoughts (2) WPS-G / Grid processing Scalability / high availability Easy to implement, but also possible via alternatives Parallelization new class of problems
  • 21. No easy solution for splitting (spatial) problems
  • 22. split/merge might still be the bottleneck
  • 23. Questions ? [email_address] Milford Sound, NZ. Photo: Niels Hoffmann