ºÝºÝߣ

ºÝºÝߣShare a Scribd company logo

Foss4G 2009 Scenz Grid

•Download as PPT, PDF•
•
N
noho

SCENZ-Grid proposes a science collaboration and computation environment to enable researchers to do science research online together by sharing data and computational resources without duplicating them. It would allow researchers to collaboratively develop and use shared models and workflows, and connect researchers directly to policymakers, managers, educators and the public. The environment would use spatial, computational and web services and technologies like Web Processing Services, 52North, GeoServer and OpenLayers to query and analyze data from multiple sources on distributed computational resources. It aims to establish relationships between different datasets from various organizations through similarity matrices and expertise to combine attributes from different classifications.

1 of 21
Download to read offline
SCENZ-Grid The implementation of a Science Collaboration and Computation Environment Niels Hoffmann Landcare Research
Manaaki Whenua / Landcare Research is New Zealand's foremost environmental research organisation. Our research focuses on three key areas: Sustaining and restoring biodiversity; Sustaining land environments; Sustaining business and living. Three themes relate to these three areas: Climate change mitigation and adaptation Maori sustainability Invasive species and disease impacts
Sustaining biodiversity & restoration Sustaining land environments Sustaining business & living Climate change Maori sustainable futures Weeds, pests and diseases Capability and collaboration Landcare Research Manaaki Whenua Key outcomes Cross-cutting outcomes Underpinning strengths
Data and Computational pressure NOW – 25m national data density NEAR FUTURE – sub 5m national data density / (peri-)urban sub 1m LIDAR data density Modelling environment NOW – essentially batch oriented & 2.5D DESIRED – interactive 4D, with real-time visualisation feedback Managed data NOW – preserve the data, memorise the model DESIRED – keep the model for on-demand re-use
science : collaboration : environment SCE NZ-Grid proposes that we can Do science research on-line together Share each other’s data – not duplicate it Collaboratively develop & use shared models / workflows Use shared compute resources Connect researchers directly to consumers : policy / managers / educators / public spatial : computation : engine
Dublin Core / RDF Repository Web Services Doc Img Geo Name Search Consume Query Tag Comment Create Workflow OAI-ORE SPARQL
A personal home page with links to relevant work for organising and retrieving a large variety of digital resources
Detailed view of a resource – in this case a modelling workflow, that can be edited by maybe changing the inputs or logic and then re-used
Databases Content (WMS, WFS, WCS) Functionality (WPS) Desktop Web
PostGIS / SqlServer2008 GeoServer GeoWebCache 52North Geoprocessing ArcGIS / Udig OpenLayers GeoExt
Why WPS ? Distributed Architecture Interoperability Modeling approach (as opposed to data centric outcome) (Grid-) Computing
Why 52North ? Build upon robust OS Libraries (JTS, Geotools, xmlBeans, Servlet API) Pluggable framework for algorithms Support for raster processing Support for Grid-Computing
Currently using Unicore Middleware Planning a migration to Globus Middleware to integrate with BeSTGRID Landcare repository Sextante repository
WPS Geoserver 104 Intel Xeon cores 2.8GHz each 386GB RAM 2.6TB storage ~1.16 TFLOPS Air cooled Gb Ethernet nterconnects 4.2kW power
Spatially query across datasets from multiple organisations. Qmap: Geology from GNS NZFSL: Soil Data from Landcare Research What kind of relationships exist between soil type and bedrock. What is the association between groundwater quality and soil. Investigate relationships between ecology, ground water and soil.
Regolith: Layer between Bedrock and Soil Erosion modelling Groundwater flow modelling WPS Algorithm to combine 2 datasets based on a similarity matrix User interface to enable experts to adapt the similarity matrix
Establish the ‘similarity’ of the datasets Different origin of attributes Different classification Similarity Matrix quantify where the classifications match Expert to decide similarity based on documentation
QMAP WMS NZFSL WMS Lookup WS Portal User WPS User Interface View Portlet WebService WorkFlow Engine WPS Web Services Data: WMS Business Logic: SOAP
Lookup WS WPS User
Ìý
Thank You, Any questions? [email_address] ºÝºÝߣ Credits: Chris McDowall, Aaron Hicks science : collaboration : environment spatial : computation : engine

More Related Content

Foss4G 2009 Scenz Grid

  • 1. SCENZ-Grid The implementation of a Science Collaboration and Computation Environment Niels Hoffmann Landcare Research
  • 2. Manaaki Whenua / Landcare Research is New Zealand's foremost environmental research organisation. Our research focuses on three key areas: Sustaining and restoring biodiversity; Sustaining land environments; Sustaining business and living. Three themes relate to these three areas: Climate change mitigation and adaptation Maori sustainability Invasive species and disease impacts
  • 3. Sustaining biodiversity & restoration Sustaining land environments Sustaining business & living Climate change Maori sustainable futures Weeds, pests and diseases Capability and collaboration Landcare Research Manaaki Whenua Key outcomes Cross-cutting outcomes Underpinning strengths
  • 4. Data and Computational pressure NOW – 25m national data density NEAR FUTURE – sub 5m national data density / (peri-)urban sub 1m LIDAR data density Modelling environment NOW – essentially batch oriented & 2.5D DESIRED – interactive 4D, with real-time visualisation feedback Managed data NOW – preserve the data, memorise the model DESIRED – keep the model for on-demand re-use
  • 5. science : collaboration : environment SCE NZ-Grid proposes that we can Do science research on-line together Share each other’s data – not duplicate it Collaboratively develop & use shared models / workflows Use shared compute resources Connect researchers directly to consumers : policy / managers / educators / public spatial : computation : engine
  • 6. Dublin Core / RDF Repository Web Services Doc Img Geo Name Search Consume Query Tag Comment Create Workflow OAI-ORE SPARQL
  • 7. A personal home page with links to relevant work for organising and retrieving a large variety of digital resources
  • 8. Detailed view of a resource – in this case a modelling workflow, that can be edited by maybe changing the inputs or logic and then re-used
  • 9. Databases Content (WMS, WFS, WCS) Functionality (WPS) Desktop Web
  • 10. PostGIS / SqlServer2008 GeoServer GeoWebCache 52North Geoprocessing ArcGIS / Udig OpenLayers GeoExt
  • 11. Why WPS ? Distributed Architecture Interoperability Modeling approach (as opposed to data centric outcome) (Grid-) Computing
  • 12. Why 52North ? Build upon robust OS Libraries (JTS, Geotools, xmlBeans, Servlet API) Pluggable framework for algorithms Support for raster processing Support for Grid-Computing
  • 13. Currently using Unicore Middleware Planning a migration to Globus Middleware to integrate with BeSTGRID Landcare repository Sextante repository
  • 14. WPS Geoserver 104 Intel Xeon cores 2.8GHz each 386GB RAM 2.6TB storage ~1.16 TFLOPS Air cooled Gb Ethernet nterconnects 4.2kW power
  • 15. Spatially query across datasets from multiple organisations. Qmap: Geology from GNS NZFSL: Soil Data from Landcare Research What kind of relationships exist between soil type and bedrock. What is the association between groundwater quality and soil. Investigate relationships between ecology, ground water and soil.
  • 16. Regolith: Layer between Bedrock and Soil Erosion modelling Groundwater flow modelling WPS Algorithm to combine 2 datasets based on a similarity matrix User interface to enable experts to adapt the similarity matrix
  • 17. Establish the ‘similarity’ of the datasets Different origin of attributes Different classification Similarity Matrix quantify where the classifications match Expert to decide similarity based on documentation
  • 18. QMAP WMS NZFSL WMS Lookup WS Portal User WPS User Interface View Portlet WebService WorkFlow Engine WPS Web Services Data: WMS Business Logic: SOAP
  • 20. Ìý
  • 21. Thank You, Any questions? [email_address] ºÝºÝߣ Credits: Chris McDowall, Aaron Hicks science : collaboration : environment spatial : computation : engine

Editor's Notes

  • #4: Manaaki Whenua / Landcare Research is New Zealand's foremost environmental research organisation. Manaaki Whenua = Care for the land We do Environmental Landscape modelling
  • #5: Trends Catchment modeling nutrient loss => hydrology model needs 1m dem 3d soil characterization => micron scale data
  • #6: To respond to the challenges posed by our Scientists, Informatics came up with the SCENZ-Grid vision Data and Computational pressure Modelling environment Managed data Two Pillars: Collaboration and Computation
  • #7: Collaboration Open architecture with a content repository at it’s heart, a cross cutting ‘catalog’ wider than just geospatial metadata or a content management system Spatial is ‘just a datatype’ in the database, likewise a dataset is just an object in the repository Standards: Open Archives Initiative - Object Reuse and Exchange SPARQL - S PARQL P rotocol a nd R DF Q uery L anguage Web Services in general
  • #8: MyExperiment Evernote Social Networking DSpace
  • #9: MyExperiment Taverna Kepler
  • #10: Where does it fit in the Open Geo Stack?
  • #11: Which components in the Geo Stack are we using?
  • #12: Extensions to core WPS: WPS-T WPS-G
  • #15: 1 x SGI Altix XE250 6 x SGI Altix XE320 Each XE320 is a pair of servers 1 + 6 x 2 = 13 nodes 2 x Intel quad core Xeons per node = 8 cores per node 13 x 8 = 104 cores
  • #16: Example Focused on Collaboration as much as compute power, the analysis in this example does not necessarily require a cluster
  • #17: What we’re talking about is the ‘regolith’ layer Solution has 2 components: Computation = WPS Algorithm Collaboration = Working on Similarity matrix
  • #18: Q-Map is Geological mapping done by Geologists with their background LRIS / NZFSL is Soil map, has an attribute for bedrock which is mapped by Soil Scientists What we want to know is how the mappings from those 2 communities match LRIS is a pure hierarchical Classification: Order>Group>Subgroup>Class QMAP is a multi-attribute Classification Sedimentary>clastic Sediment>mud>mud but also mudstone>mud We assigned weights to the attributes in the classification > the more exact the description the higher the weight. If you add the weights you get the ‘similarity value’ a number between 0 and 20