際際滷

際際滷Share a Scribd company logo
Cyberinfrastructure in Louisiana: From Black Holes to HurricanesGabrielle AllenDepartment of Computer ScienceCenter for Computation & TechnologyLouisiana State Universityhttp://www.cct.lsu.edu/~gallen
Complex Problem SolvingComputation as Third Pillar of Scientific InvestigationReal world     applicationsCommunitiesNo single group, university, or state can do these problemsMust integrate CS, Math, Bio, Sensors, Engineering, more...Data everywhereSupercomputers generate petabytes2
3    LIGO:  Laser Interferometric Gravitational Wave ObservatoryTies together theory, computation, and experimentGravitational Wave Astronomy
4Louisiana Coastal AreaRich dynamic environment for modeling: coupled models, multi-scale, realtime data (sensors, satellites)Models, Data, Grids for Hurricane forecastsEmergency preparedness Wetland reconstructionEcological studies and fish populationsOilspillbehaviourLevee designRescueHypoxia Dead ZoneAlgae blooms24/7/365 shipping forecasts
Hurricane Gustav
6Highly scalable algorithms (Petascale, exascale computing, heterogeneous architectures)Storage & networks: provenance, metadataVisualization (Large data, interactive, remote, AMR)Software engineering (code generation, verification & validation, interfaces, interoperability, etc)Workflows, Grid & distributed computingEducation Research Issues
PITAC Report Summary (June 2005): 7Computational science -- the use of advanced computing capabilities to understand and solve complex problems -- is critical to scientific leadership, economic competitiveness, and national security. It is one of the most important technical fields of the 21st century because it is essential to advances throughout society.Universities must significantly change organizational structures:  multidisciplinary & collaborative research are needed [for US] to remain competitive in global scienceComplex problems:  Innovations will occur at boundaries
Louisiana Response: Center for Computation & Technology (CCT)8State commitment for IT in 2002$25M/year for Vision 20/20, $9M to LSUUniversity commitment to build new programsOpportunity to build new world class program in interdisciplinary research & educationEd Seidel recruited to LSU, created & implemented vision for state-wide collaborationCenter for Computation & Technology (2003)Louisiana Optical Network Initiative (2005)CyberTools (2007), LONI Institute (2007)Multidisciplinary Hiring Initiatives (2007)
CCT Plan: Designing a Computational Science Initiative at LSUCyberinfrastructure: access to local/national compute, storage, network, visualization resources; support; end-to-end integration. Interdisciplinary research: university policies on joint appointments, university-wide curricula, appreciation of computational science outcomes (e.g. software, data)Computational research: faculty who develop or use computation as part of their research; undergraduate and graduate courses to train and educate students10CCT Strategic PlanMission: The CCT at LSU is an innovative and interdisciplinary research environment, advancing computational sciences, technologies, and the disciplines they touch. The CCT serves Louisiana through international collaboration, leading progress through revolutionary advancement in academia and industry.Strategic Plan:ResearchEducationServiceInfrastructureEconomic Developmenthttps://cct.lsu.edu/uploads/CCTStrategicPlan20062010.pdf
Growth of CCTSeptember 2003May 2008
CCT Focus Areas: Faculty LedAreas chosen to bring together interdisciplinary researchers towards common theme: Faculty, Research Staff, Postdocs, StudentsSystem Science & Engineering / Core Computing Science / Coast to Cosmos / Material World / Cultural Computing
Recruiting FacultyJoint appointments: 50% CCT, 50% home departmentTenure resides in department, but CCT has input in P&TReduced teaching load to accommodate higher level serviceAlso associate faculty from campus
Recruiting FacultyJoint appointments: 50% CCT, 50% home departmentTenure resides in department, but CCT has input in P&TReduced teaching load to accommodate higher level serviceAlso associate faculty from campusCurrently recruiting faculty in digital media and arts
External Funding: Increasingly able to respond to large solicitations
Vision: combine research and infrastructureResearchComputer scienceApplicationsToolsCyD Division at CCT to coordinate thisCyDstaff:PhDs in CS and applications who understand the whole picture and want to grow the ecosystemInfrastructure
Hardware
Operations
Policies1616CyberinfrastructureDevelopment
CI in LouisianaTeraGridLONI Institute: People and CollaborationsCybertools: Tools and ServicesLA TechLONI: ~100TF IBM, Dell SupercomputersLSULONI: 40 GbpsnetworkSUBRUNONational Lambda RailTulaneUL-L17
18State initiative ($50M) to support research (2004):40 Gbps optical network + NLRConnects stateuniversities, health science centers Compute resources: ~100 Tflops across stateData resources ~500TB with NSF PetaShareLONI customers: MS universities, K12, hospitals, LPBLouisiana Optical Network (LONI)
LONI Compute Resources
Support: HPC@LSUPartnership between LSU CCT and LSU ITS to support and maintain LSU HPC resources and provide user trainingHPC@LSU also supports/maintains LONI, through support from Louisiana BoRHPC@LSU also supports TeraGrid, through support from NSF20
NSF EPSCOR RIICyberTools Project: Enabler and Driver21$12M from NSF/BOR, 2007-2010, 9 research institutions in Louisiana
Cybertools Infrastructure22WP1: Scheduling and Data ServicesInfrastructure deployment, high availability, scheduling, data archiving and retrieval, metadataWP2: Information Services and PortalsInformation services (infrastructure, apps, experiments), application interfaces for scientists, portals for information gatheringWP3: Visualization servicesData/viz integration, HD streaming viz, advanced viz facilities, integration with application toolkitsWP4: Application toolkitsToolkits to support similation codes (CFD, MD, other), application managers, SAGA interfaces
Cybertools Infrastructure23WP1: Scheduling and Data ServicesInfrastructure deployment, high availability, scheduling, data archiving and retrieval, metadataWP2: Information Services and PortalsInformation services (infrastructure, apps, experiments), application interfaces for scientists, portals for information gatheringWP3: Visualization servicesData/viz integration, HD streaming viz, advanced viz facilities, integration with application toolkitsWP4: Application toolkitsToolkits to support similation codes (CFD, MD, other), application managers, SAGA interfacesEach Workpackage has state-wide collaboration meetings bi-weeklyGraduate students have interdisciplinary committeesWorking with LONI staff to support state softwareCyberinfrastructure software developed in collaboration with science drivers
Ci days notre_dame_april2010
Extensible, open-sourced co-allocation system
Can already reserve:
Time on supercomputers (advance reservation), and
Dedicated paths on GMPLS-based networks with simple topologies
Uses Paxos Commit to atomically reserve multiple resources, while providing a highly-available service
Used to coordinate bookings across EnLIGHTened and G-lambda testbedsin largest demonstration of its kind to date
Used for setting up the network for Thomas Sterlings HPC Class which goes out live in HDCredit: Jon MacLaren25HARC: Highly Available Resource Co-scheduler
Vish
Above left: co-located and distributed users collaboratively manipulate a 3D visualization in an AccessGrid meeting using viz tangibles interaction devices Right: an interaction tray is used together with RFID-tagged tangible menus to access and manipulate scientific visualizations27Credit: Brygg UlmerViz Tangibles
Simple API for Grid Applications
Cactus Framework
CactusComponent-based HPC framework Freely-available environment forcollaborative application developmentCutting edge CSGrid computing, petascale, accelerators, steering, remote vizActive user & developer communities10 year pedigree, >$10M support

More Related Content

Ci days notre_dame_april2010

  • 1. Cyberinfrastructure in Louisiana: From Black Holes to HurricanesGabrielle AllenDepartment of Computer ScienceCenter for Computation & TechnologyLouisiana State Universityhttp://www.cct.lsu.edu/~gallen
  • 2. Complex Problem SolvingComputation as Third Pillar of Scientific InvestigationReal world applicationsCommunitiesNo single group, university, or state can do these problemsMust integrate CS, Math, Bio, Sensors, Engineering, more...Data everywhereSupercomputers generate petabytes2
  • 3. 3 LIGO: Laser Interferometric Gravitational Wave ObservatoryTies together theory, computation, and experimentGravitational Wave Astronomy
  • 4. 4Louisiana Coastal AreaRich dynamic environment for modeling: coupled models, multi-scale, realtime data (sensors, satellites)Models, Data, Grids for Hurricane forecastsEmergency preparedness Wetland reconstructionEcological studies and fish populationsOilspillbehaviourLevee designRescueHypoxia Dead ZoneAlgae blooms24/7/365 shipping forecasts
  • 6. 6Highly scalable algorithms (Petascale, exascale computing, heterogeneous architectures)Storage & networks: provenance, metadataVisualization (Large data, interactive, remote, AMR)Software engineering (code generation, verification & validation, interfaces, interoperability, etc)Workflows, Grid & distributed computingEducation Research Issues
  • 7. PITAC Report Summary (June 2005): 7Computational science -- the use of advanced computing capabilities to understand and solve complex problems -- is critical to scientific leadership, economic competitiveness, and national security. It is one of the most important technical fields of the 21st century because it is essential to advances throughout society.Universities must significantly change organizational structures: multidisciplinary & collaborative research are needed [for US] to remain competitive in global scienceComplex problems: Innovations will occur at boundaries
  • 8. Louisiana Response: Center for Computation & Technology (CCT)8State commitment for IT in 2002$25M/year for Vision 20/20, $9M to LSUUniversity commitment to build new programsOpportunity to build new world class program in interdisciplinary research & educationEd Seidel recruited to LSU, created & implemented vision for state-wide collaborationCenter for Computation & Technology (2003)Louisiana Optical Network Initiative (2005)CyberTools (2007), LONI Institute (2007)Multidisciplinary Hiring Initiatives (2007)
  • 9. CCT Plan: Designing a Computational Science Initiative at LSUCyberinfrastructure: access to local/national compute, storage, network, visualization resources; support; end-to-end integration. Interdisciplinary research: university policies on joint appointments, university-wide curricula, appreciation of computational science outcomes (e.g. software, data)Computational research: faculty who develop or use computation as part of their research; undergraduate and graduate courses to train and educate students10CCT Strategic PlanMission: The CCT at LSU is an innovative and interdisciplinary research environment, advancing computational sciences, technologies, and the disciplines they touch. The CCT serves Louisiana through international collaboration, leading progress through revolutionary advancement in academia and industry.Strategic Plan:ResearchEducationServiceInfrastructureEconomic Developmenthttps://cct.lsu.edu/uploads/CCTStrategicPlan20062010.pdf
  • 10. Growth of CCTSeptember 2003May 2008
  • 11. CCT Focus Areas: Faculty LedAreas chosen to bring together interdisciplinary researchers towards common theme: Faculty, Research Staff, Postdocs, StudentsSystem Science & Engineering / Core Computing Science / Coast to Cosmos / Material World / Cultural Computing
  • 12. Recruiting FacultyJoint appointments: 50% CCT, 50% home departmentTenure resides in department, but CCT has input in P&TReduced teaching load to accommodate higher level serviceAlso associate faculty from campus
  • 13. Recruiting FacultyJoint appointments: 50% CCT, 50% home departmentTenure resides in department, but CCT has input in P&TReduced teaching load to accommodate higher level serviceAlso associate faculty from campusCurrently recruiting faculty in digital media and arts
  • 14. External Funding: Increasingly able to respond to large solicitations
  • 15. Vision: combine research and infrastructureResearchComputer scienceApplicationsToolsCyD Division at CCT to coordinate thisCyDstaff:PhDs in CS and applications who understand the whole picture and want to grow the ecosystemInfrastructure
  • 19. CI in LouisianaTeraGridLONI Institute: People and CollaborationsCybertools: Tools and ServicesLA TechLONI: ~100TF IBM, Dell SupercomputersLSULONI: 40 GbpsnetworkSUBRUNONational Lambda RailTulaneUL-L17
  • 20. 18State initiative ($50M) to support research (2004):40 Gbps optical network + NLRConnects stateuniversities, health science centers Compute resources: ~100 Tflops across stateData resources ~500TB with NSF PetaShareLONI customers: MS universities, K12, hospitals, LPBLouisiana Optical Network (LONI)
  • 22. Support: HPC@LSUPartnership between LSU CCT and LSU ITS to support and maintain LSU HPC resources and provide user trainingHPC@LSU also supports/maintains LONI, through support from Louisiana BoRHPC@LSU also supports TeraGrid, through support from NSF20
  • 23. NSF EPSCOR RIICyberTools Project: Enabler and Driver21$12M from NSF/BOR, 2007-2010, 9 research institutions in Louisiana
  • 24. Cybertools Infrastructure22WP1: Scheduling and Data ServicesInfrastructure deployment, high availability, scheduling, data archiving and retrieval, metadataWP2: Information Services and PortalsInformation services (infrastructure, apps, experiments), application interfaces for scientists, portals for information gatheringWP3: Visualization servicesData/viz integration, HD streaming viz, advanced viz facilities, integration with application toolkitsWP4: Application toolkitsToolkits to support similation codes (CFD, MD, other), application managers, SAGA interfaces
  • 25. Cybertools Infrastructure23WP1: Scheduling and Data ServicesInfrastructure deployment, high availability, scheduling, data archiving and retrieval, metadataWP2: Information Services and PortalsInformation services (infrastructure, apps, experiments), application interfaces for scientists, portals for information gatheringWP3: Visualization servicesData/viz integration, HD streaming viz, advanced viz facilities, integration with application toolkitsWP4: Application toolkitsToolkits to support similation codes (CFD, MD, other), application managers, SAGA interfacesEach Workpackage has state-wide collaboration meetings bi-weeklyGraduate students have interdisciplinary committeesWorking with LONI staff to support state softwareCyberinfrastructure software developed in collaboration with science drivers
  • 29. Time on supercomputers (advance reservation), and
  • 30. Dedicated paths on GMPLS-based networks with simple topologies
  • 31. Uses Paxos Commit to atomically reserve multiple resources, while providing a highly-available service
  • 32. Used to coordinate bookings across EnLIGHTened and G-lambda testbedsin largest demonstration of its kind to date
  • 33. Used for setting up the network for Thomas Sterlings HPC Class which goes out live in HDCredit: Jon MacLaren25HARC: Highly Available Resource Co-scheduler
  • 34. Vish
  • 35. Above left: co-located and distributed users collaboratively manipulate a 3D visualization in an AccessGrid meeting using viz tangibles interaction devices Right: an interaction tray is used together with RFID-tagged tangible menus to access and manipulate scientific visualizations27Credit: Brygg UlmerViz Tangibles
  • 36. Simple API for Grid Applications
  • 38. CactusComponent-based HPC framework Freely-available environment forcollaborative application developmentCutting edge CSGrid computing, petascale, accelerators, steering, remote vizActive user & developer communities10 year pedigree, >$10M support
  • 39. Numerical Relativity, CFD, Coastal, Reservoir Engineering, Domain-specific toolkits, e.g. CFD toolkitFD/FV/FE numerical methods
  • 41. Uses PETSc, Trilinos, MUMPS, HYPRE30
  • 42. LONI Institute$15M 5-year project (2007-2012)$7M BoR, $8M from universitiesCreate bold new inter-university superstructureNew faculty (12), staff (6), students (36); train others. Focus on CS, Bio, Materials, but all disciplines impactedPromote research at interfaces for innovationDraw on, enhance strengths of all universitiesSolve complex problems through collaboration & computationMuch stronger recruiting opportunities for all institutionsStatewide interdisciplinary education & research programCreate University-Industry Research Centers (UIRCs)Transform our stateSuch committed cooperation between sites extraordinary31
  • 44. Motivation:Gamma-Ray Burst Grand ChallengeMost energetic events in the universeMechanism still a riddle;grand challenge in astrophysicsModelling requires expertise in many fields of physics(general relativity, magneto-hydrodynamics, neutrinos, ...)Requires petascale computingACM, doi:10.1145/1341811.1341831
  • 45. Einstein Toolkithttp://www.einsteintoolkit.orgBased on Cactus FrameworkOver 130 open, community developed Cactus modulesBuilding a consortium of usersGovernance and software developmentMembers40 listed on web page10 different groupsUS, Japan, Mexico, Spain, Germany, Canada300 science publications, 50 student theses
  • 46. Einstein ToolkitResearch/FundingRelativistic astrophysics (black holes, supernova, Gamma Ray bursts) (PIF, ++)Large scale computing (PetaApps, PRAC, Blue Waters)Verification, performance (Alpaca, HPCOPS)Data and metadata (PIF)Visualization, interaction, high speed networks (CyberTools, EAGER)Other apps: Coastal, CFD (CyberTools)
  • 47. Winners IEEE SCALE09! (Hutanu, Schnetter et al)2048 core simulation code
  • 49. Remote LONI nodes as data servers
  • 50. 10243 spatial resolution (1GByte/timestep)
  • 52. Scalable Approach!!!Conclusions37Cyberinfrastructure has played a crucial role in Louisiana in building an environment able to support and nurture high quality researchAdvances were needed in many areas:Networks, Computing, Tools & services, Applications, PeopleReal results:REU program in computational science, 16 undergrads 2 of 16 Blue Waters undergraduate internships to LAPetaApps, PRAC, CAREER awardsLouisiana is developing cyberinfrastructure for its own researchers, and for the nation (Cactus, PetaShare, SAGA, HARC, Vish, .)
  • 53. ReferencesLouisiana: A Model for Advancing Regional e-Research through Cyberinfrastructure,D. S. Katz, G. Allen, R. Cortez, C. Cruz-Neira, R. Gottumukkala, Z. D. Green- wood, L. Guice, S. Jha, R. Kolluru, T. Kosar, L. Leger, H. Liu, C. McMahon, J. Nabrzyski, B. Rodriguez-Milla, E. Seidel, G. Speyrer, M. Stubblefield, B. Voss, S. Whittenburg, Phil. Trans. R. Soc. A 28, June 2009, vol. 367, no. 1897, 2459-2469, (2009).Computational Science, Infrastructure and Interdisciplinary Research on University Campuses: Experiences and Lessons from the Center for Computation & Technology,G. Allen, D. S. Katz,, CCT Technical Report Series, CCT-TR-2010-1, 2010 (Full version soon)Einstein Toolkit Consortiumwww.einsteintoolkit.orgCactus Frameworkwww.cactuscode.org