際際滷

際際滷Share a Scribd company logo

EGI Federated Cloud - May 2019

Download as PPTX, PDF
EGI Federation
EGI Federation

The EGI Federated Cloud is experiencing a boost of demand of cloud services since 2018. More than 26 Million hours of CPU wall time were delivered and Natural Sciences are the most active scientific group.

1 of 15
Download to read offline
www.egi.eu @EGI_eInfra The work of the EGI Foundation is partly funded by the European Commission under H2020 Framework Programme EGI: Advanced Computing for Research May 2019 EGI Cloud Compute Service enol.fernandez@egi.eu Enol Fern叩ndez
@EGI_eInfrawww.egi.eu 05/06/2019 2 Multi-cloud IaaS with Single Sign-On Federation features: Common VM image catalogue Discovery, accounting, SLO monitoring Unified GUI dashboard EGI Cloud Federation Cloud Compute Cloud Container Compute BETA Training Infrastructure Online Storage Applications on Demand BETA Notebooks BETA EGI Services powered by the Cloud Federation
@EGI_eInfrawww.egi.eu 05/06/2019 3 EGI Cloud enables research oriented computing IaaS providers Federation Services Orchestration Platforms Check-in : Common AuthN and AuthZ across all layers Research Platforms Operators Research Communities Research Communities
@EGI_eInfrawww.egi.eu 05/06/2019 4 The infrastructure Finalising integration!New!
@EGI_eInfrawww.egi.eu 05/06/2019 5 Access and capacity allocation VO fedcloud.egi.eu access.egi.eu (AoD) Community VOs Membership requirements X.509 certificate Member of European research institutions (i.e. Check-in account) Depends on VO (most still require X.509 certificate), new ones can be Check-in only AppDB VMOps Depends on VO CLI/API access Membership duration 6 m, extensible up to 1 yr 6 m, extensible up to 1 yr 1 year renewable membership Resource limits Opportunistic, varying quota at providers. Limited lifetime per VMs may be applied Opportunistic, varying quota at providers. Limited lifetime per VMs may be applied Specified in the SLA: opportunistic, pledged, time based Available providers All providers Providers of the Applications on Demand platform: INFN-Catania- Stack, CESGA, RECAS-Bari As specified in the SLA
@EGI_eInfrawww.egi.eu 05/06/2019 6 Scientific Disciplines/Cloud Humanities: CLARIN (2018), DARIAH (+28%, 2017), Life Sciences: ELIXIR (+104%, 2017) Environment: EMSO (2018), LifeWatch (+45%, 2015) BIOISI (+45%, 2016), BIOMED (+575%, 2016), CHIPSTER (+256%, 2016), GEOHAZARDS (+19%, 2016), OpenCOAST (2018), EXTRAS (2018),
@EGI_eInfrawww.egi.eu 05/06/2019 7 0.0E+00 5.0E+05 1.0E+06 1.5E+06 2.0E+06 2.5E+06 3.0E+06 3.5E+06 4.0E+06 4.5E+06 5.0E+06 2018 Apr 2018 May 2018 Jun 2018 Jul 2018 Aug 2018 Sep 2018 Oct 2018 Nov 2018 Dec 2019 Jan 2019 Feb 2019 Mar 2019 Apr French NGI Vos EOSC-hub VOS fedcloud.egi.eu vo.lifewatch.eu geohazards.terradue.com bioisi vo.access.egi.eu vo.emsodev.eu peachnote.com vo.nextgeoss.eu vo.nbis.se d4science.org chipster.csc.fi ericll.org enmr.eu biomed training.egi.eu Usage fedcloud.egi.eu Last 12 Months: 26,4M CPU hours 500K VMs
@EGI_eInfrawww.egi.eu 05/06/2019 8 Check-in provides: Single Sign-On through eduGAIN, social media and other institutional or community- managed identity providers Harmonised authorisation information, aggregated from multiple sources Industry Standard OpenID Connect technology allowing web and non-web access to services Integration: Native support at all the EGI Cloud layers (IaaS providers, IaaS Orchestration, AppDB VMOps) and at EGI services/platforms running on top: Notebooks, AoD, Container FedCloud client https://aai.egi.eu/fedcloud for easily getting individual tokens for CLI/API access New: Check-in & EGI Cloud
@EGI_eInfrawww.egi.eu 05/06/2019 9 Cloud Management Framework IaaS API Cloud Management Framework IaaS API Direct API Access Interfaces and Check-in EGI Federation features: Accounting, Monitoring, Conf. DB, Info Discovery, AppDB AppDB VMOpsGUI Access IaaS Federated Access Tools Federated Access Developers/ Advanced users AAI: Check-in GUI Users
@EGI_eInfrawww.egi.eu 05/06/2019 10 Check-in web based tools 15/06/2018 10
@EGI_eInfrawww.egi.eu 05/06/2019 11 Check-in CLI / API access
@EGI_eInfrawww.egi.eu 05/06/2019 12 AppDB VMOps - User Friendly GUI Single Web dashboard to manage VMs in the federation Point-and-click solution to create new VMs Integrated with: Check-in, discovery, VM catalogue, monitoring Powered by Infrastructure Manager
@EGI_eInfrawww.egi.eu 05/06/2019 13 Information discovery Transition to message-based information discovery GlueSchema 2.1 Native API support OpenNebula as a first-class citizen of the federation AppDB improvements to support native APIs Cloud Container Automated deployment of Kubernetes Coming next, federation features
@EGI_eInfrawww.egi.eu 05/06/2019 14 Keystone Nova Glance ooi cloudkeeperinfo-providercASO Keystone-VOMS Sync VM images Legacy VOMS Authentication Extract information EGI Federation Services EGI Check-in Extract usage information OpenID Connect Authentication OCCI API Social Logins Neutron OpenStack APIs Evolution, site perspective
@EGI_eInfrawww.egi.eu 05/06/2019 15 EGI Cloud brings together IaaS resources from NGIs to support (federated) research communities Pilot and test with fedcloud.egi.eu or access.egi.eu VOs, production usage with community VOs with SLAs Check-in enabled federation: access the service with your own credentials, both web and CLI/API AppDB VMOps, a common dashboard for the federation Evolution Improved native API support, more complete information Even more lightweight federation, while keeping same user features Summary

More Related Content

EGI Federated Cloud - May 2019

  • 1. www.egi.eu @EGI_eInfra The work of the EGI Foundation is partly funded by the European Commission under H2020 Framework Programme EGI: Advanced Computing for Research May 2019 EGI Cloud Compute Service enol.fernandez@egi.eu Enol Fern叩ndez
  • 2. @EGI_eInfrawww.egi.eu 05/06/2019 2 Multi-cloud IaaS with Single Sign-On Federation features: Common VM image catalogue Discovery, accounting, SLO monitoring Unified GUI dashboard EGI Cloud Federation Cloud Compute Cloud Container Compute BETA Training Infrastructure Online Storage Applications on Demand BETA Notebooks BETA EGI Services powered by the Cloud Federation
  • 3. @EGI_eInfrawww.egi.eu 05/06/2019 3 EGI Cloud enables research oriented computing IaaS providers Federation Services Orchestration Platforms Check-in : Common AuthN and AuthZ across all layers Research Platforms Operators Research Communities Research Communities
  • 4. @EGI_eInfrawww.egi.eu 05/06/2019 4 The infrastructure Finalising integration!New!
  • 5. @EGI_eInfrawww.egi.eu 05/06/2019 5 Access and capacity allocation VO fedcloud.egi.eu access.egi.eu (AoD) Community VOs Membership requirements X.509 certificate Member of European research institutions (i.e. Check-in account) Depends on VO (most still require X.509 certificate), new ones can be Check-in only AppDB VMOps Depends on VO CLI/API access Membership duration 6 m, extensible up to 1 yr 6 m, extensible up to 1 yr 1 year renewable membership Resource limits Opportunistic, varying quota at providers. Limited lifetime per VMs may be applied Opportunistic, varying quota at providers. Limited lifetime per VMs may be applied Specified in the SLA: opportunistic, pledged, time based Available providers All providers Providers of the Applications on Demand platform: INFN-Catania- Stack, CESGA, RECAS-Bari As specified in the SLA
  • 6. @EGI_eInfrawww.egi.eu 05/06/2019 6 Scientific Disciplines/Cloud Humanities: CLARIN (2018), DARIAH (+28%, 2017), Life Sciences: ELIXIR (+104%, 2017) Environment: EMSO (2018), LifeWatch (+45%, 2015) BIOISI (+45%, 2016), BIOMED (+575%, 2016), CHIPSTER (+256%, 2016), GEOHAZARDS (+19%, 2016), OpenCOAST (2018), EXTRAS (2018),
  • 7. @EGI_eInfrawww.egi.eu 05/06/2019 7 0.0E+00 5.0E+05 1.0E+06 1.5E+06 2.0E+06 2.5E+06 3.0E+06 3.5E+06 4.0E+06 4.5E+06 5.0E+06 2018 Apr 2018 May 2018 Jun 2018 Jul 2018 Aug 2018 Sep 2018 Oct 2018 Nov 2018 Dec 2019 Jan 2019 Feb 2019 Mar 2019 Apr French NGI Vos EOSC-hub VOS fedcloud.egi.eu vo.lifewatch.eu geohazards.terradue.com bioisi vo.access.egi.eu vo.emsodev.eu peachnote.com vo.nextgeoss.eu vo.nbis.se d4science.org chipster.csc.fi ericll.org enmr.eu biomed training.egi.eu Usage fedcloud.egi.eu Last 12 Months: 26,4M CPU hours 500K VMs
  • 8. @EGI_eInfrawww.egi.eu 05/06/2019 8 Check-in provides: Single Sign-On through eduGAIN, social media and other institutional or community- managed identity providers Harmonised authorisation information, aggregated from multiple sources Industry Standard OpenID Connect technology allowing web and non-web access to services Integration: Native support at all the EGI Cloud layers (IaaS providers, IaaS Orchestration, AppDB VMOps) and at EGI services/platforms running on top: Notebooks, AoD, Container FedCloud client https://aai.egi.eu/fedcloud for easily getting individual tokens for CLI/API access New: Check-in & EGI Cloud
  • 9. @EGI_eInfrawww.egi.eu 05/06/2019 9 Cloud Management Framework IaaS API Cloud Management Framework IaaS API Direct API Access Interfaces and Check-in EGI Federation features: Accounting, Monitoring, Conf. DB, Info Discovery, AppDB AppDB VMOpsGUI Access IaaS Federated Access Tools Federated Access Developers/ Advanced users AAI: Check-in GUI Users
  • 10. @EGI_eInfrawww.egi.eu 05/06/2019 10 Check-in web based tools 15/06/2018 10
  • 12. @EGI_eInfrawww.egi.eu 05/06/2019 12 AppDB VMOps - User Friendly GUI Single Web dashboard to manage VMs in the federation Point-and-click solution to create new VMs Integrated with: Check-in, discovery, VM catalogue, monitoring Powered by Infrastructure Manager
  • 13. @EGI_eInfrawww.egi.eu 05/06/2019 13 Information discovery Transition to message-based information discovery GlueSchema 2.1 Native API support OpenNebula as a first-class citizen of the federation AppDB improvements to support native APIs Cloud Container Automated deployment of Kubernetes Coming next, federation features
  • 14. @EGI_eInfrawww.egi.eu 05/06/2019 14 Keystone Nova Glance ooi cloudkeeperinfo-providercASO Keystone-VOMS Sync VM images Legacy VOMS Authentication Extract information EGI Federation Services EGI Check-in Extract usage information OpenID Connect Authentication OCCI API Social Logins Neutron OpenStack APIs Evolution, site perspective
  • 15. @EGI_eInfrawww.egi.eu 05/06/2019 15 EGI Cloud brings together IaaS resources from NGIs to support (federated) research communities Pilot and test with fedcloud.egi.eu or access.egi.eu VOs, production usage with community VOs with SLAs Check-in enabled federation: access the service with your own credentials, both web and CLI/API AppDB VMOps, a common dashboard for the federation Evolution Improved native API support, more complete information Even more lightweight federation, while keeping same user features Summary

Editor's Notes

  • #3: List of services of EGI powered by the EGI Cloud Federation
  • #5: IN2P3 highlighted just in case
  • #10: User communities build either on top of orchestration tools that allow to deal with multiple providers in a homogeneous way or directly interact with the native APIs of the provides. Both cases they can use single sign-on thanks to Check-in. A common GUI provided by AppDB VMOps brings a user-friendly dashboard to manage the resources at the distributed providers The EGI federation services are integrated with the providers using their native APIs to deliver the extra features of EGI Cloud mentioned in previous slide GUI access: AppDB VMOps https://dashboard.appdb.egi.eu/vmops API/CLI access: Discovery: AppDB IS API (REST and GraphQL) https://wiki.egi.eu/wiki/Federated_Cloud_Discovery#AppDB IaaS Federated Access Tools: https://wiki.egi.eu/wiki/Federated_Cloud_IaaS_Orchestration Direct IaaS access, several APIs depending on the provider: https://wiki.egi.eu/wiki/Federated_Cloud_APIs_and_SDKs