際際滷

際際滷Share a Scribd company logo

The Kubernetes Operator Pattern - ContainerConf Nov 2017

J
Jakob Karalus

The document discusses the operator pattern in Kubernetes for managing stateful services. It describes how operators define custom APIs using CustomResourceDefinitions and create controllers that react to changes in the API to manage applications and infrastructure. Operators allow automating tasks like backups, scaling, healing and configuration that are difficult to do at the application level alone. Some examples of using operators for services like Kafka and Elasticsearch are provided.

1 of 24
Downloaded 81 times
The Operator Pattern 1 Managing Stateful Services in Kubernetes Jakob Karalus, @krallistic
2 Data Science + DevOps Codecentric CKA Twiiter: @krallistic Github: github.com/krallistic $whoami
3 Write some Deployment, Services, Configmaps etc Deploy them to K8s Maybe create a Helm Chart YAML YAML YAML Normal Kubernetes Deployment
4 Success?!
5 Backups? Upscaling? Reshuffle Data? Downscaling? Without Dataloss? Healing? Restore Backups? Configuration? Tedious Templating? But Day 2 Operations?
If only we could automate this! 6 In a Kubernetes native way!
7 Operators Kubernetes Apache Kafka Operator
8 Human Operational Software Custom Software Kubernetes Native: CustomResourceDefinition So lets write one: High Level Operators
Defines a new API Seamless integration with existing API Kubectl support 9 CustomResourceDefinition apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: crontabs.stable.example.com spec: group: stable.example.com version: v1 scope: Namespaced names: plural: crontabs singular: crontab kind: CronTab shortNames: - ct validation: # openAPIV3Schema is the schema for validating custom objects.
Actual Object in new API No functionality. 10 CustomResourceDefinition apiVersion: "stable.example.com/v1" kind: CronTab metadata: name: my-new-cron-object spec: cronSpec: "* * * * */5" image: my-awesome-cron-image replicas: 5
11 Operator create WATCH on CR Objects Analyze difference Actual vs Desired State Act on changes Control Loop Observe Analyze Act
Kafka Basics 12 Broker 0 Topic: test Partitions: 4 Replicas: 2 Partition 0 Replica 0 Partition 2 Replica 1 Partition 3 Replica 1 Broker 1 Broker 2 Partition 0 Replica 1 Partition 1 Replica 1 Partition 2 Replica 0 Partition 1 Replica 0 Partition 3 Replica 0 Topic: hello Partitions: 8 Replicas: 3
13 Create Cluster Kind: KafkaCluster Name: analytics Replicas: 3 ADD Event Kind: ConfigMap Name: kafka Data: | Kind: Statefulset Name: analytics Replicas: 3 Image: Kafka POST Create Pods
14 Downsize Cluster Kind: KafkaCluster Name: analytics Replicas: 2 Update Event Desired: Kind: Statefulset Name: analytics Replicas: 2 Image: Kafka UPDATE Current: Kind: Statefulset Name: analytics Replicas: 3 Image: Kafka Cluster Rebalanced Rebalance Topic Delete Pod
15 Rebalance Topics with Hot Partitions
Elasticsearch Postgres Tensorflow 16 Other Operators
17 Are we reinventing the Wheel? Helm? Mesos Frameworks? Nomad Custom Scheduler? Docker Swarm Plugins? Take a step back
18 Helm itself a Operator (somewhat, working on it https://github.com/kubernetes/helm/issues/3089) ) Controllers Operator = Controller + CRD Operator = External Software Controller = Internal Only do operators if you cant solve it with Helm. Operators vs Helm vs Controller
19 Create API Spec Generate some Objects needed by Informer etc (Since 1.8) See: https://blog.openshift.com/kubernetes-deep-dive-code-generation- customresources/ (Excellent, by sttts) Generator Controller Informer Main Code!
20 Microservices, single Deployment Stateless, use CRD for States Operations should be Idempotent Leverage K8S Objects as most as possible CRD should be versioned, backwards compatible Best Practice Operators
Questions? Discuss! 21
22 Custom Resource Definitions API Aggregation Initializers Scheduler Extenders Custom Schedulers Flex Volumes Cloud Provider CRI & CNI Admission Webhook Kubernetes Extensibility
23 Task Mesos Kuberentes Custom Resource Placement Write a framework Write a custom scheduler Special resource init Write a framework Initializer API access Every Framework has its own API Unified API Special lifecyle Write a framework Kubernetes Operator Custom execution Write a framework + executioner CRI Interface + Scheduler Comparison
24 codecentric AG Hochstrae 11 42697 Solingen E-Mail: info@codecentric.de www.codecentric.de Telefon: +49 (0) 212. 23 36 28 0 Telefax: +49 (0) 212.23 36 28 79 Address Contact Info Telephone 24 Stay connected

More Related Content

The Kubernetes Operator Pattern - ContainerConf Nov 2017

  • 1. The Operator Pattern 1 Managing Stateful Services in Kubernetes Jakob Karalus, @krallistic
  • 2. 2 Data Science + DevOps Codecentric CKA Twiiter: @krallistic Github: github.com/krallistic $whoami
  • 3. 3 Write some Deployment, Services, Configmaps etc Deploy them to K8s Maybe create a Helm Chart YAML YAML YAML Normal Kubernetes Deployment
  • 5. 5 Backups? Upscaling? Reshuffle Data? Downscaling? Without Dataloss? Healing? Restore Backups? Configuration? Tedious Templating? But Day 2 Operations?
  • 6. If only we could automate this! 6 In a Kubernetes native way!
  • 8. 8 Human Operational Software Custom Software Kubernetes Native: CustomResourceDefinition So lets write one: High Level Operators
  • 9. Defines a new API Seamless integration with existing API Kubectl support 9 CustomResourceDefinition apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: crontabs.stable.example.com spec: group: stable.example.com version: v1 scope: Namespaced names: plural: crontabs singular: crontab kind: CronTab shortNames: - ct validation: # openAPIV3Schema is the schema for validating custom objects.
  • 10. Actual Object in new API No functionality. 10 CustomResourceDefinition apiVersion: "stable.example.com/v1" kind: CronTab metadata: name: my-new-cron-object spec: cronSpec: "* * * * */5" image: my-awesome-cron-image replicas: 5
  • 11. 11 Operator create WATCH on CR Objects Analyze difference Actual vs Desired State Act on changes Control Loop Observe Analyze Act
  • 12. Kafka Basics 12 Broker 0 Topic: test Partitions: 4 Replicas: 2 Partition 0 Replica 0 Partition 2 Replica 1 Partition 3 Replica 1 Broker 1 Broker 2 Partition 0 Replica 1 Partition 1 Replica 1 Partition 2 Replica 0 Partition 1 Replica 0 Partition 3 Replica 0 Topic: hello Partitions: 8 Replicas: 3
  • 13. 13 Create Cluster Kind: KafkaCluster Name: analytics Replicas: 3 ADD Event Kind: ConfigMap Name: kafka Data: | Kind: Statefulset Name: analytics Replicas: 3 Image: Kafka POST Create Pods
  • 14. 14 Downsize Cluster Kind: KafkaCluster Name: analytics Replicas: 2 Update Event Desired: Kind: Statefulset Name: analytics Replicas: 2 Image: Kafka UPDATE Current: Kind: Statefulset Name: analytics Replicas: 3 Image: Kafka Cluster Rebalanced Rebalance Topic Delete Pod
  • 15. 15 Rebalance Topics with Hot Partitions
  • 17. 17 Are we reinventing the Wheel? Helm? Mesos Frameworks? Nomad Custom Scheduler? Docker Swarm Plugins? Take a step back
  • 18. 18 Helm itself a Operator (somewhat, working on it https://github.com/kubernetes/helm/issues/3089) ) Controllers Operator = Controller + CRD Operator = External Software Controller = Internal Only do operators if you cant solve it with Helm. Operators vs Helm vs Controller
  • 19. 19 Create API Spec Generate some Objects needed by Informer etc (Since 1.8) See: https://blog.openshift.com/kubernetes-deep-dive-code-generation- customresources/ (Excellent, by sttts) Generator Controller Informer Main Code!
  • 20. 20 Microservices, single Deployment Stateless, use CRD for States Operations should be Idempotent Leverage K8S Objects as most as possible CRD should be versioned, backwards compatible Best Practice Operators
  • 22. 22 Custom Resource Definitions API Aggregation Initializers Scheduler Extenders Custom Schedulers Flex Volumes Cloud Provider CRI & CNI Admission Webhook Kubernetes Extensibility
  • 23. 23 Task Mesos Kuberentes Custom Resource Placement Write a framework Write a custom scheduler Special resource init Write a framework Initializer API access Every Framework has its own API Unified API Special lifecyle Write a framework Kubernetes Operator Custom execution Write a framework + executioner CRI Interface + Scheduler Comparison
  • 24. 24 codecentric AG Hochstrae 11 42697 Solingen E-Mail: info@codecentric.de www.codecentric.de Telefon: +49 (0) 212. 23 36 28 0 Telefax: +49 (0) 212.23 36 28 79 Address Contact Info Telephone 24 Stay connected