kubernetes : From beginner to Advanced
- 1. Copyright 息 2017, Oracle and/or its affiliates. All rights reserved. | Confidential Oracle Internal/Restricted/Highly Restricted 1 th 螳誤 inho.kang@oracle.com Kubernetes : from Beginner to Advanced 2019.03.16 9th Oracle Developer Meetup
- 2. Copyright 息 2017, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracles products remains at the sole discretion of Oracle. Confidential Oracle Internal/Restricted/Highly Restricted 2
- 3. 則 .Net Developer 則 CBD, SOA Methodology Consulting 則 ITA/EA, ISP Consulting 則 Oracle Corp. 則 Middleware 則 Cloud Native Application, Container Native 則 Emerging Technology Team 則 k8s korea user group innoshom@gamil.com
- 4. Confidential Oracle Internal/Restricted/Highly Restricted 4 Kubernetes Concept
- 6. Confidential Oracle Internal/Restricted/Highly Restricted 6 Kubernetes Concept - Container
- 8. 豢豌 : kubernetes in action
- 13. 豢豌 : kubernetes in action
- 15. kube-apiserver 則 貎覯ろ一 API襯 語 襷ろ 貉危. 貎覯ろ一 貉碁， 語 襦語企. 則 れ(讀, 襷 語ろ伎るゼ 襦危 れ) り. etcd 則 覈 企ろ 一危磯ゼ 企 貎覯ろ一 結 レ襦 手餌穴螳 -螳 レ. 則 貎覯ろ一 企ろ 覲企ゼ 願 etcd 一危一 覦煙 螻 企. kube-scheduler 則 碁螳 覦一讌 襦 焔 襯 螳讌螻 蠏瑚 蟲 碁襯 襷ろ 貉危. 則 れ譴襷 蟆一 伎拘危 る 螳覲 覦 螻給 襴 蟇, /語/豈 , 豺覦 覦 覦郁鴬 覈, 一危 讌, 襦-螳 螳, 磯殊碁れ . kube-controller-manager 則 貉碁，襯 蟲 襷ろ 貉危. 則 朱Μ朱, 螳 貉碁， 螳覲 襦語れ伎襷, 覲旧′煙 豢蠍 覈 覦企襴襦 貉危朱螻 襦語 伎 ろ. 則 碁 貉碁，: 碁螳 れ企 旧 蟯 豈 螳讌. 則 襴貅伎 貉碁，: 襷 れ 讌貅 譯朱 豈 螳讌. 則 誤 貉碁，: 觜れ 襯 郁屋
- 16. kubelet 則 企ろ一 螳 碁 ろ 伎碁 貉企螳 ろ 譴語 誤. 則 kubelet れ 覃貉る讀 牛 螻給 PodSpec 螳 企 PodSpec る 貉企螳 ろ 譴企 蟇願 襯 讌襦 . 則 kubelet Kubernetes螳 燕讌 貉企襯 蟯襴讌 給. kube-proxy 則 kube-proxy 語ろ ろ語 蠏豺 讌螻 郁屋 朱 貎覯ろ一 觜 豢螳 螳ロ襦 伎. Container Runtime 則 貉企 壱 貉企 豈讌 語企. 貎覯ろ一る 覈覈 壱 讌 Docker, containerd, cri- o, rktlet 蠏碁Μ螻 Kubernetes CRI (Container Runtime Interface)襯 蟲 覈 壱企.
- 17. OCI ?
- 18. Pod IP : 10.244.2.xx : Node2 Pod IP : 10.244.1.xx : Node1 K8snode1 cni0 : 10.244.1.1 Docker0 : 172.17.0.1 Enp0s3 : 10.0.2.15 Enp0s8 : 192.168.56.111 Flannel.1 : 10.244.1.0 Pod vethxxxx CNI0 : 10.244.1 VB Net
- 19. 則 Components 則 $kubectl get componentstatuses 則 Node 則 $kubectl get node 則 Pod 則 $kubectl get pod
- 20. 則 YAML Descriptor 則 Kind 則 Label 則 Replicas 則 Selector 則 MatchLabels 則 MatchExpression 則 Template 則 Spec 則 Container 則 Image 則 ports 則 Components 則 $kubectl get componentstatuses 則 Node 則 $kubectl get node 則 Pod 則 $kubectl get pod
- 21. kubectl create -f https://k8s.io/examples/controllers/nginx-deployment.yaml kubectl get rs kubectl get deployments kubectl get pods --show-labels https://kubernetes.io/docs/concepts/workloads/controllers/deployment/
- 22. kubectl get rs kubectl scale --replicas=4 rs/nginx-deployment-75bd58f5c7 kubectl get pods --show-labels https://kubernetes.io/docs/concepts/workloads/controllers/deployment/ kubectl get po ngnix-deployment-xxx o yaml kubectl edit po ngnix-deployment-xxx o yaml
- 23. 則 Equality-based requirement 則 Set-based requirement
- 27. Confidential Oracle Internal/Restricted/Highly Restricted 27 Dive into Kubernetes Internal
- 29. 則 Deployment -> ReplicatSet -> 則 Pods
- 30. Confidential Oracle Internal/Restricted/Highly Restricted 30 Dive into Kubernetes Internal - Pod
- 33. 則 Namespace 則 Hostname 則 Process IDs 則 File System 則 Network interfaces 則 Inter-Process Communication (IPC) 則 Cgroup(Control Group) 則 CPU 則 RAM 則 Block I/O 則 Network I/O Cgroups = limits how much you can use; namespaces = limits what you can see (and therefore use) https://www.ianlewis.org/en/what-are-kubernetes-pods-anyway
- 35. 則 Hostname 則 Network 則 IPC 則 Volume 螻旧 則 轟 則 貉企 則 れ 貉企 (Side - Car) 則 れ朱 - Deployment - StatefulSet - DamonSet https://www.ianlewis.org/en/what-are-kubernetes-pods-anyway
- 36. https://blog.2dal.com/2018/03/28/kubernetes-01-pod/ https://ssup2.github.io/theory_analysis/Kubernetes_Pod/?fbclid=IwAR1IZtyjusnW1iX8C1s1c5QpQMigmiH-Y7pFDZ3dwtL44TJ8esDZqp-lXHg Pod螳 ろ pause 朱 貉企螳 襾殊 ろ pause貉企 れろ伎るゼ Pod企 覈 貉企螳 螻旧 Pod 企 貉企 襦語り 譬觜 襦語り 讌 襦 pause 貉企 PID 1襦 れ 覈 貉企 襦語れ 覿覈 襦語り 螻, 貉企 襦語れ 覓語螳 覦 SIGHLD 蠏碁 覦朱 waitpid(企 襦語 ID)襯 伎 譴讌 貉企螳
- 37. 則 Liveness probe 則 Readyness probe p 152
- 38. Confidential Oracle Internal/Restricted/Highly Restricted 38 Master Component
- 39. Kube-apiserver Etcd Kube-controller-manager Cloud-controller-manager Kube-scheduler
- 40. 則 Components 則 $kubectl get componentstatuses
- 41. Confidential Oracle Internal/Restricted/Highly Restricted 41 Master Component - API Server
- 42. 則 RESTful API襯 牛伎 kubectl 螳 企殊伎誤語 旧 則 RESTful API襯 牛伎 企ろ 襯 貎朱Μ 覦 (CRUD)螻 則 蠏語覲企ゼ etcd ロ. 則 覈 企殊伎誤語 貉危碁 れ API 覯襯 牛伎襷 語. 則 企ろ一 Gatekeeper 覃, 語(Authen), 蟠(Authoriz), 螳豌伎 蟆(Validation) and admission control 則 JSON 蠍磯 HTTP API 螳 蠍磯蓋伎襷, 企ろ 企 旧 Protocol Buffer 讌
- 43. 則 API 覯 蠏豺 則 襴る 譴覲企る API 覯 覯 , API螳 覈襭螻, ろ 襴れ 蟯 手煙朱, 伎 讌 API ろ API 蠏殊 危 襦 蠍 企. 則 Alpha(ex: v1alpha1) 則 覯蠏瑚 螻, 蠍磯レ 燕覃 覯蠏瑚 語 . 蠍磯蓋朱 觜燕 則 Beta (ex: v2beta3) 則 貊螳 ろ碁螻, . 蠍磯蓋朱 燕 則 Stable(ex: v2) 則 覯
- 44. 則 貎覯ろ一 API襯 覲企 所 ロ蠍 伎, API 蠏碁９ 蟲. API 蠏碁９ REST 蟆暑 讌 螳豌伎 apiVersion 覈. 則 れ API 蠏碁９ 螻 . 則 旧 蠏碁９ 蟇一 蠏碁９ 企手 蠏碁９ REST 蟆暑 /api/v1 apiVersion: v1 . 則 企 蠏碁９ REST 蟆暑 /apis/$GROUP_NAME/$VERSION 朱 apiVersion: $GROUP_NAME/$VERSION (: apiVersion: batch/v1). 讌 API 蠏碁９ 豌伎 覈襦 Kubernetes API reference 誤 .
- 46. 則 kubectl create -f xxx.yaml json Converting 則 語, 蟠 則 豌 Client 語(key, token) 襯 蟆 則 豌 ′ 蟠 豌危 則 Admission Control 則 襴 ろ 暑 蟆 吴瑚企 螳朱 豈 則 Ex) AlwaysPullImages NamespaceLifeCycle 則 Resource Validation 則 ETCD /Registry 覦 螻豸糾規譟磯
- 47. 則 1. API 蠏碁９ 譟壱 則 2. 碁 襴ろ 譟壱 - http://127.0.0.1:8001/api/v1/nodes/ - http://127.0.0.1:8001/api/v1/nodes/k8snode1
- 49. Confidential Oracle Internal/Restricted/Highly Restricted 50 Master Component - Controller
- 50. kube-controller-manager 貉碁，襯 蟲 襷ろ 貉危. 朱Μ朱, 螳 貉碁， 螳覲 襦語れ伎襷, 覲旧′煙 豢蠍 覈 覦企襴襦 貉危朱螻 襦語 伎 ろ. API 覯襯 牛伎 Shared State(ETCD) 覲企ゼ Control loop襯 牛 螳(watch)螻 り Current State襯 Desired State襦 覲蟆曙 . 企 貉碁， れ . ( 26螳) 碁 貉碁，: 碁螳 れ企 旧 蟯 豈 螳讌. 襴貅伎 貉碁，: ろ 覈 襴貅伎 貉碁， る語 襷 れ 讌貅 譯朱 豈 螳讌. 誤 貉碁，: 誤 る碁ゼ 豈企(讀, 觜れ 襯 郁屋.) 觜 伎拘危 & 貉碁，: 襦 れろ伎れ 蠍磯蓋 螻螻 API 蠏 一 燕.
- 51. https://engineering.bitnami.com/articles/a-deep-dive-into-kubernetes-controllers.html Controller Loop - Controller 螳 譴 Object current state desired state襯 watch(螳) - 企ゼ API Server 螳 譟壱 襷る 譟壱 蟆 るる螳 蠍 覓語 event 伎朱 client-go library Listwatcher interface
- 53. 覈 貉碁， API 覯襯 牛 API る碁ゼ 螳讌螻 . 貉碁， Kubelet 讌 旧蟇磯 企 譬襯 覈豪 企Μ讌 . kubelet 譟伎讌 譟一姶 讌 覈詩. 貉碁，螳 API 覯 襴るゼ 一企 危襦 Kubelets 貎覯ろ一 觜 襦 貉碁， 譟伎襯 讌 覈詩. 蠏碁Μ螻 貉企襯 り ろ語 ろ襴讌 郁屋蟇磯 れ 襦 覦碁一煙 れ. 貉碁， 語 豌 ろ 覿覿 豌襴覩襦, 企 貎覯ろ一 企ろ一 企至 ｼ豎讌讌 危危る kubelet螻 觜 襦螳 殊 危危 螳 .
- 54. Confidential Oracle Internal/Restricted/Highly Restricted 55 Master Component - Scheduler
- 55. So lets see how the scheduling lifecycle really looks like: 1.A pod is created and its desired state is saved to etcd with the node name unfilled. 2.The scheduler somehow notices that there is a new pod with no node bound. 3.It finds the node that best fits that pod. 4.Tells the apiserver to bind the pod to the node -> saves the new desired state to etcd. 5.Kubelets are watching bound pods through the apiserver, and start the containers on the particular node. https://banzaicloud.com/blog/k8s-custom-scheduler/
- 56. Scheduler - client-go library watch襯 伎 Event襯 覦 FieldSelector spec.nodeName= - 碁螳 豪讌 pod襷 蟯 - Scheduling 蟯 Factor
- 57. Confidential Oracle Internal/Restricted/Highly Restricted 58 Node Component - ETCD
- 58. 則 貎覯ろ一 企ろ一 レ 則 觜襯願, 覿磯覃, 手 Key-Value Store 則 Raft Consensus algorithm 則 企ろ磯 fault-tolerant襯 3,5,7 螻 螳 螳襦 伎 則 譟煙(quorum)
- 59. Master 碁 ETCD Pod襦 れ願 蠏語 etcdctl 豌 覿襯企 譟壱 螳 ETCDCTL_API = 3 etcdctl --cacert =/etc/kubernetes/pki/etcd/ca.crt --cert =/etc/kubernetes/pki/etcd/peer.crt --key =/etc/kubernetes/pki/etcd/peer.key get /registry/namespaces/default -w=json | jq . base64襦 語 蠍 覓語 base64襦 貊伎 曙 . 企 螻襴讀 raft_term:14 覲伎碁.
- 60. Confidential Oracle Internal/Restricted/Highly Restricted 61 Node Component - Kubelet
- 61. 則 Kubelet 則 Kube-proxy 則 Container runtime engine
- 62. Node Agent 覃 覃 Pod Lifecycle 蟯襴. 讌朱 ろ譴 貉企襯 覈磯螻 企欧, 襴る煙 API 覯 覲願 . 1) Pod Management 2) Contiainer Health Check 3) Contiainer Monitoring - Resouce Usage from cAdvisor 則 File path 則 URL HTTP Endpoint 則 API Server
- 63. kubelet 蟲 則 Kubelet Server kube-apiserver 覦 metrics-server 螳 觜 語 API襯 螻牛. 襯 れ, kubectl exec Kubelet API / exec / {token} 牛 貉企 伎狩. Read Only API
- 65. kubelet 蟲 則 貉企 蟯襴 CGroups, QoS, cpuset, device 炎骸 螳 貉企 れ 蟯襴. 則 覲朱エ 蟯襴 ろ 襷, 碁 襦貉 襷危, 襷讌襷朱 襷危 蟆暑襯 貉企 蟆螻 螳 貉企 ろ襴讌 覲朱エ 蟯襴. 則 Eviction 蟯襴 豢覿讌 一 螳 貉企 伎 る 蟆螻 螳 一 螳 貉企 伎 覲伎ロ蠍 貉企 願碓襯 企麹. 則 cAdvisor 貉企 Metrics襯 螻牛. 則 覃碁Ν 覦 糾 覃碁Ν螻 螳 貉企 覦 碁 覃碁Ν 螻牛. / stats / summary襯 牛 豢豢 覯 覃碁Ν HPA レ 蠍磯蓋 . 則 朱 壱 蟯襴 CRI 覦 貉企 覦 覩碁襷 蟯襴襯 企麹 貉企 壱 蟯襴.
- 66. Confidential Oracle Internal/Restricted/Highly Restricted 67 Summary
- 68. Introduction to k8s workshop : https://docs.google.com/presentation/d/1zrfVlE5r61ZNQrmX Kx5gJmBcXnoa_WerHEnTxu5SMco/edit?usp=sharing K8s comprehensive Overview: https://docs.google.com/presentation/d/1_xwLGM6U6EDK59s9Zny- zWGGAbQk47cZPuBblU3Upus/edit#slide=id.g2c3848b8cd_0_158 Kubernetes in Action A Crash Course on Container Orchestration & k8s : https://speakerd.s3.amazonaws.com/presentations/cb7394a1868b479eb6723d120995f259/C ontainer_Orchestration_-_Interop_ITX__17.pdf Pod :https://ssup2.github.io/theory_analysis/Kubernetes_Pod/?fbclid=IwAR1IZtyjusnW1iX8C1 s1c5QpQMigmiH-Y7pFDZ3dwtL44TJ8esDZqp-lXHg Pod : https://www.ianlewis.org/en/what-are-kubernetes-pods-anyway Pod : https://blog.2dal.com/2018/03/28/kubernetes-01-pod/
- 69. Copyright 息 2017, Oracle and/or its affiliates. All rights reserved. | Confidential Oracle Internal/Restricted/Highly Restricted 70