Job 作用
创建一个或多个 Pod 来确保指定数量的 Pod 可以成功地运行终止
跟踪 Pod 状态,根据配置及时重试失败的 Pod
确定依赖关系,保证上一个任务运行完毕后再运行下一个任务
控制任务并行度,并根据配置确保 Pod 队列大小
实践 YAML 文件 job.yaml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 apiVersion: batch/v1 kind: Job metadata: name: pi spec: backoffLimit: 4 template: spec: containers: - name: pi image: perl ports: null command: - perl - '-Mbignum=bpi' - '-wle' - print bpi(2000) restartPolicy: Never
paral-job.yml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 apiVersion: batch/v1 kind: Job metadata: name: parallel-job spec: completions: 8 parallelism: 2 template: spec: containers: - name: show-date image: ubuntu command: - /bin/sh args: - '-c' - sleep 30 ; date restartPolicy: OnFailure
启动 minikube 1 2 3 4 5 6 7 8 9 10 11 12 13 14 $ minikube start --vm-driver=virtualbox --kubernetes-version=1.15.5 --image-mirror-country='cn' 😄 minikube v1.18.1 on Darwin 10.12.6 ✨ Using the virtualbox driver based on user configuration ✅ Using image repository registry.cn-hangzhou.aliyuncs.com/google_containers 👍 Starting control plane node minikube in cluster minikube 🔥 Creating virtualbox VM (CPUs=2, Memory=4000MB, Disk=20000MB) ... 🐳 Preparing Kubernetes v1.15.5 on Docker 20.10.3 ... ▪ Generating certificates and keys ... ▪ Booting up control plane ... ▪ Configuring RBAC rules ... 🔎 Verifying Kubernetes components... ▪ Using image registry.cn-hangzhou.aliyuncs.com/google_containers/storage-provisioner:v4 (global image repository) 🌟 Enabled addons: storage-provisioner, default-storageclass 🏄 Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
查看 Job 自动添加 Label:controller-uid: 9efacda0-f0a1-41e4-9a5c-de287690ecae
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 $ kubectl apply -f job.yaml job.batch/pi created $ kubectl get jobs NAME COMPLETIONS DURATION AGE pi 1/1 10s 15s $ kubectl get jobs pi -o yaml | head -n 15 apiVersion: batch/v1 kind: Job metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"batch/v1","kind":"Job","metadata":{"annotations":{},"name":"pi","namespace":"default"},"spec":{"backoffLimit":4,"template":{"spec":{"containers":[{"command":["perl","-Mbignum=bpi","-wle","print bpi(2000)"],"image":"perl","name":"pi","ports":null}],"restartPolicy":"Never"}}}} creationTimestamp: "2021-07-12T00:45:44Z" labels: controller-uid: 9efacda0-f0a1-41e4-9a5c-de287690ecae job-name: pi name: pi namespace: default resourceVersion: "5330" selfLink: /apis/batch/v1/namespaces/default/jobs/pi uid: 9efacda0-f0a1-41e4-9a5c-de287690ecae
COMPLETIONS
DURATION
AGE
完成 Pod 数量
Job 实际业务运行时长
Job 创建时长
查看 Pod Pod 名称:**${job-name}-${random-suffix}
**,Pod OwnerReference 为 Job
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 $ kubectl get pods NAME READY STATUS RESTARTS AGE pi-q5b9h 0/1 Completed 0 82s $ kubectl get pod pi-q5b9h -o yaml | head -n 17 apiVersion: v1 kind: Pod metadata: creationTimestamp: "2021-07-12T00:45:44Z" generateName: pi- labels: controller-uid: 9efacda0-f0a1-41e4-9a5c-de287690ecae job-name: pi name: pi-q5b9h namespace: default ownerReferences: - apiVersion: batch/v1 blockOwnerDeletion: true controller: true kind: Job name: pi uid: 9efacda0-f0a1-41e4-9a5c-de287690ecae $ kubectl logs pi-vjzt4 3.14159265358979323846264338327950288419716939937510582097494459230781640628620899862803482534211706798214808651...
并行运行
Key
Value
completions
Job 可以运行的总次数
parallelism
并行执行的个数(管道或缓冲队列的大小)
1 2 3 4 5 6 7 8 9 10 11 12 13 $ kubectl apply -f parallel-job.yml job.batch/parallel-job created $ kubectl get pods NAME READY STATUS RESTARTS AGE parallel-job-4md9g 0/1 Completed 0 53s parallel-job-4pdvc 0/1 Completed 0 2m58s parallel-job-d5mdr 0/1 Completed 0 57s parallel-job-grwdp 0/1 Completed 0 2m7s parallel-job-h2shl 0/1 Completed 0 92s parallel-job-qxfmp 0/1 Completed 0 88s parallel-job-rjkfc 0/1 Completed 0 2m58s parallel-job-vcv8l 0/1 Completed 0 2m4s
架构设计 管理模式
Job Controller
CronJob YAML 文件 CronJob -> Job -> Pod
cronjob.yaml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 apiVersion: batch/v1beta1 kind: CronJob metadata: name: cron-job spec: schedule: '*/3 * * * *' startingDeadlineSeconds: 10 concurrencyPolicy: Allow successfulJobsHistoryLimit: 3 jobTemplate: spec: template: spec: containers: - name: hello image: busybox command: - /bin/sh args: - '-c' - date; echo Hello from the minikube restartPolicy: OnFailure
查看 CronJob 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 $ kubectl apply -f cronjob.yml cronjob.batch/cron-job created $ kubectl get cronjobs NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE cron-job */3 * * * * False 1 7s 2m52s $ kubectl get cronjobs cron-job -o yaml | head -n 12 apiVersion: batch/v1beta1 kind: CronJob metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"batch/v1beta1","kind":"CronJob","metadata":{"annotations":{},"name":"cron-job","namespace":"default"},"spec":{"concurrencyPolicy":"Allow","jobTemplate":{"spec":{"template":{"spec":{"containers":[{"args":["-c","date; echo Hello from the minikube"],"command":["/bin/sh"],"image":"busybox","name":"hello"}],"restartPolicy":"OnFailure"}}}},"schedule":"*/3 * * * *","startingDeadlineSeconds":10,"successfulJobsHistoryLimit":3}} creationTimestamp: "2021-07-12T00:21:15Z" name: cron-job namespace: default resourceVersion: "4084" selfLink: /apis/batch/v1beta1/namespaces/default/cronjobs/cron-job uid: cee590cc-3341-4fea-a2bd-d56337c6d6ca
查看 Job Job 的 OwnerReferences 为 CronJob
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 $ kubectl get jobs NAME COMPLETIONS DURATION AGE cron-job-1626049440 1/1 4s 22s $ kubectl get jobs cron-job-1626049440 -o yaml | head -n 19 apiVersion: batch/v1 kind: Job metadata: creationTimestamp: "2021-07-12T00:24:05Z" labels: controller-uid: e871c373-06ff-4ae7-a6c3-333a5fdd20ed job-name: cron-job-1626049440 name: cron-job-1626049440 namespace: default ownerReferences: - apiVersion: batch/v1beta1 blockOwnerDeletion: true controller: true kind: CronJob name: cron-job uid: cee590cc-3341-4fea-a2bd-d56337c6d6ca resourceVersion: "4223" selfLink: /apis/batch/v1/namespaces/default/jobs/cron-job-1626049440 uid: e871c373-06ff-4ae7-a6c3-333a5fdd20ed
查看 Pod Pod 的 OwnerReferences 为 Job
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 $ kubectl get pods NAME READY STATUS RESTARTS AGE cron-job-1626049440-vfcb8 0/1 Completed 0 60s $ kubectl logs cron-job-1626049440-vfcb8 Mon Jul 12 00:24:09 UTC 2021 Hello from the minikube $ kubectl get pods cron-job-1626049440-vfcb8 -o yaml | head -n 20 apiVersion: v1 kind: Pod metadata: creationTimestamp: "2021-07-12T00:24:05Z" generateName: cron-job-1626049440- labels: controller-uid: e871c373-06ff-4ae7-a6c3-333a5fdd20ed job-name: cron-job-1626049440 name: cron-job-1626049440-vfcb8 namespace: default ownerReferences: - apiVersion: batch/v1 blockOwnerDeletion: true controller: true kind: Job name: cron-job-1626049440 uid: e871c373-06ff-4ae7-a6c3-333a5fdd20ed resourceVersion: "4222" selfLink: /api/v1/namespaces/default/pods/cron-job-1626049440-vfcb8 uid: 5dd30c4c-2fac-4bd0-80e4-9d1fc92d79b9
参考资料
CNCF × Alibaba 云原生技术公开课