It has been 687 days since the last update, the content of the article may be outdated.
Job 作用
创建一个或多个 Pod 来确保指定数量的 Pod 可以成功地运行终止
跟踪 Pod 状态,根据配置及时重试失败的 Pod
确定依赖关系,保证上一个任务运行完毕后再运行下一个任务
控制任务并行度,并根据配置确保 Pod 队列大小
实践 YAML 文件
job.yaml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 apiVersion: batch/v1 kind: Job metadata: name: pi spec: backoffLimit: 4 template: spec: containers: - name: pi image: perl ports: null command: - perl - '-Mbignum=bpi' - '-wle' - print bpi(2000) restartPolicy: Never
paral-job.yml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 apiVersion: batch/v1 kind: Job metadata: name: parallel-job spec: completions: 8 parallelism: 2 template: spec: containers: - name: show-date image: ubuntu command: - /bin/sh args: - '-c' - sleep 30 ; date restartPolicy: OnFailure
启动 minikube 1 2 3 4 5 6 7 8 9 10 11 12 13 14 $ minikube start --vm-driver=virtualbox --kubernetes-version=1.15.5 --image-mirror-country='cn' 😄 minikube v1.18.1 on Darwin 10.12.6 ✨ Using the virtualbox driver based on user configuration ✅ Using image repository registry.cn-hangzhou.aliyuncs.com/google_containers 👍 Starting control plane node minikube in cluster minikube 🔥 Creating virtualbox VM (CPUs=2, Memory=4000MB, Disk=20000MB) ... 🐳 Preparing Kubernetes v1.15.5 on Docker 20.10.3 ... ▪ Generating certificates and keys ... ▪ Booting up control plane ... ▪ Configuring RBAC rules ... 🔎 Verifying Kubernetes components... ▪ Using image registry.cn-hangzhou.aliyuncs.com/google_containers/storage-provisioner:v4 (global image repository) 🌟 Enabled addons: storage-provisioner, default-storageclass 🏄 Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
查看 Job 自动添加 Label:controller-uid: 9efacda0-f0a1-41e4-9a5c-de287690ecae
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 $ kubectl apply -f job.yaml job.batch/pi created $ kubectl get jobs NAME COMPLETIONS DURATION AGE pi 1/1 10s 15s $ kubectl get jobs pi -o yaml | head -n 15 apiVersion: batch/v1 kind: Job metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"batch/v1","kind":"Job","metadata":{"annotations":{},"name":"pi","namespace":"default"},"spec":{"backoffLimit":4,"template":{"spec":{"containers":[{"command":["perl","-Mbignum=bpi","-wle","print bpi(2000)"],"image":"perl","name":"pi","ports":null}],"restartPolicy":"Never"}}}} creationTimestamp: "2021-07-12T00:45:44Z" labels: controller-uid: 9efacda0-f0a1-41e4-9a5c-de287690ecae job-name: pi name: pi namespace: default resourceVersion: "5330" selfLink: /apis/batch/v1/namespaces/default/jobs/pi uid: 9efacda0-f0a1-41e4-9a5c-de287690ecae
COMPLETIONS
DURATION
AGE
完成 Pod 数量
Job 实际业务运行时长
Job 创建时长
查看 Pod Pod 名称:**${job-name}-${random-suffix}
**,Pod OwnerReference 为 Job
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 $ kubectl get pods NAME READY STATUS RESTARTS AGE pi-q5b9h 0/1 Completed 0 82s $ kubectl get pod pi-q5b9h -o yaml | head -n 17 apiVersion: v1 kind: Pod metadata: creationTimestamp: "2021-07-12T00:45:44Z" generateName: pi- labels: controller-uid: 9efacda0-f0a1-41e4-9a5c-de287690ecae job-name: pi name: pi-q5b9h namespace: default ownerReferences: - apiVersion: batch/v1 blockOwnerDeletion: true controller: true kind: Job name: pi uid: 9efacda0-f0a1-41e4-9a5c-de287690ecae $ kubectl logs pi-vjzt4 3.14159265358979323846264338327950288419716939937510582097494459230781640628620899862803482534211706798214808651...
并行运行
Key
Value
completions
Job 可以运行的总次数
parallelism
并行执行的个数(管道或缓冲队列的大小)
1 2 3 4 5 6 7 8 9 10 11 12 13 $ kubectl apply -f parallel-job.yml job.batch/parallel-job created $ kubectl get pods NAME READY STATUS RESTARTS AGE parallel-job-4md9g 0/1 Completed 0 53s parallel-job-4pdvc 0/1 Completed 0 2m58s parallel-job-d5mdr 0/1 Completed 0 57s parallel-job-grwdp 0/1 Completed 0 2m7s parallel-job-h2shl 0/1 Completed 0 92s parallel-job-qxfmp 0/1 Completed 0 88s parallel-job-rjkfc 0/1 Completed 0 2m58s parallel-job-vcv8l 0/1 Completed 0 2m4s
架构设计 管理模式
image-20210712082830228
Job Controller
image-20210712083006743
CronJob YAML 文件 CronJob -> Job -> Pod
cronjob.yaml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 apiVersion: batch/v1beta1 kind: CronJob metadata: name: cron-job spec: schedule: '*/3 * * * *' startingDeadlineSeconds: 10 concurrencyPolicy: Allow successfulJobsHistoryLimit: 3 jobTemplate: spec: template: spec: containers: - name: hello image: busybox command: - /bin/sh args: - '-c' - date; echo Hello from the minikube restartPolicy: OnFailure
查看 CronJob
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 $ kubectl apply -f cronjob.yml cronjob.batch/cron-job created $ kubectl get cronjobs NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE cron-job */3 * * * * False 1 7s 2m52s $ kubectl get cronjobs cron-job -o yaml | head -n 12 apiVersion: batch/v1beta1 kind: CronJob metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"batch/v1beta1","kind":"CronJob","metadata":{"annotations":{},"name":"cron-job","namespace":"default"},"spec":{"concurrencyPolicy":"Allow","jobTemplate":{"spec":{"template":{"spec":{"containers":[{"args":["-c","date; echo Hello from the minikube"],"command":["/bin/sh"],"image":"busybox","name":"hello"}],"restartPolicy":"OnFailure"}}}},"schedule":"*/3 * * * *","startingDeadlineSeconds":10,"successfulJobsHistoryLimit":3}} creationTimestamp: "2021-07-12T00:21:15Z" name: cron-job namespace: default resourceVersion: "4084" selfLink: /apis/batch/v1beta1/namespaces/default/cronjobs/cron-job uid: cee590cc-3341-4fea-a2bd-d56337c6d6ca
查看 Job Job 的 OwnerReferences 为 CronJob
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 $ kubectl get jobs NAME COMPLETIONS DURATION AGE cron-job-1626049440 1/1 4s 22s $ kubectl get jobs cron-job-1626049440 -o yaml | head -n 19 apiVersion: batch/v1 kind: Job metadata: creationTimestamp: "2021-07-12T00:24:05Z" labels: controller-uid: e871c373-06ff-4ae7-a6c3-333a5fdd20ed job-name: cron-job-1626049440 name: cron-job-1626049440 namespace: default ownerReferences: - apiVersion: batch/v1beta1 blockOwnerDeletion: true controller: true kind: CronJob name: cron-job uid: cee590cc-3341-4fea-a2bd-d56337c6d6ca resourceVersion: "4223" selfLink: /apis/batch/v1/namespaces/default/jobs/cron-job-1626049440 uid: e871c373-06ff-4ae7-a6c3-333a5fdd20ed
查看 Pod Pod 的 OwnerReferences 为 Job
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 $ kubectl get pods NAME READY STATUS RESTARTS AGE cron-job-1626049440-vfcb8 0/1 Completed 0 60s $ kubectl logs cron-job-1626049440-vfcb8 Mon Jul 12 00:24:09 UTC 2021 Hello from the minikube $ kubectl get pods cron-job-1626049440-vfcb8 -o yaml | head -n 20 apiVersion: v1 kind: Pod metadata: creationTimestamp: "2021-07-12T00:24:05Z" generateName: cron-job-1626049440- labels: controller-uid: e871c373-06ff-4ae7-a6c3-333a5fdd20ed job-name: cron-job-1626049440 name: cron-job-1626049440-vfcb8 namespace: default ownerReferences: - apiVersion: batch/v1 blockOwnerDeletion: true controller: true kind: Job name: cron-job-1626049440 uid: e871c373-06ff-4ae7-a6c3-333a5fdd20ed resourceVersion: "4222" selfLink: /api/v1/namespaces/default/pods/cron-job-1626049440-vfcb8 uid: 5dd30c4c-2fac-4bd0-80e4-9d1fc92d79b9
参考资料
CNCF × Alibaba 云原生技术公开课