$ k apply -f nginx-deployment.yaml deployment.apps/nginx-deployment created
$ k get po -l app=nginx NAME READY STATUS RESTARTS AGE nginx-deployment-7f8c49f5b4-98fr6 1/1 Running 0 3m15s nginx-deployment-7f8c49f5b4-bp9pq 1/1 Running 0 3m15s nginx-deployment-7f8c49f5b4-pj5v8 1/1 Running 0 3m15s
$ k apply -f pdb.yaml poddisruptionbudget.policy/nginx-deployment created
$ k get pdb NAME MIN AVAILABLE MAX UNAVAILABLE ALLOWED DISRUPTIONS AGE nginx-deployment 1 N/A 2 43s
Development manifests Generate WebhookConfiguration, ClusterRole and CustomResourceDefinition objects. generate Generate code containing DeepCopy, DeepCopyInto, and DeepCopyObject method implementations. fmt Run go fmt against code. vet Run go vet against code. test Run tests.
Build build Build manager binary. run Run a controller from your host. docker-build Build docker image with the manager. docker-push Push docker image with the manager.
Deployment install Install CRDs into the K8s cluster specified in ~/.kube/config. uninstall Uninstall CRDs from the K8s cluster specified in ~/.kube/config. Call with ignore-not-found=true to ignore resource not found errors during deletion. deploy Deploy controller to the K8s cluster specified in ~/.kube/config. undeploy Undeploy controller from the K8s cluster specified in ~/.kube/config. Call with ignore-not-found=true to ignore resource not found errors during deletion.
Build Dependencies kustomize Download kustomize locally if necessary. controller-gen Download controller-gen locally if necessary. envtest Download envtest-setup locally if necessary.
修改 CRD
为 Spec 定义字段 Image
1 2 3 4 5 6 7 8
// MyDaemonSetSpec defines the desired state of MyDaemonSet type MyDaemonSetSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file
// Foo is an example field of MyDaemonSet. Edit mydaemonset_types.go to remove/update Image string`json:"image,omitempty"` }
1 2 3 4 5 6
// MyDaemonSetStatus defines the observed state of MyDaemonSet type MyDaemonSetStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster // Important: Run "make" to regenerate code after modifying this file AvailableReplicas int`json:"availableReplicas,omitempty"` }
重新生成
1 2
$ make generate /Users/zhongmingmao/workspace/go/src/github.com/zhongmingmao/my-operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
1 2
$ make manifests /Users/zhongmingmao/workspace/go/src/github.com/zhongmingmao/my-operator/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
--- apiVersion:apiextensions.k8s.io/v1 kind:CustomResourceDefinition metadata: annotations: controller-gen.kubebuilder.io/version:v0.8.0 creationTimestamp:null name:mydaemonsets.apps.zhongmingmao.io spec: group:apps.zhongmingmao.io names: kind:MyDaemonSet listKind:MyDaemonSetList plural:mydaemonsets singular:mydaemonset scope:Namespaced versions: -name:v1beta1 schema: openAPIV3Schema: description:MyDaemonSetistheSchemaforthemydaemonsetsAPI properties: apiVersion: description:'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type:string kind: description:'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type:string metadata: type:object spec: description:MyDaemonSetSpecdefinesthedesiredstateofMyDaemonSet properties: image: description:FooisanexamplefieldofMyDaemonSet.Editmydaemonset_types.go toremove/update type:string type:object status: description:MyDaemonSetStatusdefinestheobservedstateofMyDaemonSet properties: availableReplicas: description:'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' type:integer type:object type:object served:true storage:true subresources: status: {} status: acceptedNames: kind:"" plural:"" conditions: [] storedVersions: []
$ make build /Users/zhongmingmao/workspace/go/src/github.com/zhongmingmao/my-operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..." go fmt ./... go vet ./... go build -o bin/manager main.go
安装 CRD
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
$ make install GOBIN=/Users/zhongmingmao/workspace/go/src/github.com/zhongmingmao/my-operator/bin go install sigs.k8s.io/controller-tools/cmd/[email protected] /Users/zhongmingmao/workspace/go/src/github.com/zhongmingmao/my-operator/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" | bash -s -- 3.8.7 /Users/zhongmingmao/workspace/go/src/github.com/zhongmingmao/my-operator/bin Version v3.8.7 does not exist for darwin/arm64, trying darwin/amd64 instead. {Version:kustomize/v3.8.7 GitCommit:ad092cc7a91c07fdf63a2e4b7f13fa588a39af4f BuildDate:2020-11-11T23:19:38Z GoOs:darwin GoArch:amd64} kustomize installed to /Users/zhongmingmao/workspace/go/src/github.com/zhongmingmao/my-operator/bin/kustomize /Users/zhongmingmao/workspace/go/src/github.com/zhongmingmao/my-operator/bin/kustomize build config/crd | kubectl apply -f - customresourcedefinition.apiextensions.k8s.io/mydaemonsets.apps.zhongmingmao.io created
$ k api-resources| grep my mydaemonsets apps.zhongmingmao.io/v1beta1 true MyDaemonSet
$ k get crd -owide | grep my mydaemonsets.apps.zhongmingmao.io 2022-12-17T13:46:34Z
$ k get mydaemonsets -A No resources found
运行 Controller
1 2 3 4 5 6 7 8 9 10 11 12 13 14
$ make run GOBIN=/Users/zhongmingmao/workspace/go/src/github.com/zhongmingmao/my-operator/bin go install sigs.k8s.io/controller-tools/cmd/[email protected] /Users/zhongmingmao/workspace/go/src/github.com/zhongmingmao/my-operator/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases /Users/zhongmingmao/workspace/go/src/github.com/zhongmingmao/my-operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..." go fmt ./... go vet ./... go run ./main.go 1.7028209916749592e+09 INFO controller-runtime.metrics Metrics server is starting to listen {"addr": ":8080"} 1.702820991675578e+09 INFO setup starting manager 1.70282099167606e+09 INFO Starting server {"kind": "health probe", "addr": "[::]:8081"} 1.70282099167611e+09 INFO Starting server {"path": "/metrics", "kind": "metrics", "addr": "[::]:8080"} 1.7028209916764538e+09 INFO controller.mydaemonset Starting EventSource {"reconciler group": "apps.zhongmingmao.io", "reconciler kind": "MyDaemonSet", "source": "kind source: *v1beta1.MyDaemonSet"} 1.702820991676507e+09 INFO controller.mydaemonset Starting Controller {"reconciler group": "apps.zhongmingmao.io", "reconciler kind": "MyDaemonSet"} 1.7028209917778409e+09 INFO controller.mydaemonset Starting workers {"reconciler group": "apps.zhongmingmao.io", "reconciler kind": "MyDaemonSet", "worker count": 1}
测试
1 2 3
$ tree config/samples config/samples └── apps_v1beta1_mydaemonset.yaml
$ k apply -f config/samples/apps_v1beta1_mydaemonset.yaml mydaemonset.apps.zhongmingmao.io/mydaemonset-sample created
$ k get mydaemonsets -A NAMESPACE NAME AGE default mydaemonset-sample 22s
$ k get po -owide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES zhongmingmao-b8kzg 1/1 Running 0 2m12s 192.168.216.211 zhongmingmao <none> <none>
$ kubebuilder create webhook --group apps --version v1beta1 --kind MyDaemonSet --defaulting --programmatic-validation Writing kustomize manifests for you to edit... Writing scaffold for you to edit... api/v1beta1/mydaemonset_webhook.go Update dependencies: $ go mod tidy Running make: $ make generate /Users/zhongmingmao/workspace/go/src/github.com/zhongmingmao/my-operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..." Next: implement your new Webhook and generate the manifests with: $ make manifests
1 2 3 4 5 6 7 8 9 10 11
// ValidateCreate implements webhook.Validator so a webhook will be registered for the type func(r *MyDaemonSet) ValidateCreate() error { mydaemonsetlog.Info("validate create", "name", r.Name)
// TODO(user): fill in your validation logic upon object creation. if r.Spec.Image == "" { return fmt.Errorf("image is required") }
returnnil }
1 2
$ make manifests /Users/zhongmingmao/workspace/go/src/github.com/zhongmingmao/my-operator/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
$ make deploy GOBIN=/Users/zhongmingmao/workspace/go/src/github.com/zhongmingmao/my-operator/bin go install sigs.k8s.io/controller-tools/cmd/[email protected] /Users/zhongmingmao/workspace/go/src/github.com/zhongmingmao/my-operator/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases cd config/manager && /Users/zhongmingmao/workspace/go/src/github.com/zhongmingmao/my-operator/bin/kustomize edit set image controller=zhongmingmao/controller:0.0.1 /Users/zhongmingmao/workspace/go/src/github.com/zhongmingmao/my-operator/bin/kustomize build config/default | kubectl apply -f - namespace/my-operator-system created customresourcedefinition.apiextensions.k8s.io/mydaemonsets.apps.zhongmingmao.io configured serviceaccount/my-operator-controller-manager created role.rbac.authorization.k8s.io/my-operator-leader-election-role created clusterrole.rbac.authorization.k8s.io/my-operator-manager-role created clusterrole.rbac.authorization.k8s.io/my-operator-metrics-reader created clusterrole.rbac.authorization.k8s.io/my-operator-proxy-role created rolebinding.rbac.authorization.k8s.io/my-operator-leader-election-rolebinding created clusterrolebinding.rbac.authorization.k8s.io/my-operator-manager-rolebinding created clusterrolebinding.rbac.authorization.k8s.io/my-operator-proxy-rolebinding created configmap/my-operator-manager-config created service/my-operator-controller-manager-metrics-service created service/my-operator-webhook-service created deployment.apps/my-operator-controller-manager created mutatingwebhookconfiguration.admissionregistration.k8s.io/my-operator-mutating-webhook-configuration created validatingwebhookconfiguration.admissionregistration.k8s.io/my-operator-validating-webhook-configuration created
1 2 3 4 5 6 7 8 9 10 11 12 13
$ k get all -n my-operator-system NAME READY STATUS RESTARTS AGE pod/my-operator-controller-manager-57f4b55f6-nh9bq 0/2 ContainerCreating 0 4m20s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/my-operator-controller-manager-metrics-service ClusterIP 10.109.32.208 <none> 8443/TCP 4m20s service/my-operator-webhook-service ClusterIP 10.99.60.181 <none> 443/TCP 4m20s
NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/my-operator-controller-manager 0/1 1 0 4m20s
NAME DESIRED CURRENT READY AGE replicaset.apps/my-operator-controller-manager-57f4b55f6 1 1 0 4m20s
k get po -n my-operator-system my-operator-controller-manager-57f4b55f6-nh9bq -oyaml
