Kubernetes operators automate the deployment and operations of Kubernetes based software. This article describes the Operator Lifecycle Manager which provides a declarative way to install, manage, and upgrade operators on a cluster.
I’m working on an operator sample implemented in Go that shows typical operator patterns. There are instructions how to run the operator:
- Run and debug the operator locally
- Deploy the operator manually to Kubernetes
- Deploy the operator via Operator Lifecycle Manager (focus of this article)
There is a really good video Intro to the Operator Lifecycle Manager describing OLM. Watch it first before reading on.
The Operator SDK and the Operator Framework make it pretty simple to build and deploy operators. Without repeating everything from the video here are the necessary commands and highlights that you need to know. Note that you can also deploy operators via the OLM without using the operator-sdk CLI by using kubectl and yaml files instead. See the bottom of this article.
First the OLM needs to be installed.
$ operator-sdk olm install latest $ kubectl get all -n olm
Next the bundle is created, the bundle image is built and pushed and then the operator is run.
$ export REGISTRY='docker.io' $ export ORG='nheidloff' $ export IMAGE='application-controller:v11' $ make bundle IMG="$REGISTRY/$ORG/$IMAGE"
$ export BUNDLEIMAGE="application-controller-bundle:v11" $ make bundle-build BUNDLE_IMG="$REGISTRY/$ORG/$BUNDLEIMAGE" $ docker push "$REGISTRY/$ORG/$BUNDLEIMAGE" $ operator-sdk run bundle "$REGISTRY/$ORG/$BUNDLEIMAGE" -n operators
The key artifact that is created, is the cluster service version (CSV) which contains all metadata describing the operator, or more precisely, one version of the operator.
apiVersion: operators.coreos.com/v1alpha1 kind: ClusterServiceVersion ... spec: apiservicedefinitions: {} customresourcedefinitions: owned: - displayName: Application kind: Application name: applications.application.sample.ibm.com version: v1alpha1 ... clusterPermissions: - rules: - apiGroups: - application.sample.ibm.com resources: - applications verbs: - create ... deployments: - name: operator-application-controller-manager spec: replicas: 1 ... image: docker.io/nheidloff/application-controller:v10 ... installModes: - supported: true type: AllNamespaces version: 0.0.1
Additionally annotations.yaml is created with defaults that can be overwritten.
annotations: # Core bundle annotations. operators.operatorframework.io.bundle.mediatype.v1: registry+v1 operators.operatorframework.io.bundle.manifests.v1: manifests/ operators.operatorframework.io.bundle.metadata.v1: metadata/ operators.operatorframework.io.bundle.package.v1: operator-application operators.operatorframework.io.bundle.channels.v1: alpha operators.operatorframework.io.metrics.builder: operator-sdk-v1.18.0 operators.operatorframework.io.metrics.mediatype.v1: metrics+v1 operators.operatorframework.io.metrics.project_layout: go.kubebuilder.io/v3
Let’s take a look which Kubernetes resources have been created as result of ‘operator-sdk run bundle’. The CatalogSource contains a link to the bundle image. A catalog is a repository of metadata that the OLM uses to discover and install operators and their dependencies.
$ kubectl get catalogsource -n operators NAME DISPLAY TYPE PUBLISHER AGE operator-application-catalog operator-application grpc operator-sdk 3d1h $ kubectl get catalogsource operator-application-catalog -n operators -oyaml apiVersion: operators.coreos.com/v1alpha1 kind: CatalogSource metadata: annotations: operators.operatorframework.io/index-image: quay.io/operator-framework/opm:latest operators.operatorframework.io/injected-bundles: '[{"imageTag":"docker.io/nheidloff/application-controller-bundle:v11","mode":"semver"}]' operators.operatorframework.io/registry-pod-name: docker-io-nheidloff-application-controller-bundle-v11 ...
Additionally the CSV resource is created which contains the information above plus some state information:
$ kubectl get csv -n operators NAME DISPLAY VERSION REPLACES PHASE operator-application.v0.0.1 operator-application 0.0.1 Succeeded $ kubectl get csv operator-application.v0.0.1 -n operators -oyaml
The subscription resource is the glue between the catalog and the CSV:
kubectl get subscriptions -n operators NAME PACKAGE SOURCE CHANNEL operator-application-v0-0-1-sub operator-application operator-application-catalog alpha $kubectl get subscriptions operator-application-v0-0-1-sub -n operators -oyaml apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: creationTimestamp: "2022-03-21T15:57:40Z" generation: 1 labels: operators.coreos.com/operator-application.operators: "" name: operator-application-v0-0-1-sub namespace: operators spec: channel: alpha installPlanApproval: Manual name: operator-application source: operator-application-catalog sourceNamespace: operators startingCSV: operator-application.v0.0.1
This is the created install plan:
$ kubectl get installplans -n operators $ kubectl get installplans install-xxxxx -n operators -oyaml apiVersion: operators.coreos.com/v1alpha1 kind: InstallPlan metadata: ... name: install-2gxl7 namespace: operators ownerReferences: - apiVersion: operators.coreos.com/v1alpha1 kind: Subscription name: operator-database-v0-0-1-sub ...spec: approval: Manual approved: true clusterServiceVersionNames: - operator-database.v0.0.1 - operator-application.v0.0.1 generation: 1
Last, but not least the operator resource is created.
$ kubectl config set-context --current --namespace=test1 $ kubectl get operators -n operators NAME AGE operator-application.operators 3d2h $ kubectl get operators operator-application.operators -n operators -oyaml apiVersion: operators.coreos.com/v1 kind: Operator metadata: ... manager: olm operation: Update subresource: status time: '2022-03-18T12:48:10Z' name: operator-application.operators ... status: components: labelSelector: matchExpressions: - key: operators.coreos.com/operator-application.operators operator: Exists ... - apiVersion: operators.coreos.com/v1alpha1 conditions: - lastTransitionTime: '2022-03-18T12:48:58Z' lastUpdateTime: '2022-03-18T12:48:58Z' message: install strategy completed with no errors reason: InstallSucceeded status: 'True' type: Succeeded kind: ClusterServiceVersion name: operator-application.v0.0.1 namespace: operators ...
Deployment with kubectl
You can also deploy operators via OLM using kubectl.
$ kubectl apply -f olm/catalogsource.yaml $ kubectl apply -f olm/subscription.yaml $ kubectl get installplans -n operators $ kubectl -n operators patch installplan install-xxxxx -p '{"spec":{"approved":true}}' --type merge
This creates the same resources as above.
$ kubectl get all -n operators $ kubectl get catalogsource operator-application-catalog -n operators -oyaml $ kubectl get subscriptions operator-application-v0-0-1-sub -n operators -oyaml $ kubectl get csv operator-application.v0.0.1 -n operators -oyaml $ kubectl get installplans -n operators $ kubectl get installplans install-xxxxx -n operators -oyaml $ kubectl get operators operator-application.operators -n operators -oyaml
The real value of the OLM is the management of different versions via a subscription model. I’d like to blog about this soon as well as other operator based topics. Check out the repo and keep an eye on my blog.