Kubernetes operators automate the deployment and operations of Kubernetes based software. This article describes the Operator Lifecycle Manager which provides a declarative way to install, manage, and upgrade operators on a cluster.
I’m working on an operator sample implemented in Go that shows typical operator patterns. There are instructions how to run the operator:
- Run and debug the operator locally
- Deploy the operator manually to Kubernetes
- Deploy the operator via Operator Lifecycle Manager (focus of this article)
There is a really good video Intro to the Operator Lifecycle Manager describing OLM. Watch it first before reading on.
The Operator SDK and the Operator Framework make it pretty simple to build and deploy operators. Without repeating everything from the video here are the necessary commands and highlights that you need to know. Note that you can also deploy operators via the OLM without using the operator-sdk CLI by using kubectl and yaml files instead. See the bottom of this article.
First the OLM needs to be installed.
1
2
$ operator-sdk olm install latest
$ kubectl get all -n olm
Next the bundle is created, the bundle image is built and pushed and then the operator is run.
1
2
3
4
$ export REGISTRY='docker.io'
$ export ORG='nheidloff'
$ export IMAGE='application-controller:v11'
$ make bundle IMG="$REGISTRY/$ORG/$IMAGE"
1
2
3
4
$ export BUNDLEIMAGE="application-controller-bundle:v11"
$ make bundle-build BUNDLE_IMG="$REGISTRY/$ORG/$BUNDLEIMAGE"
$ docker push "$REGISTRY/$ORG/$BUNDLEIMAGE"
$ operator-sdk run bundle "$REGISTRY/$ORG/$BUNDLEIMAGE" -n operators
The key artifact that is created, is the cluster service version (CSV) which contains all metadata describing the operator, or more precisely, one version of the operator.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
apiVersion: operators.coreos.com/v1alpha1
kind: ClusterServiceVersion
...
spec:
apiservicedefinitions: {}
customresourcedefinitions:
owned:
- displayName: Application
kind: Application
name: applications.application.sample.ibm.com
version: v1alpha1
...
clusterPermissions:
- rules:
- apiGroups:
- application.sample.ibm.com
resources:
- applications
verbs:
- create
...
deployments:
- name: operator-application-controller-manager
spec:
replicas: 1
...
image: docker.io/nheidloff/application-controller:v10
...
installModes:
- supported: true
type: AllNamespaces
version: 0.0.1
Additionally annotations.yaml is created with defaults that can be overwritten.
1
2
3
4
5
6
7
8
9
10
annotations:
# Core bundle annotations.
operators.operatorframework.io.bundle.mediatype.v1: registry+v1
operators.operatorframework.io.bundle.manifests.v1: manifests/
operators.operatorframework.io.bundle.metadata.v1: metadata/
operators.operatorframework.io.bundle.package.v1: operator-application
operators.operatorframework.io.bundle.channels.v1: alpha
operators.operatorframework.io.metrics.builder: operator-sdk-v1.18.0
operators.operatorframework.io.metrics.mediatype.v1: metrics+v1
operators.operatorframework.io.metrics.project_layout: go.kubebuilder.io/v3
Let’s take a look which Kubernetes resources have been created as result of ‘operator-sdk run bundle’. The CatalogSource contains a link to the bundle image. A catalog is a repository of metadata that the OLM uses to discover and install operators and their dependencies.
1
2
3
4
5
6
7
8
9
10
11
12
$ kubectl get catalogsource -n operators
NAME DISPLAY TYPE PUBLISHER AGE
operator-application-catalog operator-application grpc operator-sdk 3d1h
$ kubectl get catalogsource operator-application-catalog -n operators -oyaml
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
annotations:
operators.operatorframework.io/index-image: quay.io/operator-framework/opm:latest
operators.operatorframework.io/injected-bundles: '[{"imageTag":"docker.io/nheidloff/application-controller-bundle:v11","mode":"semver"}]'
operators.operatorframework.io/registry-pod-name: docker-io-nheidloff-application-controller-bundle-v11
...
Additionally the CSV resource is created which contains the information above plus some state information:
1
2
3
4
$ kubectl get csv -n operators
NAME DISPLAY VERSION REPLACES PHASE
operator-application.v0.0.1 operator-application 0.0.1 Succeeded
$ kubectl get csv operator-application.v0.0.1 -n operators -oyaml
The subscription resource is the glue between the catalog and the CSV:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
kubectl get subscriptions -n operators
NAME PACKAGE SOURCE CHANNEL
operator-application-v0-0-1-sub operator-application operator-application-catalog alpha
$kubectl get subscriptions operator-application-v0-0-1-sub -n operators -oyaml
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
creationTimestamp: "2022-03-21T15:57:40Z"
generation: 1
labels:
operators.coreos.com/operator-application.operators: ""
name: operator-application-v0-0-1-sub
namespace: operators
spec:
channel: alpha
installPlanApproval: Manual
name: operator-application
source: operator-application-catalog
sourceNamespace: operators
startingCSV: operator-application.v0.0.1
This is the created install plan:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
$ kubectl get installplans -n operators
$ kubectl get installplans install-xxxxx -n operators -oyaml
apiVersion: operators.coreos.com/v1alpha1
kind: InstallPlan
metadata:
...
name: install-2gxl7
namespace: operators
ownerReferences:
- apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
name: operator-database-v0-0-1-sub
...spec:
approval: Manual
approved: true
clusterServiceVersionNames:
- operator-database.v0.0.1
- operator-application.v0.0.1
generation: 1
Last, but not least the operator resource is created.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
$ kubectl config set-context --current --namespace=test1
$ kubectl get operators -n operators
NAME AGE
operator-application.operators 3d2h
$ kubectl get operators operator-application.operators -n operators -oyaml
apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
...
manager: olm
operation: Update
subresource: status
time: '2022-03-18T12:48:10Z'
name: operator-application.operators
...
status:
components:
labelSelector:
matchExpressions:
- key: operators.coreos.com/operator-application.operators
operator: Exists
...
- apiVersion: operators.coreos.com/v1alpha1
conditions:
- lastTransitionTime: '2022-03-18T12:48:58Z'
lastUpdateTime: '2022-03-18T12:48:58Z'
message: install strategy completed with no errors
reason: InstallSucceeded
status: 'True'
type: Succeeded
kind: ClusterServiceVersion
name: operator-application.v0.0.1
namespace: operators
...
Deployment with kubectl
You can also deploy operators via OLM using kubectl.
1
2
3
4
$ kubectl apply -f olm/catalogsource.yaml
$ kubectl apply -f olm/subscription.yaml
$ kubectl get installplans -n operators
$ kubectl -n operators patch installplan install-xxxxx -p '{"spec":{"approved":true}}' --type merge
This creates the same resources as above.
1
2
3
4
5
6
7
$ kubectl get all -n operators
$ kubectl get catalogsource operator-application-catalog -n operators -oyaml
$ kubectl get subscriptions operator-application-v0-0-1-sub -n operators -oyaml
$ kubectl get csv operator-application.v0.0.1 -n operators -oyaml
$ kubectl get installplans -n operators
$ kubectl get installplans install-xxxxx -n operators -oyaml
$ kubectl get operators operator-application.operators -n operators -oyaml
The real value of the OLM is the management of different versions via a subscription model. I’d like to blog about this soon as well as other operator based topics. Check out the repo and keep an eye on my blog.