Testing Kubebuilder Operators

Goal

We are going to write a test that will do the following:

• Run our controller in the background
• Create a stateless AbstractWorkload
• Check for a created Deployment
• Delete the Deployment
• Check for a created Deployment
• Change the replica number
• Check for an updated Deployment

Setting up the environment

Kubebuilder uses Kubernetes’ envtest package. With this package, we can run Kubernetes’ control plane locally, without the need to install a cluster.

Next to your controller go file, Kubebuilder generated another file: controllers/suite_test.go — this file is responsible for setting up the test environment and configuring and initializing anything needed before and after our tests.

The BeforeSuite is a function responsible for setting up the environment before the testing start to run. Let’s talk about what happens there:

First, it creates an envtest environment configuration. the CRDDirectoryPaths should contain a list of paths to custom CRD we want our control plane to know.

At the time of writing, Kubebuilder doesn’t have good support for remote CRDs. Sadly, I don’t have any recommendations for this problem at the moment (You can download the remote CRDs and store them locally, but that’s not a long-term solution).

Then we are starting our control plane, update our runtime scheme and initialize a Kubernetes client with that scheme — we will use it in our tests:

The AfterSuite is a function responsible for tearing up the environment after the testing finish run. It is just stopping the envtest environment.

We’re missing one important thing: running our controller. We will need to add this part by ourselves, It's a good question why Kubebuilder didn’t generate a default behavior for us.

First of all, let’s make sure we all have a common ground for that one. You should have those variables at the top. If you don’t have, make sure you add them:

Add this line at the start of the BeforeSuite function:

add this line at the start of the AfterSuite function:

Now we can initialize our manager, register our controller and run it in a new goroutine so it won’t block any cleanup. At the end of the BeforeSuite:

We set up the environment, now we’re ready to start creating our test!

Testing the Controller

We will need to create a new test file for our controller: controllers/abstractworkload_controller_test.go

We’re going to use the Ginkgo framework which is yet another testing framework. Since Kubebuilder uses it in their documentation, I would suggest you use it.

I’m adding here the final imports for you to use since GoLand likes to delete unused imports:

We will describe the testing topic we’re working on. Create some helpful variables and created the context for our testing. Then stating what should happen:

This is a very general test, in real life, you should be more descriptive and modular with your testing, checking just one thing at a time instead of the entire workflow.

we can now declare how we check this statement. Inside the It statement:

Create a stateless AbstractWorkload :

Check a Deployment is created with the right spec:

Check if AbstractWorkload will create a new Deployment after it is deleted:

Change the replica number and check for an updated Deployment

After we have everything written and compiled, we can run our testing with a simple Makefile command:

If you followed everything and did it correctly, tests should pass.

Summary

We now know how to test our controller logic, and created confidence in our operator before deploying to production.

We learned how to set up our local integration testing environment using envtest, how to work with some testing frameworks, and how to interact with the API server using a Kubernetes client.

More tests can be added (check the OwnerReference implementation, check the AbstractWorkload status after creation).

WE ARE NOW READY TO DEPLOY OUR CONTROLLER TO A RUNNING KUBERNETES CLUSTER.

In the next parts, I will guide you through the controller’s deployment process and options. We will cover the cert-manager and webhooks options.

For any questions you have, you can reach me via Linkedin.

If you like my content and want to improve your skills, please take a look at the Devops Culture Project and get all the resources you didn’t know you need.