Scaling our CI/CD Cluster

Cloud Native GitLab Runners on Kubernetes: Autoscaling and Observability

92%

K3s is a great choice to setup and run your CI/CD environment and it's a lightweight Kubernetes distribution designed for production workloads in resource-constrained environments. You can use it to run your GitLab instance itself, your CI/CD pipelines, your observability tools and more. The fact that it's easy to manage while not being a resource hog makes it an excellent choice for teams that want to focus on building and deploying applications rather than managing infrastructure. Usually, solutions that are easy to install are, by design, not ready for production, let's cite as a matter of example miniKube, Kind and Docker Compose. K3s breaks this rule and is at the same time easy to install and scalable.

In this section, we are going to scale our CI/CD cluster by adding nodes to it. The first node we have - called "dev" - is the master node. Our cluster is running on that single node. To add a new node as a worker, we need first to create the virtual machine and then join it to the cluster.

Previsouly, we used a Terraform module to create the "dev" VM, we are going to use the same file, update it by adding a new VM called "dev2" and apply the changes. As a reminder, we used Terraform from our local machines, therefore, it's required to apply the changes from the same folder where the Terraform files (like "terraform.tfstate") are located (in our case, it's the "$HOME/terraform/digitalocean" folder).

Here is the updated version:

cat < $HOME/terraform/digitalocean/variables.tf
variable "region" {
  // Default region for deploying the resources.
  default = "fra1"
}

variable "image" {
  // Default image for the droplets.
  default = "ubuntu-24-04-x64"
}

// Define a list of names for the droplets
variable "names" {
  // A list of VMs names to be created.
  default = [
    "dev",
    "dev2"
  ]
}

variable "size" {
  // Default size for the VM
  default = "s-2vcpu-4gb"
}

variable "ssh_key_name" {
  description = "Name for the SSH key resource"
  default     = "learning"
}

variable "ssh_public_key_path" {
  description = "Path to the SSH public key file"
  default     = "~/.ssh/learning.pub"
}

variable "vpc_name" {
  description = "Name for the VPC"
  default     = "my-vpc"
}

variable "project_name" {
  description = "Name for the DigitalOcean project"
  default     = "Learning"
}

variable "project_description" {
  description = "Description of the DigitalOcean project"
  default     = "A project to learn and have fun"
}

variable "project_purpose" {
  description = "Purpose of the DigitalOcean project"
  default     = "Learning"
}

variable "project_environment" {
  description = "Environment for the DigitalOcean project (e.g., Development)"
  default     = "Development"
}
EOF

Plan and apply the changes:

cd $HOME/terraform/digitalocean

# Set the DigitalOcean token as an environment variable
export DIGITALOCEAN_TOKEN=

# Initialize Terraform
terraform init

# Preview the changes
terraform plan

# Apply the changes
terraform apply

# Show the IP addresses of the VMs
terraform show | grep "\bipv4_address\b"

Cloud Native CI/CD with GitLab

From Commit to Production Ready

Enroll now to unlock all content and receive all future updates for free.

Unlock now $29.99 Learn More

Previous Next