Scaling our CI/CD Cluster

Cloud Native, Scalable and Observable GitLab Runner on Kubernetes

92%

K3s is a great choice to setup and run your CI/CD environment and it's not just suitable for GitLab Runners. You can use it to run your GitLab instance itself, your CI/CD pipelines, your observability tools and more. The fact that it's easy to manage while not being a resource hog makes it an excellent choice for teams that want to focus on building and deploying applications rather than managing infrastructure. Usually, solutions that are easy to install are not ready for production, let's cite as a matter of example miniKube, Kind and Docker Compose. K3s breaks this rule and is at the same time easy to install and scalable.

In this section, we are going to scale our CI/CD cluster by adding nodes to it. The first node we have - called "dev" - is the master node. Our cluster is running on that single node. To add a new node as a worker, we need first to create the virtual machine and then join it to the cluster.

Previsouly, we used a Terraform module to create the "dev" VM, we are going to use the same file, update it by adding a new VM called "dev2" and apply the changes. As a reminder, we used Terraform from our local machines, therefore, it's required to apply the changes from the same folder where the Terraform files (like "terraform.tfstate") are located (in our case, it's the "$HOME/terraform/digitalocean" folder).

Here is the updated version:

cat < $HOME/terraform/digitalocean/main.tf
terraform {
  required_providers {
    digitalocean = {
      source = "digitalocean/digitalocean"
      version = "~> 2.0"      
    }
  }
}

variable "region" {
  default = "fra1"
}

variable "image" {
  default = "ubuntu-24-04-x64"
}

resource "digitalocean_ssh_key" "my_ssh_key" {
  name       = "learning" 
  public_key = file("~/.ssh/learning.pub")  
}

variable "names" {
  default = [    
    "dev",
    "dev2",
  ]
}

variable "size" {
  default = "s-2vcpu-4gb"
}

resource "digitalocean_vpc" "my_vpc" {
  name   = "my-vpc"        
  region = var.region      
}


resource "digitalocean_droplet" "my_droplets" {
  for_each = { for name in var.names : name => name } 
  image      = var.image                            
  name       = each.value                           
  region     = var.region                           
  size       = var.size                             
  ssh_keys   = [digitalocean_ssh_key.my_ssh_key.id] 
  monitoring = false                               
  vpc_uuid   = digitalocean_vpc.my_vpc.id           
}

resource "digitalocean_project" "learning_project" {
  name        = "Learning"            
  description = "A project to learn and have fun" 
  purpose     = "Learning"            
  environment = "Development"         

  resources = [for droplet in digitalocean_droplet.my_droplets : droplet.urn]
}
EOF

Plan and apply the changes:

cd $HOME/terraform/digitalocean
export DIGITALOCEAN_TOKEN

Cloud Native CI/CD with GitLab

From Commit to Production Ready

Enroll now to unlock all content and receive all future updates for free.

Unlock now ($29.99) Learn More

Previous Next