Design and Implementation

Build the Docker Image

We need a unique Docker image for each branch. We use the branch name and the commit SHA to create a unique tag.

Both CI_COMMIT_REF_SLUG and CI_COMMIT_SHORT_SHA are predefined GitLab CI/CD variables that represent the branch name and the commit SHA. We use them to create a unique tag for the Docker image.

We also create a tar file containing the Docker image and make it available to the next job through artifacts.

variables:
    # A unique ID for the deployment
    DEPLOYMENT_ID: $CI_COMMIT_REF_SLUG-$CI_COMMIT_SHORT_SHA
    # The name of the Docker image
    IMAGE_NAME: $CI_REGISTRY_IMAGE:$DEPLOYMENT_ID

build_docker_image:
    stage: build
    image: docker:24.0.5
    services:
        - docker:24.0.5-dind
    script:
        # Build the Docker image
        - docker build -t $IMAGE_NAME .
        # Create a directory in the project workspace
        - mkdir -p ./docker-artifacts
        # Save the image in a relative path
        - docker save -o ./docker-artifacts/image.tar $IMAGE_NAME
    artifacts:
        paths:
            # Make the tar file available for the next job
            - docker-artifacts/image.tar

(i) The services keyword provides supporting infrastructure or dependencies that the primary container needs during the execution of a job. In our case, docker:24.0.5-dind is a Docker-in-Docker service that lets us run Docker commands inside the primary container.

Push the Docker Image to the GitLab Registry

In this step, we push the Docker image to registry.gitlab.com. We use the predefined GitLab CI/CD variables CI_REGISTRY_USER, CI_REGISTRY_PASSWORD, and CI_REGISTRY to authenticate.

push_docker_image_to_gitlab_registry:
    stage: push
    image: docker:24.0.5
    services:
        - docker:24.0.5-dind
    script:
        # Load the image from the tar file
        - docker load -i ./docker-artifacts/image.tar
        # Login to the GitLab registry
        - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
        # Push the image to the GitLab registry
        - docker push $IMAGE_NAME
    needs:
        # The build_docker_image job must complete before this job starts
        - job: build_docker_image

(i) needs defines the dependencies of a job. The push_docker_image_to_gitlab_registry job depends on build_docker_image, so it only starts after build_docker_image completes.

Create a Helm Package

In this step, we need to achieve the following:

Change the chart version to make it unique per commit.
Update appVersion in Chart.yaml with the deployment ID for traceability.
Change the image tag in values.yaml.
Create a Helm package for the chart.
Make the Helm package available to the next job through artifacts.

variables:
    # A unique ID for the deployment
    DEPLOYMENT_ID: $CI_COMMIT_REF_SLUG-$CI_COMMIT_SHORT_SHA
    # The name of the Docker image
    IMAGE_NAME: $CI_REGISTRY_IMAGE:$DEPLOYMENT_ID
    # The path to the Helm package folder
    HELM_PACKAGE_FOLDER: $CI_PROJECT_DIR/manifests/helm/todo
    # The version of the Helm chart, made unique per commit
    HELM_CHART_VERSION: 0.1.0-$DEPLOYMENT_ID

build_helm_package:
    stage: build
    image: okteto/helm:3.16.2
    script:
        - 'cd $HELM_PACKAGE_FOLDER'
        # Remove any existing chart package to avoid conflicts
        - 'rm -f *.tgz'
        # Change the chart version in the Chart.yaml file
        - 'sed -i "s/^version:.*/version: $HELM_CHART_VERSION/" Chart.yaml'
        # Change the chart appVersion in the Chart.yaml file
        - 'sed -i "s/^appVersion:.*/appVersion: $DEPLOYMENT_ID/" Chart.yaml'
        # Change the image tag in the values.yaml file
        - 'sed -i "s/^  tag:.*/  tag: $DEPLOYMENT_ID/" values.yaml'
        # Package the chart
        - 'helm package .'
    artifacts:
        paths:
            - $HELM_PACKAGE_FOLDER/*.tgz

We use the branch name, the commit SHA, and the chart version to create a unique chart version per commit. With version: 0.1.0 as the base, a commit on feature-xyz with short SHA abc1234 produces a chart version 0.1.0-feature-xyz-abc1234. The package file then becomes todo-0.1.0-feature-xyz-abc1234.tgz.

This is the general structure of the package file name helm package produces:

-.tgz

Where comes from name in Chart.yaml (here, todo) and comes from version in Chart.yaml. The chart name does not change per commit; the chart version does.

Reminder: Helm uses name and version from Chart.yaml to build the package file name. For example, if Chart.yaml contains:

apiVersion: v2
name: todo
description: A Helm chart to install the todo app
type: application
version: 0.2.0
appVersion: "v1"

The package file name is todo-0.2.0.tgz.

We rewrite version per commit so each pipeline run produces a chart version the registry has not seen before. Reusing the same version across commits is unreliable: the GitLab Helm registry accepts the duplicate upload, but client and repo-server caches make it uncertain which copy gets pulled on the next sync. Unique versions side-step that.

We also change appVersion in Chart.yaml to include the deployment ID. This has no impact on the package name or our CI/CD pipeline, but it is useful for tracking and reporting.

Push the Helm Package to the GitLab Registry

The previous step created a Helm package for the application and passed it to the next job through artifacts. In this step, we push the Helm package to the GitLab Helm Package Registry. We use the GitLab API to upload the package.

We need to define these variables:

HELM_CHANNEL: the channel name (stable, dev, test, and so on).
HELM_PACKAGE_REGISTRY_API_ENDPOINT: the URL of the Helm API endpoint.

variables:
    # A unique ID for the deployment
    DEPLOYMENT_ID: $CI_COMMIT_REF_SLUG-$CI_COMMIT_SHORT_SHA
    # The name of the Docker image
    IMAGE_NAME: $CI_REGISTRY_IMAGE:$DEPLOYMENT_ID
    # The path to the Helm package folder
    HELM_PACKAGE_FOLDER: $CI_PROJECT_DIR/manifests/helm/todo
    # The channel name (stable, dev, test, etc.)
    HELM_CHANNEL: "stable"
    # The URL for the Helm API endpoint
    HELM_PACKAGE_REGISTRY_API_ENDPOINT: $CI_API_V4_URL/projects/$CI_PROJECT_ID/packages/helm/api/$HELM_CHANNEL/charts

push_helm_package_to_gitlab_package_registry:
    stage: push
    image: okteto/helm:3.16.2
    script:
        # Export the path to the Helm package
        - 'export HELM_PACKAGE_PATH=$(ls -d $HELM_PACKAGE_FOLDER/*.tgz)'
        # Upload the Helm chart to the GitLab package registry
        - |
          curl --fail-with-body \
            --request POST \
            --form "chart=@$HELM_PACKAGE_PATH" \
            --user gitlab-ci-token:$CI_JOB_TOKEN \
            "$HELM_PACKAGE_REGISTRY_API_ENDPOINT"            
    needs:
        - job: build_helm_package

As a reminder, to push a Helm package to the GitLab registry, we use the GitLab API and post to the Helm API endpoint. The URL is:

https://gitlab.com/api/v4/projects//packages/helm/api//charts

Where is the project ID, is the channel name (e.g. stable, dev, test, etc.), and charts is the endpoint for uploading. This URL is built from predefined GitLab CI/CD variables:

$CI_API_V4_URL/projects/$CI_PROJECT_ID/packages/helm/api/$HELM_CHANNEL/charts

Where $CI_API_V4_URL is a predefined variable that contains the URL of the GitLab API (v4), $CI_PROJECT_ID is the project ID, and $HELM_CHANNEL is the channel name.

In our previous examples, we used the following curl command:

curl \
    --fail-with-body \
    --request POST \
    --form 'chart=@todo-0.1.0.tgz' \
    --user $GITLAB_USERNAME:$GITLAB_TOKEN \
    $HELM_CHARTS_URL

Where $GITLAB_USERNAME and $GITLAB_TOKEN are the GitLab username and the GitLab token. In a pipeline, however, we use the predefined variable $CI_JOB_TOKEN and the literal username gitlab-ci-token to authenticate with the API.

(i) gitlab-ci-token is a special predefined username for authenticating with the GitLab API from inside a pipeline.

Update the Argo CD Application

Since the Docker image and the Helm package are now available in the GitLab Container Registry and the Helm Package Registry, we can update the Argo CD Application to deploy the new version of the application to a different namespace. The steps:

Replace the variables in the Application manifest template.
Apply the new manifest to update the Argo CD Application.

As a reminder, this is the template we use:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  # A unique name for the application
  name: $ARGOCD_APP_NAME
  namespace: argocd
spec:
  destination:
    # The namespace where the application will be deployed
    namespace: $HELM_NAMESPACE
    server: https://kubernetes.default.svc
  project: default
  source:
    chart: todo
    # The Helm repository URL
    repoURL: $HELM_REPO_URL
    # The target revision of the Helm chart
    targetRevision: $HELM_TARGET_REVISION
    helm:
      valueFiles:
      - values.yaml
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
      allowEmpty: false
    syncOptions:
      - CreateNamespace=true

ARGOCD_APP_NAME, HELM_NAMESPACE, HELM_REPO_URL, and HELM_TARGET_REVISION are variables we define in the pipeline. We use envsubst to substitute them in the manifest file.

This is the YAML for the job that updates the Argo CD Application:

variables:
    # A unique ID for the deployment
    DEPLOYMENT_ID: $CI_COMMIT_REF_SLUG-$CI_COMMIT_SHORT_SHA
    # The name of the Docker image
    IMAGE_NAME: $CI_REGISTRY_IMAGE:$DEPLOYMENT_ID
    # The path to the Helm package folder
    HELM_PACKAGE_FOLDER: $CI_PROJECT_DIR/manifests/helm/todo
    # The channel name (stable, dev, test, etc.)
    HELM_CHANNEL: "stable"
    # The URL for the Helm API endpoint
    HELM_PACKAGE_REGISTRY_API_ENDPOINT: $CI_API_V4_URL/projects/$CI_PROJECT_ID/packages/helm/api/$HELM_CHANNEL/charts
    # The URL for the Helm package registry
    HELM_PACKAGE_REGISTRY_URL: $CI_API_V4_URL/projects/$CI_PROJECT_ID/packages/helm/$HELM_CHANNEL
    # The URL for the Helm repository
    HELM_REPO_URL: $HELM_PACKAGE_REGISTRY_URL
    # The namespace where the application will be deployed
    HELM_NAMESPACE: $CI_COMMIT_REF_SLUG
    # The name of the Argo CD application
    ARGOCD_APP_NAME: todo-$DEPLOYMENT_ID
    # The version of the Helm chart, made unique per commit
    HELM_CHART_VERSION: 0.1.0-$DEPLOYMENT_ID
    # The target revision of the Helm chart
    HELM_TARGET_REVISION

GitOps the Hard Way, with Argo CD

Build Real GitOps Pipelines From Empty Clusters to Automated Deploys

Enroll now to unlock all content and receive all future updates for free.

Unlock now $20.99 Learn More

Previous Next