Parallelism and Resource Groups: Improving Pipeline Performance and Resource Utilization
Parallel Matrix Builds
Another way to accelerate your pipeline is to use matrix builds.
A matrix build is a way to run the same job multiple times with different configurations. Let's say we want to build our application with different versions of Python: 3.9, 3.10, 3.11 and 3.12. We can use a matrix build to run the build job with different versions of Python.
Initially, since we are using jobs running on Docker, we'll need to use different images for each version of Python. The following table shows the Python version and the corresponding Docker image name:
| Python version | Docker image |
|---|---|
| 3.9 | python:3.9 |
| 3.10 | python:3.10 |
| 3.11 | python:3.11 |
| 3.12 | python:3.12 |
This is how it can be done:
stages:
- build
build-3.9:
stage: build
image: python:3.9
script:
- pip install -r requirements.txt
build-3.10:
stage: build
image: python:3.10
script:
- pip install -r requirements.txt
build-3.11:
stage: build
image: python:3.11
script:
- pip install -r requirements.txt
build-3.12:
stage: build
image: python:3.12
script:
- pip install -r requirements.txt
However, there's a better way to do this using the matrix keyword. Test the corresponding pipeline with the following command:
cat <$HOME/todo/app/.gitlab-ci.yml && \
cd $HOME/todo/app && \
git add . && \
git commit -m "Add matrix" && \Cloud Native CI/CD with GitLab
From Commit to Production ReadyEnroll now to unlock all content and receive all future updates for free.
