Cloud Native CI/CD with GitLab

Making Your Pipeline Faster Using Artifacts

In our previous example, we installed the dependencies of our application in both the build and test jobs. This is not efficient because we are installing the same dependencies twice, which increases the execution time of our pipeline.

[...]

build:
  stage: build
  script:
    # We install the dependencies here
    - pip install -r requirements.txt

test:
  stage: test
  script:
    # We install the dependencies again here
    - pip install -r requirements.txt
    - [....]

We can further improve the pipeline by passing the libraries installed by the pip install -r requirements.txt command from the build job to the test job. This can be done by using the folder /usr/local/lib/python3.12/site-packages, which is the default location where Python libraries are installed in the container.

However, there is a problem, the /usr/local/lib/python3.12/site-packages folder contains system libraries that are not related to our application + the libraries installed by our application. We should not pass all the folder to avoid any conflict.

A simple solution that is specific to Python is to use a virtual environment to isolate the libraries of our application from the system libraries. We can create a virtual environment in the build job, install the dependencies in this virtual environment, and then pass the virtual environment to the test job.

Since Flake8 is not a dependency of our application, we will install it directly in the test job.

Also, when running flake8, we will exclude the virtual environment folder to avoid checking the code of the libraries installed in the virtual environment as our main focus is to check the coding style of our application only.

Here is how it is done:

cat <$HOME/todo/app/.gitlab-ci.yml && \
cd $HOME/todo/app && \
git add . && \
git commit -m "Pass libraries between jobs" && \
git push origin main
image: python:3.12

stages:
  - build
  - test
  - report

build:
  stage: build
  script:
    # Create a virtual environment
    -