Cloud Native CI/CD with GitLab

Choosing When to Download Artifacts in DAG Pipelines

Another feature of the needs keyword is the ability to choose when to download artifacts from previous jobs.

Let's say we have three jobs: job_1, job_2, and job_3 in a pipeline:

• job_2 and job_3 depend on job_1.
• The second job, job_2, needs the artifacts generated by job_1, but job_3 does not.

Here is the initial implementation as a starting point:

job_1:
  script:
    - echo "This is job 1"
    - echo "some data" > artifact1.txt
  artifacts:
    paths:
      - artifact1.txt

job_2:
  needs:
    - job_1
  script:
    - echo "This is job 2"
    - cat artifact1.txt

job_3:
  needs:
    - job_1
  script:
    - echo "This is job 3"

There is no problem with our pipeline, however, we are downloading the artifact1.txt file in the job_3 job, even though it is not needed. This can be time-consuming if the artifact is large. To avoid this, you can add artifacts: false to the job_3 job. This way, the artifact will not be downloaded even though it is generated by job_1.

Here is how you can achieve this:

job_3:
  needs: