Cloud Native CI/CD with GitLab

DAG Pipelines and Artifacts

In GitLab CI, needs is both an execution-order tool and an artifact-wiring tool. The important gotcha is that once you use needs, the default artifact behavior changes.

By default (no needs, no dependencies), a job in a later stage downloads artifacts from all jobs in earlier stages.

Once a job uses needs, it no longer downloads "all previous artifacts". It can only download artifacts from the jobs listed in needs, and you control that with artifacts: true|false.

Let's take an example, we have this pipeline as a starting point:

job_1:
  script:
    - echo "This is job 1" > file.txt

job_2:
  needs:
    - job_1
  script:
    - cat file.txt | grep "This is job 1" || echo "file.txt not found"

We want to pass the file.txt artifact generated by job_1 to job_2. We do so by adding artifacts: paths to job_1 and artifacts: true to the needs declaration in job_2:

job_1:
  script:
    - echo "This is job 1" > file.txt
  artifacts:
    paths:
      - file.txt