GitLab CI/CD Pipeline Patterns#

GitLab CI/CD runs pipelines defined in a .gitlab-ci.yml file at the repository root. Every push, merge request, or tag triggers a pipeline consisting of stages that contain jobs. The pipeline configuration is version-controlled alongside your code, so the build process evolves with the application.

Basic .gitlab-ci.yml Structure#

A minimal pipeline defines stages and jobs. Stages run sequentially; jobs within the same stage run in parallel:

stages:
  - build
  - test
  - deploy

build-app:
  stage: build
  image: golang:1.22
  script:
    - go build -o myapp ./cmd/myapp
  artifacts:
    paths:
      - myapp
    expire_in: 1 hour

unit-tests:
  stage: test
  image: golang:1.22
  script:
    - go test ./... -v -coverprofile=coverage.out
  artifacts:
    reports:
      coverage_report:
        coverage_format: cobertura
        path: coverage.out

deploy-staging:
  stage: deploy
  image: bitnami/kubectl:latest
  script:
    - kubectl set image deployment/myapp myapp=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
  environment:
    name: staging
    url: https://staging.example.com
  rules:
    - if: $CI_COMMIT_BRANCH == "main"

Every job must have a stage and a script. The image field specifies the Docker image the job runs inside. If omitted, it falls back to the pipeline-level default image or the runner’s default.

Jobs in Detail#

Variables#

Define variables at the pipeline level, the job level, or in the GitLab UI under Settings > CI/CD > Variables:

variables:
  GOFLAGS: "-mod=vendor"
  APP_NAME: "myapp"

build-app:
  stage: build
  variables:
    CGO_ENABLED: "0"
  script:
    - go build -o $APP_NAME ./cmd/$APP_NAME

Pipeline-level variables apply to all jobs. Job-level variables override pipeline-level ones. Variables set in the GitLab UI are available to all pipelines and can be masked (hidden in logs) or protected (only available on protected branches).

Rules#

rules replaced the older only/except syntax. They control when a job runs:

deploy-production:
  stage: deploy
  script:
    - ./deploy.sh production
  rules:
    - if: $CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/
      when: manual
      allow_failure: false
    - if: $CI_COMMIT_BRANCH == "main"
      when: never

Rules are evaluated in order. The first match wins. when: manual creates a play button in the pipeline UI. when: never skips the job. when: on_success (the default) runs the job only if previous stages succeeded.

Artifacts#

Artifacts are files produced by a job and passed to subsequent stages or downloaded from the GitLab UI:

build-app:
  stage: build
  script:
    - make build
  artifacts:
    paths:
      - build/output/
    reports:
      junit: build/test-results/*.xml
    expire_in: 7 days
    when: always

The reports section has special integrations. junit reports appear in merge request widgets. coverage_report shows coverage diff. dotenv exports variables to downstream jobs. Setting when: always preserves artifacts even when the job fails, which is critical for test reports.

Caching#

Caching stores dependencies between pipeline runs. Unlike artifacts, caches are not guaranteed – they are best-effort and shared across pipelines:

build-app:
  stage: build
  image: golang:1.22
  cache:
    key:
      files:
        - go.sum
    paths:
      - .go-cache/
    policy: pull-push
  variables:
    GOPATH: $CI_PROJECT_DIR/.go-cache
  script:
    - go build -o myapp ./cmd/myapp

key.files generates the cache key from file content hashes. When go.sum changes, the cache is invalidated. policy: pull-push reads the cache at the start and writes it at the end. Use policy: pull on jobs that only consume the cache (like test jobs) to avoid redundant uploads.

For Node.js projects:

cache:
  key: $CI_COMMIT_REF_SLUG
  paths:
    - node_modules/

$CI_COMMIT_REF_SLUG gives each branch its own cache. This prevents dependency version collisions between branches.

DAG Pipelines#

Standard pipelines enforce strict stage ordering: all jobs in stage N must complete before stage N+1 starts. Directed Acyclic Graph (DAG) pipelines break this constraint using the needs keyword:

stages:
  - build
  - test
  - deploy

build-frontend:
  stage: build
  script:
    - npm run build
  artifacts:
    paths:
      - dist/

build-backend:
  stage: build
  script:
    - go build -o server ./cmd/server
  artifacts:
    paths:
      - server

test-frontend:
  stage: test
  needs: ["build-frontend"]
  script:
    - npm test

test-backend:
  stage: test
  needs: ["build-backend"]
  script:
    - go test ./...

deploy:
  stage: deploy
  needs: ["test-frontend", "test-backend"]
  script:
    - ./deploy.sh

test-frontend starts as soon as build-frontend finishes, without waiting for build-backend. This can significantly reduce total pipeline duration when independent workstreams have different execution times.

DAG jobs only receive artifacts from their needs dependencies, not from all previous stages. This is both a performance optimization (less artifact download) and a correctness feature (no accidental dependency on unrelated job output).

Includes and Extends for DRY Configuration#

includes#

Split large .gitlab-ci.yml files into reusable components:

include:
  - local: .gitlab/ci/build.yml
  - local: .gitlab/ci/test.yml
  - local: .gitlab/ci/deploy.yml
  - project: myorg/ci-templates
    ref: main
    file: /templates/docker-build.yml
  - remote: https://example.com/ci/security-scan.yml
  - template: Security/SAST.gitlab-ci.yml

local includes files from the same repo. project includes files from another GitLab project – this is how you build shared CI libraries across an organization. remote fetches from any URL. template uses GitLab’s built-in templates.

extends#

Inherit from hidden jobs (prefixed with .) to avoid repeating configuration:

.deploy-base:
  image: bitnami/kubectl:latest
  before_script:
    - kubectl config use-context $KUBE_CONTEXT
  retry:
    max: 2
    when: runner_system_failure

deploy-staging:
  extends: .deploy-base
  stage: deploy
  variables:
    KUBE_CONTEXT: staging
  script:
    - kubectl apply -f k8s/staging/
  environment:
    name: staging

deploy-production:
  extends: .deploy-base
  stage: deploy
  variables:
    KUBE_CONTEXT: production
  script:
    - kubectl apply -f k8s/production/
  environment:
    name: production
  rules:
    - if: $CI_COMMIT_TAG
      when: manual

Hidden jobs starting with . never run directly. They exist only to be extended. extends performs a deep merge – the child job inherits everything from the parent and can override specific fields.

Container Registry Integration#

GitLab includes a container registry per project. Every project can push and pull images without external registry configuration:

build-image:
  stage: build
  image: docker:24
  services:
    - docker:24-dind
  variables:
    DOCKER_TLS_CERTDIR: "/certs"
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA .
    - docker build -t $CI_REGISTRY_IMAGE:latest .
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
    - docker push $CI_REGISTRY_IMAGE:latest

The $CI_REGISTRY_* variables are automatically available. $CI_REGISTRY_IMAGE resolves to registry.gitlab.com/group/project. No manual configuration required.

For Kaniko builds (no Docker daemon needed):

build-image:
  stage: build
  image:
    name: gcr.io/kaniko-project/executor:v1.22.0-debug
    entrypoint: [""]
  script:
    - /kaniko/executor
      --context $CI_PROJECT_DIR
      --dockerfile $CI_PROJECT_DIR/Dockerfile
      --destination $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
      --destination $CI_REGISTRY_IMAGE:latest

Kaniko does not require privileged mode or Docker-in-Docker, making it safer for shared runners.

Environments and Review Apps#

Environments track where code is deployed. GitLab links deployments to merge requests and provides rollback buttons:

deploy-review:
  stage: deploy
  script:
    - helm upgrade --install review-$CI_COMMIT_REF_SLUG ./chart
      --set image.tag=$CI_COMMIT_SHA
      --set ingress.host=$CI_COMMIT_REF_SLUG.review.example.com
      --namespace review-$CI_COMMIT_REF_SLUG
      --create-namespace
  environment:
    name: review/$CI_COMMIT_REF_SLUG
    url: https://$CI_COMMIT_REF_SLUG.review.example.com
    on_stop: stop-review
    auto_stop_in: 1 week
  rules:
    - if: $CI_MERGE_REQUEST_IID

stop-review:
  stage: deploy
  script:
    - helm uninstall review-$CI_COMMIT_REF_SLUG -n review-$CI_COMMIT_REF_SLUG
    - kubectl delete namespace review-$CI_COMMIT_REF_SLUG
  environment:
    name: review/$CI_COMMIT_REF_SLUG
    action: stop
  rules:
    - if: $CI_MERGE_REQUEST_IID
      when: manual

Every merge request gets its own deployment accessible at a unique URL. The auto_stop_in setting automatically triggers the stop job after the specified duration, preventing resource accumulation from abandoned merge requests.

Auto DevOps#

Auto DevOps provides a complete CI/CD pipeline with zero configuration. Enable it in Settings > CI/CD > Auto DevOps. It automatically detects your language, builds a container image, runs tests, performs security scanning, and deploys to Kubernetes.

Auto DevOps works best as a starting point. For production workloads, extract the generated pipeline into a .gitlab-ci.yml and customize it. The templates are available at https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates.

Override specific Auto DevOps stages by defining jobs with the same name in your .gitlab-ci.yml. GitLab merges your configuration with the Auto DevOps template, and your definitions take precedence.

Common Mistakes#

  1. Using only/except instead of rules. The only/except syntax is legacy and cannot express complex conditions. rules is more powerful and explicit about job inclusion logic.
  2. Not setting expire_in on artifacts. Artifacts default to 30 days. Large artifacts from frequent pipelines fill storage quickly. Set explicit expiration on every artifact.
  3. Caching build output instead of dependencies. Caches are for dependency directories (node_modules, .go-cache). Build output should be artifacts. Caches are not guaranteed to exist; artifacts are.
  4. Running all test jobs sequentially when they have no dependencies. Use needs to create DAG pipelines. Independent test suites should run in parallel immediately after their build dependency completes.
  5. Not using include: project for shared configuration. Copy-pasting CI configuration across repositories guarantees drift. Centralize shared templates in a dedicated project and include them.