Azure DevOps Pipelines#

Azure DevOps Pipelines uses YAML files stored in your repository to define build and deployment workflows. The pipeline model has three levels: stages contain jobs, jobs contain steps. This hierarchy maps directly to how you think about CI/CD – build stage, test stage, deploy-to-staging stage, deploy-to-production stage – with each stage containing one or more parallel jobs.

Pipeline Structure#

A complete pipeline in azure-pipelines.yml:

trigger:
  branches:
    include:
      - main
      - release/*
  paths:
    exclude:
      - docs/**
      - README.md

pool:
  vmImage: 'ubuntu-latest'

variables:
  - group: common-vars
  - name: buildConfiguration
    value: 'Release'

stages:
  - stage: Build
    jobs:
      - job: BuildApp
        steps:
          - task: GoTool@0
            inputs:
              version: '1.22'
          - script: |
              go build -o $(Build.ArtifactStagingDirectory)/myapp ./cmd/myapp
            displayName: 'Build binary'
          - publish: $(Build.ArtifactStagingDirectory)
            artifact: drop

  - stage: Test
    dependsOn: Build
    jobs:
      - job: UnitTests
        steps:
          - task: GoTool@0
            inputs:
              version: '1.22'
          - script: go test ./... -v -coverprofile=coverage.out
            displayName: 'Run tests'
          - task: PublishCodeCoverageResults@2
            inputs:
              summaryFileLocation: coverage.out
              codecoverageTool: 'Cobertura'

  - stage: DeployStaging
    dependsOn: Test
    condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
    jobs:
      - deployment: DeployToStaging
        environment: staging
        strategy:
          runOnce:
            deploy:
              steps:
                - download: current
                  artifact: drop
                - script: echo "Deploying to staging"

trigger controls which branches and paths trigger the pipeline. dependsOn creates stage ordering. condition adds logic – succeeded() checks the previous stage passed, and you can combine it with variable checks to restrict certain stages to specific branches.

Stages, Jobs, and Steps#

Stages are the top-level grouping. Each stage can target a different environment, have its own approval gates, and run on different agent pools. Stages run sequentially by default; use dependsOn: [] to run a stage with no dependencies (parallel to others).

Jobs run on an agent. A job is a generic execution unit. A deployment job is specialized for deploying to an environment – it tracks deployment history and supports strategies like runOnce, rolling, and canary.

Steps are the atomic units: script (inline shell), task (built-in or marketplace tasks), checkout (source code), download (pipeline artifacts), and publish (upload artifacts).

stages:
  - stage: Build
    jobs:
      - job: BuildLinux
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - script: make build-linux
      - job: BuildWindows
        pool:
          vmImage: 'windows-latest'
        steps:
          - script: make build-windows

Jobs within a stage run in parallel by default. Use dependsOn between jobs for sequential ordering within a stage.

Service Connections#

Service connections authenticate pipelines to external services – Azure subscriptions, Docker registries, Kubernetes clusters, AWS accounts. They are created in Project Settings and referenced by name in YAML:

steps:
  - task: Docker@2
    inputs:
      containerRegistry: 'acr-connection'
      repository: 'myapp'
      command: 'buildAndPush'
      Dockerfile: '**/Dockerfile'
      tags: |
        $(Build.BuildId)
        latest

  - task: KubernetesManifest@1
    inputs:
      action: 'deploy'
      connectionType: 'kubernetesServiceConnection'
      kubernetesServiceConnection: 'aks-staging'
      namespace: 'myapp'
      manifests: 'k8s/*.yaml'
      containers: 'myacr.azurecr.io/myapp:$(Build.BuildId)'

Service connections use Azure AD service principals or managed identities. The credentials are managed centrally and never appear in pipeline YAML. You can restrict which pipelines access a connection through pipeline permissions on the service connection settings.

Variable Groups and Secrets#

Variable groups bundle related variables and are managed in Library. Link them to pipelines with the group keyword:

variables:
  - group: staging-config
  - group: common-secrets
  - name: localVar
    value: 'inline-value'

steps:
  - script: |
      echo "Deploying to $(environment-url)"
      echo "Using registry $(acr-name)"
    env:
      DB_PASSWORD: $(db-password)

Variables marked as secret in the group are masked in logs. Reference them with $(variable-name) in YAML or map them to environment variables with env. Secret variables cannot be used in conditions or printed directly – they are only available as environment variables within steps.

For Azure Key Vault integration, link a variable group to a Key Vault:

variables:
  - group: keyvault-secrets  # Linked to Azure Key Vault

steps:
  - script: ./deploy.sh
    env:
      API_KEY: $(api-key)        # Pulled from Key Vault at runtime
      DB_CONN: $(db-connection)  # Pulled from Key Vault at runtime

This keeps secrets in Key Vault with full audit logging and rotation capabilities, while making them available as pipeline variables without duplication.

Environments and Approvals#

Environments represent deployment targets. They provide approval gates, deployment history, and Kubernetes resource tracking:

stages:
  - stage: DeployProduction
    jobs:
      - deployment: ProductionDeploy
        environment: production.myapp-namespace
        strategy:
          runOnce:
            deploy:
              steps:
                - task: KubernetesManifest@1
                  inputs:
                    action: deploy
                    kubernetesServiceConnection: 'aks-production'
                    namespace: 'myapp'
                    manifests: 'k8s/production/*.yaml'

Configure approvals in the Azure DevOps UI under Environments > production > Approvals and checks. Options include:

Approvals: Require one or more specific users or groups to approve before deployment proceeds.
Branch control: Restrict which branches can deploy to the environment.
Business hours: Only allow deployments during specified hours.
Exclusive lock: Prevent parallel deployments to the same environment.

The production.myapp-namespace syntax targets a specific Kubernetes namespace within the environment. Azure DevOps tracks which resources are deployed there and shows deployment history per namespace.

Template References#

Templates enable reusable pipeline components. They can define steps, jobs, stages, or variables:

# templates/build-go.yml
parameters:
  - name: goVersion
    type: string
    default: '1.22'
  - name: outputBinary
    type: string

steps:
  - task: GoTool@0
    inputs:
      version: ${{ parameters.goVersion }}
  - script: |
      CGO_ENABLED=0 go build -o $(Build.ArtifactStagingDirectory)/${{ parameters.outputBinary }} ./cmd/${{ parameters.outputBinary }}
    displayName: 'Build ${{ parameters.outputBinary }}'
  - publish: $(Build.ArtifactStagingDirectory)
    artifact: ${{ parameters.outputBinary }}

Reference templates in your main pipeline:

stages:
  - stage: Build
    jobs:
      - job: BuildApp
        steps:
          - template: templates/build-go.yml
            parameters:
              goVersion: '1.22'
              outputBinary: 'myapp'

Stage-level templates enable full pipeline composition:

# templates/deploy-stage.yml
parameters:
  - name: environment
    type: string
  - name: serviceConnection
    type: string

stages:
  - stage: Deploy_${{ parameters.environment }}
    jobs:
      - deployment: Deploy
        environment: ${{ parameters.environment }}
        strategy:
          runOnce:
            deploy:
              steps:
                - task: AzureCLI@2
                  inputs:
                    azureSubscription: ${{ parameters.serviceConnection }}
                    scriptType: 'bash'
                    scriptLocation: 'inlineScript'
                    inlineScript: |
                      az aks get-credentials --resource-group myapp-rg --name myapp-aks
                      helm upgrade --install myapp ./chart --set image.tag=$(Build.BuildId)

# azure-pipelines.yml
stages:
  - stage: Build
    jobs:
      - job: Build
        steps:
          - template: templates/build-go.yml
            parameters:
              outputBinary: myapp

  - template: templates/deploy-stage.yml
    parameters:
      environment: staging
      serviceConnection: azure-staging

  - template: templates/deploy-stage.yml
    parameters:
      environment: production
      serviceConnection: azure-production

Templates can live in the same repository or in a separate template repository referenced with resources.repositories:

resources:
  repositories:
    - repository: templates
      type: git
      name: MyOrg/pipeline-templates
      ref: refs/tags/v2.0

stages:
  - template: stages/deploy.yml@templates
    parameters:
      environment: production

Pin template references to tags or specific commits. Using ref: refs/heads/main means template changes immediately affect all consuming pipelines.

AKS and ACR Integration#

The typical Azure pipeline builds a container, pushes to ACR, and deploys to AKS:

variables:
  acrName: 'myacr'
  acrLoginServer: 'myacr.azurecr.io'
  imageRepository: 'myapp'

stages:
  - stage: Build
    jobs:
      - job: BuildAndPush
        steps:
          - task: Docker@2
            displayName: 'Build and push to ACR'
            inputs:
              containerRegistry: 'acr-service-connection'
              repository: $(imageRepository)
              command: 'buildAndPush'
              Dockerfile: 'Dockerfile'
              tags: |
                $(Build.BuildId)
                $(Build.SourceBranchName)-latest

  - stage: Deploy
    dependsOn: Build
    jobs:
      - deployment: DeployToAKS
        environment: staging.myapp
        strategy:
          runOnce:
            deploy:
              steps:
                - task: KubernetesManifest@1
                  displayName: 'Deploy to AKS'
                  inputs:
                    action: 'deploy'
                    connectionType: 'kubernetesServiceConnection'
                    kubernetesServiceConnection: 'aks-staging-connection'
                    namespace: 'myapp'
                    manifests: 'k8s/deployment.yaml'
                    containers: '$(acrLoginServer)/$(imageRepository):$(Build.BuildId)'
                    imagePullSecrets: 'acr-secret'

The containers field in KubernetesManifest performs image substitution – it replaces image references in your manifests with the fully-qualified tag. The imagePullSecrets field injects the pull secret into deployed pods.

For Helm deployments, use HelmDeploy@0:

- task: HelmDeploy@0
  inputs:
    connectionType: 'Kubernetes Service Connection'
    kubernetesServiceConnection: 'aks-production'
    namespace: 'myapp'
    command: 'upgrade'
    chartType: 'FilePath'
    chartPath: 'charts/myapp'
    releaseName: 'myapp'
    overrideValues: 'image.tag=$(Build.BuildId),image.repository=$(acrLoginServer)/$(imageRepository)'

Common Mistakes#

Using classic (UI-based) pipelines instead of YAML. Classic pipelines are not version-controlled. YAML pipelines live in the repo and follow the same review process as code.
Not using template references for repeated patterns. Copy-pasting deploy stages across pipelines guarantees drift. Extract templates and share them via a central repository.
Putting secrets in variable definitions instead of variable groups. Inline variables in YAML are visible to anyone with repo access. Use variable groups with secret masking or Key Vault-linked groups.
Ignoring environment checks. Without approvals on production environments, any push to main deploys directly to production. Always configure at least one approval gate on production environments.
Not pinning template repository references. Using ref: refs/heads/main for shared templates means any template change immediately affects all pipelines. Pin to tags and upgrade deliberately.