ArgoCD Notifications#

ArgoCD Notifications is a built-in component (since ArgoCD 2.5) that monitors applications and sends alerts when specific events occur – sync succeeded, sync failed, health degraded, new version deployed. Before notifications existed, teams polled the ArgoCD UI or built custom watchers. Notifications eliminates that.

Architecture#

ArgoCD Notifications runs as a controller alongside the ArgoCD application controller. It watches Application resources for state changes and matches them against triggers. When a trigger fires, it renders a template and sends it through a configured service (Slack, Teams, webhook, email, etc.).

Application state changes
    → Notifications controller detects change
    → Evaluates trigger conditions
    → Matches trigger → template
    → Renders template with application data
    → Sends via configured service

All configuration lives in two ConfigMaps in the ArgoCD namespace:

  • argocd-notifications-cm — Triggers, templates, and services
  • argocd-notifications-secret — Tokens and credentials for services

Configuring Slack#

Create a Slack App#

  1. Go to https://api.slack.com/apps and create a new app.
  2. Under OAuth & Permissions, add the chat:write scope.
  3. Install the app to your workspace.
  4. Copy the Bot User OAuth Token (xoxb-...).

Store the Token#

kubectl -n argocd patch secret argocd-notifications-secret --type merge -p \
  '{"stringData": {"slack-token": "xoxb-your-bot-token"}}'

Configure the Service#

apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-notifications-cm
  namespace: argocd
data:
  service.slack: |
    token: $slack-token

The $slack-token references the key in argocd-notifications-secret.

Configuring Microsoft Teams#

Teams uses incoming webhooks rather than bot tokens:

  1. In a Teams channel, add the Incoming Webhook connector.
  2. Copy the webhook URL.
kubectl -n argocd patch secret argocd-notifications-secret --type merge -p \
  '{"stringData": {"teams-webhook": "https://outlook.office.com/webhook/..."}}'
data:
  service.teams: |
    recipientUrls:
      deployments-channel: $teams-webhook

Configuring Webhooks#

For generic HTTP endpoints (PagerDuty, Opsgenie, custom services):

data:
  service.webhook.deployment-tracker: |
    url: https://deploy-tracker.example.com/api/events
    headers:
      - name: Authorization
        value: Bearer $webhook-token
      - name: Content-Type
        value: application/json

Triggers#

Triggers define when a notification fires. Each trigger has a name, a condition expression, and a list of templates to render when the condition is true.

Built-In Trigger Conditions#

ArgoCD provides several condition functions:

data:
  trigger.on-sync-succeeded: |
    - when: app.status.operationState.phase in ['Succeeded']
      send: [sync-succeeded]

  trigger.on-sync-failed: |
    - when: app.status.operationState.phase in ['Error', 'Failed']
      send: [sync-failed]

  trigger.on-health-degraded: |
    - when: app.status.health.status == 'Degraded'
      send: [health-degraded]

  trigger.on-deployed: |
    - when: app.status.operationState.phase in ['Succeeded'] and app.status.health.status == 'Healthy'
      send: [deployed]

Custom Trigger Conditions#

Trigger conditions are written in expr syntax with access to the full Application object:

data:
  trigger.on-prod-sync: |
    - when: app.status.operationState.phase in ['Succeeded'] and app.spec.destination.namespace == 'production'
      send: [prod-deployed]

  trigger.on-image-update: |
    - when: app.status.operationState.phase in ['Succeeded']
      oncePer: app.status.sync.revision
      send: [new-version]

The oncePer field prevents duplicate notifications. Without it, every reconciliation loop that matches the condition sends a notification.

Templates#

Templates define the notification content. They have access to the full Application object and support Go template syntax.

Slack Template#

data:
  template.sync-succeeded: |
    slack:
      attachments: |
        [{
          "color": "#18be52",
          "title": "{{.app.metadata.name}} synced successfully",
          "fields": [
            {"title": "Application", "value": "{{.app.metadata.name}}", "short": true},
            {"title": "Revision", "value": "{{.app.status.sync.revision | trunc 7}}", "short": true},
            {"title": "Namespace", "value": "{{.app.spec.destination.namespace}}", "short": true},
            {"title": "Cluster", "value": "{{.app.spec.destination.server}}", "short": true}
          ]
        }]

  template.sync-failed: |
    slack:
      attachments: |
        [{
          "color": "#e8272e",
          "title": ":x: {{.app.metadata.name}} sync failed",
          "text": "{{.app.status.operationState.message}}",
          "fields": [
            {"title": "Application", "value": "{{.app.metadata.name}}", "short": true},
            {"title": "Revision", "value": "{{.app.status.sync.revision | trunc 7}}", "short": true}
          ]
        }]

Webhook Template#

data:
  template.deployed: |
    webhook:
      deployment-tracker:
        method: POST
        body: |
          {
            "application": "{{.app.metadata.name}}",
            "revision": "{{.app.status.sync.revision}}",
            "namespace": "{{.app.spec.destination.namespace}}",
            "cluster": "{{.app.spec.destination.server}}",
            "status": "{{.app.status.health.status}}",
            "timestamp": "{{.app.status.operationState.finishedAt}}"
          }

Teams Template#

data:
  template.sync-succeeded: |
    teams:
      title: "{{.app.metadata.name}} deployed"
      text: "Application **{{.app.metadata.name}}** synced to revision `{{.app.status.sync.revision | trunc 7}}` in namespace `{{.app.spec.destination.namespace}}`."
      themeColor: "#18be52"

Subscribing Applications#

Applications opt into notifications using annotations. This is the link between triggers, templates, and where the message goes.

Per-Application Annotations#

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-app
  annotations:
    notifications.argoproj.io/subscribe.on-sync-succeeded.slack: deployments-channel
    notifications.argoproj.io/subscribe.on-sync-failed.slack: deployments-channel
    notifications.argoproj.io/subscribe.on-health-degraded.slack: alerts-channel

The format is notifications.argoproj.io/subscribe.<trigger>.<service>: <recipient>. The recipient is the Slack channel name (without #), the Teams recipient key, or the webhook service name.

Subscribe via ApplicationSet#

For fleet-wide notifications, add the annotations in the ApplicationSet template:

spec:
  template:
    metadata:
      name: '{{path.basename}}'
      annotations:
        notifications.argoproj.io/subscribe.on-sync-succeeded.slack: deployments-channel
        notifications.argoproj.io/subscribe.on-sync-failed.slack: alerts-channel

Every Application generated by the ApplicationSet inherits the notification subscriptions.

Default Triggers#

Set default triggers for all applications that subscribe to a service, without needing per-trigger annotations:

data:
  defaultTriggers: |
    - on-sync-succeeded
    - on-sync-failed
    - on-health-degraded

With default triggers configured, this simpler annotation is enough:

annotations:
  notifications.argoproj.io/subscribe.slack: deployments-channel

Full Working Configuration#

A complete argocd-notifications-cm for Slack:

apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-notifications-cm
  namespace: argocd
data:
  service.slack: |
    token: $slack-token

  trigger.on-sync-succeeded: |
    - when: app.status.operationState.phase in ['Succeeded']
      oncePer: app.status.sync.revision
      send: [sync-succeeded]

  trigger.on-sync-failed: |
    - when: app.status.operationState.phase in ['Error', 'Failed']
      oncePer: app.status.sync.revision
      send: [sync-failed]

  trigger.on-health-degraded: |
    - when: app.status.health.status == 'Degraded'
      oncePer: app.status.health.status
      send: [health-degraded]

  template.sync-succeeded: |
    slack:
      attachments: |
        [{
          "color": "#18be52",
          "title": "{{.app.metadata.name}} synced",
          "text": "Revision: {{.app.status.sync.revision | trunc 7}}\nNamespace: {{.app.spec.destination.namespace}}"
        }]

  template.sync-failed: |
    slack:
      attachments: |
        [{
          "color": "#e8272e",
          "title": "{{.app.metadata.name}} sync failed",
          "text": "{{.app.status.operationState.message}}"
        }]

  template.health-degraded: |
    slack:
      attachments: |
        [{
          "color": "#f4c030",
          "title": "{{.app.metadata.name}} health degraded",
          "text": "Application health status: {{.app.status.health.status}}"
        }]

Testing Notifications#

# List configured triggers
kubectl get cm argocd-notifications-cm -n argocd -o yaml | grep "trigger\."

# Manually trigger a notification for testing
argocd admin notifications trigger run on-sync-succeeded my-app --context /

# Check notification controller logs for errors
kubectl logs -n argocd -l app.kubernetes.io/name=argocd-notifications-controller --tail=100

If notifications are not firing, the most common causes are:

  1. Missing or incorrect annotation on the Application.
  2. The Slack token does not have chat:write permission for the target channel.
  3. The oncePer field prevents re-sending because the value has not changed.
  4. The trigger condition does not match the actual Application state field paths.

Common Mistakes#

  1. Not using oncePer on triggers. Without it, ArgoCD sends a notification on every reconciliation cycle (default 3 minutes) while the condition is true. For on-sync-succeeded, use oncePer: app.status.sync.revision so it fires once per new revision.
  2. Putting tokens directly in the ConfigMap instead of the Secret. argocd-notifications-cm is not encrypted. Always store credentials in argocd-notifications-secret and reference them with $variable-name syntax.
  3. Subscribing to too many triggers on noisy channels. Start with sync-failed and health-degraded on an alerts channel. Add sync-succeeded on a separate channel for audit purposes.
  4. Forgetting to invite the Slack bot to the channel. The bot app must be a member of every channel it posts to. Add it with /invite @your-bot-name in the channel.
  5. Using default triggers without understanding them. Default triggers apply to every subscribed application. If you add on-sync-succeeded as a default trigger and have 200 applications, you get 200 messages per deployment wave.