ArgoCD Secrets Management#

GitOps says everything should be in Git. Kubernetes Secrets are base64-encoded, not encrypted. Committing base64 secrets to Git is equivalent to committing plaintext – anyone with repo access can decode them. This is the fundamental tension of GitOps secrets management.

Three approaches solve this, each with different tradeoffs.

Approach 1: Sealed Secrets#

Sealed Secrets encrypts secrets client-side so the encrypted form can be safely committed to Git. Only the Sealed Secrets controller running in-cluster can decrypt them.

How It Works#

Developer encrypts Secret with kubeseal (uses controller's public key)
    → SealedSecret resource committed to Git
    → ArgoCD syncs SealedSecret to cluster
    → Sealed Secrets controller decrypts it
    → Regular Kubernetes Secret is created in the namespace
    → Pods consume the Secret normally

Installation#

helm repo add sealed-secrets https://bitnami-labs.github.io/sealed-secrets
helm install sealed-secrets sealed-secrets/sealed-secrets \
  --namespace kube-system \
  --set-string fullnameOverride=sealed-secrets-controller

Install the CLI:

# macOS
brew install kubeseal

# Linux
KUBESEAL_VERSION=$(curl -s https://api.github.com/repos/bitnami-labs/sealed-secrets/releases/latest | grep tag_name | cut -d '"' -f4 | cut -c2-)
curl -OL "https://github.com/bitnami-labs/sealed-secrets/releases/download/v${KUBESEAL_VERSION}/kubeseal-${KUBESEAL_VERSION}-linux-amd64.tar.gz"
tar -xvzf kubeseal-*.tar.gz kubeseal
sudo install -m 755 kubeseal /usr/local/bin/kubeseal

Encrypting a Secret#

Start with a regular Secret manifest:

# my-secret.yaml (DO NOT commit this file)
apiVersion: v1
kind: Secret
metadata:
  name: db-credentials
  namespace: my-app
stringData:
  username: admin
  password: s3cret-p4ssword
  connection-string: "postgresql://admin:s3cret-p4ssword@db:5432/mydb"

Encrypt it:

kubeseal --format yaml < my-secret.yaml > sealed-secret.yaml

The output is a SealedSecret resource:

apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  name: db-credentials
  namespace: my-app
spec:
  encryptedData:
    username: AgBy3i4OJSWK+PiTySYZZA9rO...
    password: AgCtr8OJSWK+PiReSYZZA9rO...
    connection-string: AgDf5OJSWK+PiTySYZZBt7q...
  template:
    metadata:
      name: db-credentials
      namespace: my-app

Commit sealed-secret.yaml to Git. Delete the plaintext my-secret.yaml.

Scoping#

Sealed Secrets supports three scopes that control where the decrypted Secret can be created:

• strict (default): Sealed to a specific name and namespace. Cannot be moved or renamed.
• namespace-wide: Can be used with any name within the sealed namespace.
• cluster-wide: Can be decrypted in any namespace with any name. Use with caution.

# Namespace-wide scope
kubeseal --format yaml --scope namespace-wide < my-secret.yaml > sealed-secret.yaml

# Cluster-wide scope
kubeseal --format yaml --scope cluster-wide < my-secret.yaml > sealed-secret.yaml

ArgoCD Integration#

No special ArgoCD configuration is needed. ArgoCD applies the SealedSecret resource like any other manifest. The Sealed Secrets controller handles decryption asynchronously.

Add a custom health check so ArgoCD can track SealedSecret status:

# In argocd-cm ConfigMap
data:
  resource.customizations.health.bitnami.com_SealedSecret: |
    hs = {}
    if obj.status ~= nil then
      if obj.status.conditions ~= nil then
        for i, condition in ipairs(obj.status.conditions) do
          if condition.type == "Synced" and condition.status == "True" then
            hs.status = "Healthy"
            hs.message = "Secret has been unsealed"
            return hs
          end
          if condition.type == "Synced" and condition.status == "False" then
            hs.status = "Degraded"
            hs.message = condition.message
            return hs
          end
        end
      end
    end
    hs.status = "Progressing"
    hs.message = "Waiting for secret to be unsealed"
    return hs

Key Rotation and Backup#

The Sealed Secrets controller generates a sealing key pair on startup. If you lose this key, you cannot decrypt any existing SealedSecrets. Back it up:

kubectl get secret -n kube-system -l sealedsecrets.bitnami.com/sealed-secrets-key -o yaml > sealed-secrets-key-backup.yaml

Store this backup outside the cluster in a secure location. This is the one secret you cannot seal.

The controller generates new keys every 30 days by default but keeps old keys for decryption. To re-encrypt all SealedSecrets with the latest key:

kubeseal --re-encrypt < sealed-secret.yaml > sealed-secret-new.yaml

Approach 2: External Secrets Operator#

External Secrets Operator (ESO) pulls secrets from an external secret store (AWS Secrets Manager, HashiCorp Vault, Azure Key Vault, GCP Secret Manager) and creates Kubernetes Secrets automatically. The actual secret values never appear in Git at all.

How It Works#

Secret stored in AWS Secrets Manager / Vault / etc.
    → ExternalSecret resource in Git (references the external secret by name)
    → ArgoCD syncs ExternalSecret to cluster
    → ESO reads the external store and creates a Kubernetes Secret
    → Pods consume the Secret normally

Installation#

helm repo add external-secrets https://charts.external-secrets.io
helm install external-secrets external-secrets/external-secrets \
  --namespace external-secrets \
  --create-namespace

Configuring a SecretStore#

A SecretStore tells ESO how to connect to the external provider. A ClusterSecretStore works across all namespaces; a SecretStore is namespace-scoped.

AWS Secrets Manager example:

apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
metadata:
  name: aws-secrets-manager
spec:
  provider:
    aws:
      service: SecretsManager
      region: us-east-1
      auth:
        jwt:
          serviceAccountRef:
            name: external-secrets-sa
            namespace: external-secrets

HashiCorp Vault example:

apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
metadata:
  name: vault
spec:
  provider:
    vault:
      server: "https://vault.example.com"
      path: "secret"
      version: "v2"
      auth:
        kubernetes:
          mountPath: "kubernetes"
          role: "external-secrets"
          serviceAccountRef:
            name: external-secrets-sa
            namespace: external-secrets

Creating an ExternalSecret#

This is what goes in Git – a reference to the external secret, not the secret value:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: db-credentials
  namespace: my-app
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: aws-secrets-manager
    kind: ClusterSecretStore
  target:
    name: db-credentials
    creationPolicy: Owner
  data:
    - secretKey: username
      remoteRef:
        key: production/my-app/db
        property: username
    - secretKey: password
      remoteRef:
        key: production/my-app/db
        property: password

ESO creates a regular Kubernetes Secret named db-credentials with username and password keys, pulling the values from production/my-app/db in AWS Secrets Manager.

Templating#

ESO can transform secret data before creating the Kubernetes Secret:

spec:
  target:
    name: db-connection
    template:
      type: Opaque
      data:
        connection-string: "postgresql://{{ .username }}:{{ .password }}@db.example.com:5432/mydb"
  data:
    - secretKey: username
      remoteRef:
        key: production/my-app/db
        property: username
    - secretKey: password
      remoteRef:
        key: production/my-app/db
        property: password

This constructs a connection string from individual secret fields without storing the assembled string in the external provider.

ArgoCD Integration#

Like Sealed Secrets, no special ArgoCD configuration is required. ArgoCD syncs the ExternalSecret CRD, and ESO handles the rest.

ESO’s ExternalSecret resources include status conditions that ArgoCD can read out of the box. The application shows Healthy when the external secret is successfully synced.

Approach 3: SOPS (Secrets OPerationS)#

SOPS encrypts specific values within YAML or JSON files in-place, leaving keys and structure visible. It supports multiple encryption backends: age, PGP, AWS KMS, GCP KMS, Azure Key Vault.

How It Works#

Developer encrypts values in Secret manifest using SOPS
    → Encrypted YAML committed to Git (keys visible, values encrypted)
    → ArgoCD uses KSOPS or helm-secrets plugin to decrypt during sync
    → Decrypted Secret applied to cluster

Encrypting with SOPS and age#

age is the simplest key management option for SOPS:

# Install
brew install sops age

# Generate a key pair
age-keygen -o key.txt
# Public key: age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p

# Create a .sops.yaml in the repo root to configure encryption rules
cat > .sops.yaml << 'EOF'
creation_rules:
  - path_regex: .*secrets.*\.yaml$
    age: age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p
EOF

Encrypt a Secret manifest:

sops --encrypt --in-place secrets/db-credentials.yaml

The result keeps YAML structure intact but encrypts the values:

apiVersion: v1
kind: Secret
metadata:
  name: db-credentials
  namespace: my-app
stringData:
  username: ENC[AES256_GCM,data:dGVzdA==,iv:abc...,tag:def...,type:str]
  password: ENC[AES256_GCM,data:c2VjcmV0,iv:ghi...,tag:jkl...,type:str]
sops:
  age:
    - recipient: age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p
      enc: |
        -----BEGIN AGE ENCRYPTED FILE-----
        ...

ArgoCD Integration with KSOPS#

KSOPS is a Kustomize plugin that decrypts SOPS files during kustomize build. ArgoCD’s repo server needs the SOPS binary and the decryption key.

Patch the ArgoCD repo server to include SOPS:

# In argocd values.yaml
repoServer:
  env:
    - name: SOPS_AGE_KEY_FILE
      value: /sops/age/keys.txt
  volumes:
    - name: sops-age
      secret:
        secretName: sops-age-key
  volumeMounts:
    - name: sops-age
      mountPath: /sops/age

Create the age key as a Kubernetes Secret in the ArgoCD namespace:

kubectl create secret generic sops-age-key \
  --from-file=keys.txt=key.txt \
  --namespace argocd

In your application manifests, use a Kustomize generator with KSOPS:

# kustomization.yaml
generators:
  - secret-generator.yaml

# secret-generator.yaml
apiVersion: viaduct.ai/v1
kind: ksops
metadata:
  name: db-credentials
files:
  - secrets/db-credentials.yaml

ArgoCD Integration with helm-secrets#

For Helm-based applications, the helm-secrets plugin decrypts SOPS-encrypted values files:

spec:
  source:
    path: charts/my-app
    helm:
      valueFiles:
        - values.yaml
        - secrets+age-import:///sops/age/keys.txt?secrets.yaml

Choosing an Approach#

Sealed Secrets is the simplest starting point. Everything is self-contained in the cluster. Good for small teams and single-cluster setups.

External Secrets Operator is the production choice for teams already using a secret store. Secrets are centrally managed, automatically rotated, and never touch Git even in encrypted form.

SOPS fits teams that want encrypted secrets in Git with fine-grained diff visibility (you can see which keys changed, just not the values). Works well with PR review workflows.

Common Mistakes#

1. Committing plaintext Secrets “just temporarily.” Git history is permanent. Even after removing the file, the secret is in the commit history. If this happens, rotate the secret immediately.
2. Not backing up Sealed Secrets controller keys. Lose the key, lose all your secrets. Back up the sealing key and store it outside the cluster.
3. Setting ExternalSecret refreshInterval too low. Every refresh calls the external provider API. At 10s with 100 ExternalSecrets, that is 600 API calls per minute. Start at 1h and lower only where needed.
4. Forgetting the decryption key when migrating clusters. SOPS and Sealed Secrets both need their keys present in the new cluster before ArgoCD can sync encrypted resources.
5. Using the same secret values across all environments. Each environment should have its own secrets, even for non-sensitive config. This prevents accidental cross-environment connections.