The Problem with Environment Variables#

Environment variables are the most common way to pass secrets to applications. Every framework supports them and they require zero dependencies. They are also the least secure option. Any process running as the same user can read them via /proc/<pid>/environ on Linux. Crash dumps include the full environment. Child processes inherit all variables by default.

# Anyone with host access can read another process's environment
cat /proc/$(pgrep myapp)/environ | tr '\0' '\n' | grep DB_PASSWORD

Environment variables are acceptable for local development. For production secrets, use one of the patterns below.

Mounted Files#

Mount the secret as a file instead. This is how Kubernetes Secrets work when used as volumes:

volumeMounts:
  - name: db-creds
    mountPath: /etc/secrets/db
    readOnly: true
volumes:
  - name: db-creds
    secret:
      secretName: myapp-db-credentials

The application reads /etc/secrets/db/password instead of checking $DB_PASSWORD. The secret does not appear in process listings, crash dumps, or child process environments.

Kubernetes Secrets Are Not Encrypted#

A Kubernetes Secret stores data as base64-encoded text. Base64 is encoding, not encryption. Anyone with get secrets RBAC reads them in plaintext. To add real protection, enable encryption at rest:

apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
  - resources:
      - secrets
    providers:
      - aescbc:
          keys:
            - name: key1
              secret: <base64-encoded-32-byte-key>
      - identity: {}

Pass it to the API server with --encryption-provider-config. This encrypts secrets in etcd but does not protect them at the API layer. RBAC remains your primary access control.

HashiCorp Vault#

Vault is the standard for centralized secret management. Two features matter most: KV v2 (versioned key-value store) and dynamic database credentials.

KV v2 and Dynamic Database Credentials#

vault kv put secret/myapp/db username=appuser password=s3cret-value
vault kv get secret/myapp/db

KV v2 keeps version history automatically. But the real power is dynamic credentials – Vault generates short-lived database users on demand and revokes them at expiry:

vault write database/config/mydb \
  plugin_name=postgresql-database-plugin \
  connection_url="postgresql://{{username}}:{{password}}@db.internal:5432/myapp" \
  allowed_roles="myapp-role" \
  username="vault_admin" \
  password="admin_password"

vault write database/roles/myapp-role \
  db_name=mydb \
  creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}'; GRANT SELECT, INSERT, UPDATE ON ALL TABLES IN SCHEMA public TO \"{{name}}\";" \
  default_ttl="1h" \
  max_ttl="24h"

Each application instance gets unique credentials. If one is compromised, revoke it without affecting others.

AppRole Authentication#

AppRole is the standard machine-to-machine auth method. The application presents a role ID (not secret) and a single-use secret ID:

vault auth enable approle
vault write auth/approle/role/myapp \
  token_ttl=1h \
  token_max_ttl=4h \
  secret_id_num_uses=1 \
  policies="myapp-policy"

vault read auth/approle/role/myapp/role-id
vault write -f auth/approle/role/myapp/secret-id

The application exchanges the secret ID for a Vault token at deploy time, then uses that token to read secrets.

External Secrets Operator#

The External Secrets Operator syncs secrets from Vault, AWS Secrets Manager, GCP Secret Manager, or Azure Key Vault into native Kubernetes Secrets.

apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: vault-backend
spec:
  provider:
    vault:
      server: "https://vault.internal:8200"
      path: "secret"
      auth:
        appRole:
          path: "approle"
          roleId: "db02de05-c0de-4311-a19f-your-role-id"
          secretRef:
            name: vault-approle-secret
            key: secret-id
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: myapp-db-credentials
spec:
  refreshInterval: 5m
  secretStoreRef:
    name: vault-backend
  target:
    name: myapp-db-credentials
  data:
    - secretKey: password
      remoteRef:
        key: secret/data/myapp/db
        property: password

The operator polls Vault every 5 minutes and updates the Kubernetes Secret. No Vault SDK or sidecar needed in the application.

SOPS for Encrypted Files in Git#

Mozilla SOPS encrypts secret values inside YAML, JSON, or ENV files while leaving keys and structure visible. You can commit encrypted files to git and decrypt them in CI.

sops --encrypt --age age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p \
  secrets.yaml > secrets.enc.yaml

SOPS encrypts only the values, not the keys. The file remains diffable and reviewable. Decrypt during deployment:

sops --decrypt secrets.enc.yaml | kubectl apply -f -

Sealed Secrets for Kubernetes#

Bitnami’s Sealed Secrets uses asymmetric encryption. You encrypt on your workstation with the cluster’s public key. Only the controller running inside the cluster can decrypt:

kubeseal --format yaml < secret.yaml > sealed-secret.yaml

The sealed secret is safe to commit to git. The controller watches for SealedSecret resources and creates the corresponding Kubernetes Secret.

Decision Framework#

Use environment variables only for non-sensitive configuration. Use mounted files (native k8s Secrets) when you have few secrets and a small team with strong RBAC. Use Sealed Secrets or SOPS when you need secrets in git for GitOps workflows. Use External Secrets Operator when secrets live in an external store and you want Kubernetes-native consumption. Use Vault with dynamic credentials when you need short-lived, per-instance credentials and centralized audit logging. The approaches compose: Vault as the source of truth, External Secrets Operator to sync into Kubernetes, mounted files for application consumption.