Admission Controllers and Webhooks#

Every request to the Kubernetes API server passes through a chain: authentication, authorization, and then admission control. Admission controllers are plugins that intercept requests after a user is authenticated and authorized but before the object is persisted to etcd. They can validate requests, reject them, or mutate objects on the fly. This is where you enforce organizational policy, inject sidecar containers, set defaults, and block dangerous configurations.

The Admission Chain#

The API server processes admission in two phases, always in the same order:

1. Mutating admission runs first. Mutating webhooks can modify the incoming object – inject containers, add labels, set default resource requests. Multiple mutating webhooks run sequentially, and each sees the modifications from the previous one.

2. Validating admission runs second. Validating webhooks receive the final object (after all mutations) and can accept or reject it. They cannot modify the object. Multiple validating webhooks run independently.

This ordering matters. If a mutating webhook adds a sidecar container, a validating webhook that checks resource limits will see that sidecar and can reject the pod if the sidecar is missing limits.

Built-in Admission Controllers#

Kubernetes ships with several built-in admission controllers that are enabled by default. These are not webhooks – they are compiled into the API server binary:

• NamespaceLifecycle – prevents operations in namespaces that are being deleted
• LimitRanger – applies default resource requests/limits from LimitRange objects
• ServiceAccount – auto-mounts service account tokens into pods
• DefaultStorageClass – assigns the default StorageClass to PVCs that do not specify one
• ResourceQuota – enforces resource quotas per namespace
• PodSecurity – enforces Pod Security Standards (replaced PodSecurityPolicy)
• MutatingAdmissionWebhook – calls external mutating webhooks
• ValidatingAdmissionWebhook – calls external validating webhooks

The last two are what enable custom webhooks. They are meta-controllers that dispatch to your webhook servers.

ValidatingWebhookConfiguration#

To register a validating webhook, create a ValidatingWebhookConfiguration resource:

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  name: require-resource-limits
webhooks:
- name: resource-limits.example.com
  admissionReviewVersions: ["v1"]
  sideEffects: None
  timeoutSeconds: 5
  failurePolicy: Fail
  matchPolicy: Equivalent
  namespaceSelector:
    matchExpressions:
    - key: kubernetes.io/metadata.name
      operator: NotIn
      values: ["kube-system", "kube-public", "cert-manager"]
  rules:
  - apiGroups: ["apps"]
    apiVersions: ["v1"]
    operations: ["CREATE", "UPDATE"]
    resources: ["deployments"]
    scope: Namespaced
  clientConfig:
    service:
      name: webhook-server
      namespace: webhook-system
      path: /validate-resource-limits
      port: 443
    caBundle: <base64-encoded-CA-cert>

Key fields:

• rules – which resources and operations to intercept. Be specific. Intercepting everything generates unnecessary load. The example above only catches Deployment CREATE and UPDATE operations.
• clientConfig – where to send the request. Either a service reference (for in-cluster webhooks) or an external URL.
• failurePolicy – Fail rejects the request if the webhook is unreachable. Ignore allows the request through. This is a critical decision. Fail is safer but can lock your cluster if the webhook goes down. Ignore is more available but means policy is not enforced during outages.
• namespaceSelector – which namespaces the webhook applies to. Always exclude kube-system. If your webhook intercepts pod creation and matches its own namespace, you create a deadlock: the webhook pod cannot start because the webhook is not running to approve it.
• sideEffects – set to None for webhooks that have no side effects, or NoneOnDryRun if your webhook does something on real calls but not on dry-run.
• timeoutSeconds – how long the API server waits for a response. Default is 10 seconds. Set this lower (3-5 seconds) to keep API responsiveness high.

MutatingWebhookConfiguration#

The structure is identical to ValidatingWebhookConfiguration, but mutating webhooks can modify the object by returning a JSON Patch in the response. Common mutation patterns:

• Sidecar injection – Istio’s sidecar injector is a mutating webhook that adds the envoy-proxy container to every pod in labeled namespaces.
• Default labels/annotations – automatically add team ownership labels, cost-center annotations, or environment tags.
• Resource defaults – set resource requests on containers that do not specify them.
• Environment variable injection – add standard env vars (cluster name, region, environment) to all containers.

apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
  name: inject-defaults
webhooks:
- name: defaults.example.com
  admissionReviewVersions: ["v1"]
  sideEffects: None
  reinvocationPolicy: IfNeeded
  failurePolicy: Ignore
  rules:
  - apiGroups: [""]
    apiVersions: ["v1"]
    operations: ["CREATE"]
    resources: ["pods"]
  clientConfig:
    service:
      name: defaults-webhook
      namespace: webhook-system
      path: /mutate-pod-defaults
      port: 443
    caBundle: <base64-encoded-CA-cert>

Note reinvocationPolicy: IfNeeded – if another mutating webhook modifies the object after yours ran, the API server will re-invoke your webhook so it can see the final state.

Building a Webhook Server#

A webhook server is an HTTPS server that receives AdmissionReview objects and returns AdmissionResponse. Here is the core flow in Go:

func handleValidate(w http.ResponseWriter, r *http.Request) {
    var admissionReview admissionv1.AdmissionReview
    if err := json.NewDecoder(r.Body).Decode(&admissionReview); err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

    // Extract the object from the request
    var deployment appsv1.Deployment
    if err := json.Unmarshal(admissionReview.Request.Object.Raw, &deployment); err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

    // Validate: check that all containers have resource limits
    allowed := true
    message := ""
    for _, c := range deployment.Spec.Template.Spec.Containers {
        if c.Resources.Limits.Cpu().IsZero() || c.Resources.Limits.Memory().IsZero() {
            allowed = false
            message = fmt.Sprintf("container %q must have CPU and memory limits", c.Name)
            break
        }
    }

    // Build the response
    response := admissionv1.AdmissionReview{
        TypeMeta: metav1.TypeMeta{APIVersion: "admission.k8s.io/v1", Kind: "AdmissionReview"},
        Response: &admissionv1.AdmissionResponse{
            UID:     admissionReview.Request.UID,
            Allowed: allowed,
            Result:  &metav1.Status{Message: message},
        },
    }

    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(response)
}

The webhook must serve HTTPS. The API server will not call plain HTTP endpoints. Use cert-manager to provision and rotate certificates automatically:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: webhook-cert
  namespace: webhook-system
spec:
  secretName: webhook-tls
  dnsNames:
  - webhook-server.webhook-system.svc
  - webhook-server.webhook-system.svc.cluster.local
  issuerRef:
    name: cluster-issuer
    kind: ClusterIssuer

Mount the resulting secret into the webhook deployment and configure your HTTP server to use those TLS files.

ValidatingAdmissionPolicy: CEL-Based Validation#

Starting with Kubernetes 1.28 (GA), you can write simple validation rules directly in the cluster using Common Expression Language (CEL), without deploying a webhook server:

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
  name: require-resource-limits
spec:
  matchConstraints:
    resourceRules:
    - apiGroups: ["apps"]
      apiVersions: ["v1"]
      operations: ["CREATE", "UPDATE"]
      resources: ["deployments"]
  validations:
  - expression: >
      object.spec.template.spec.containers.all(c,
        has(c.resources) && has(c.resources.limits) &&
        has(c.resources.limits.cpu) && has(c.resources.limits.memory))
    message: "All containers must have CPU and memory limits"
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicyBinding
metadata:
  name: require-resource-limits-binding
spec:
  policyName: require-resource-limits
  validationActions: ["Deny"]
  matchResources:
    namespaceSelector:
      matchExpressions:
      - key: kubernetes.io/metadata.name
        operator: NotIn
        values: ["kube-system"]

ValidatingAdmissionPolicy is faster (no network call), easier to manage (no separate server), and sufficient for many common validation cases. Use external webhooks when you need to call external systems, perform complex logic, or mutate objects.

Common Gotchas#

Self-referential deadlock. A webhook that matches pod creation in its own namespace will prevent its own pods from starting. Always use namespaceSelector to exclude the webhook’s namespace, or deploy the webhook to a namespace like webhook-system and exclude it from matching rules.

failurePolicy: Fail + webhook outage. If your webhook is down and failurePolicy is Fail, no matching resources can be created anywhere in the cluster. For non-critical policies, use Ignore. For critical security policies, ensure the webhook has high availability (multiple replicas, PodDisruptionBudget).

Performance. Every matching API request adds a network round-trip to the webhook. Set timeoutSeconds low, keep webhook logic fast, and scope rules as narrowly as possible. Do not match * resources unless you genuinely need to intercept everything.

Certificate expiry. Webhook TLS certificates must stay valid. If the cert expires, the API server cannot reach the webhook, and with failurePolicy: Fail your cluster is effectively locked. Use cert-manager with automatic renewal.