AKS Networking and Ingress#

AKS networking involves three layers: how pods communicate (CNI plugin), how traffic enters the cluster (load balancers and ingress controllers), and how the cluster connects to other Azure resources (VNet integration, private endpoints). Each layer has Azure-specific behavior that differs from generic Kubernetes.

Azure Load Balancer for Services#

When you create a Service of type LoadBalancer in AKS, Azure provisions a Standard SKU Azure Load Balancer. AKS manages the load balancer rules and health probes automatically.

apiVersion: v1
kind: Service
metadata:
  name: web-app
  annotations:
    # Use an internal load balancer (no public IP)
    service.beta.kubernetes.io/azure-load-balancer-internal: "true"
    # Place in a specific subnet (for internal LBs)
    service.beta.kubernetes.io/azure-load-balancer-internal-subnet: "internal-lb-subnet"
spec:
  type: LoadBalancer
  ports:
    - port: 443
      targetPort: 8443
  selector:
    app: web-app

Without the internal annotation, AKS assigns a public IP. Key annotations:

# Specify a static public IP (must exist in the node resource group)
service.beta.kubernetes.io/azure-load-balancer-resource-group: "mc_myapp-rg_myapp-aks_eastus2"
service.beta.kubernetes.io/azure-pip-name: "myapp-public-ip"

# Health probe configuration
service.beta.kubernetes.io/azure-load-balancer-health-probe-protocol: "http"
service.beta.kubernetes.io/azure-load-balancer-health-probe-request-path: "/healthz"

# Idle timeout (default 4 minutes, max 30)
service.beta.kubernetes.io/azure-load-balancer-tcp-idle-timeout: "10"

A common mistake: creating a static IP in your own resource group instead of the node resource group (MC_*). The AKS managed identity has permissions on the node resource group, not your custom one. Either create the IP in the MC_* group or assign the Network Contributor role on your resource group to the AKS identity.

Ingress Controllers: AGIC vs nginx-ingress#

AKS supports two main ingress approaches.

Application Gateway Ingress Controller (AGIC)#

AGIC uses Azure Application Gateway as the ingress controller. The controller runs as a pod in your cluster and syncs Ingress resources to Application Gateway configuration. Application Gateway operates outside the cluster as a Layer 7 load balancer with WAF capabilities.

# Enable AGIC as an AKS add-on
az aks enable-addons \
  --resource-group myapp-rg \
  --name myapp-aks \
  --addons ingress-appgw \
  --appgw-subnet-cidr "10.225.0.0/16" \
  --appgw-name myapp-appgw

Ingress resource for AGIC:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: app-ingress
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/ssl-redirect: "true"
    appgw.ingress.kubernetes.io/backend-protocol: "http"
    appgw.ingress.kubernetes.io/health-probe-path: "/healthz"
    appgw.ingress.kubernetes.io/waf-policy-for-path: "/subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.Network/applicationGatewayWebApplicationFirewallPolicies/myapp-waf"
spec:
  tls:
    - hosts:
        - app.example.com
      secretName: app-tls
  rules:
    - host: app.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: web-app
                port:
                  number: 80

AGIC advantages: native WAF integration, Azure-managed SSL termination, autoscaling at the gateway level, no in-cluster proxy consuming pod resources. Disadvantages: slower sync times (changes take 30-60 seconds to reflect in Application Gateway), fewer configuration knobs than nginx, debugging requires checking both the AGIC pod logs and Application Gateway diagnostics.

nginx-ingress on AKS#

For teams familiar with nginx-ingress from other platforms, it works on AKS with no special configuration. Install it behind an Azure Load Balancer:

helm install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx --create-namespace \
  --set controller.service.annotations."service\.beta\.kubernetes\.io/azure-load-balancer-health-probe-request-path"=/healthz \
  --set controller.service.annotations."service\.beta\.kubernetes\.io/azure-load-balancer-internal"=true

Use nginx-ingress when you need fast iteration, regex-based routing, rate limiting, or custom Lua snippets. Use AGIC when you need WAF, native Azure monitoring integration, or want to avoid running a reverse proxy inside the cluster.

Private Clusters#

A private AKS cluster disables the public API server endpoint. The API server gets a private IP in your VNet, accessible only from within the network or via peered networks and VPN/ExpressRoute.

az aks create \
  --resource-group myapp-rg \
  --name myapp-private-aks \
  --enable-private-cluster \
  --private-dns-zone system \
  --network-plugin azure \
  --network-plugin-mode overlay \
  --vnet-subnet-id <subnet-id> \
  --enable-managed-identity

The --private-dns-zone system option creates an Azure Private DNS zone automatically. You can also use none (and manage DNS yourself) or provide your own Private DNS zone resource ID.

To access a private cluster, you need network connectivity to the VNet. Options: an Azure VM jumpbox in the same VNet, Azure Bastion, VPN Gateway, or az aks command invoke which runs kubectl commands through the Azure API without direct network access:

az aks command invoke \
  --resource-group myapp-rg \
  --name myapp-private-aks \
  --command "kubectl get nodes"

ExternalDNS with Azure DNS#

ExternalDNS automates DNS record creation for Services and Ingresses. On AKS, it integrates with Azure DNS zones.

# Create a managed identity for ExternalDNS
az identity create --resource-group myapp-rg --name externaldns-identity

# Assign DNS Zone Contributor on your DNS zone
az role assignment create \
  --assignee <externaldns-identity-client-id> \
  --role "DNS Zone Contributor" \
  --scope /subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.Network/dnsZones/example.com

Deploy ExternalDNS with the Azure provider:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: external-dns
  namespace: kube-system
spec:
  template:
    spec:
      serviceAccountName: external-dns
      containers:
        - name: external-dns
          image: registry.k8s.io/external-dns/external-dns:v0.14.0
          args:
            - --source=service
            - --source=ingress
            - --provider=azure
            - --azure-resource-group=myapp-rg
            - --azure-subscription-id=<sub-id>
            - --domain-filter=example.com
            - --txt-prefix=externaldns-

With Workload Identity configured, ExternalDNS authenticates to Azure DNS using federated credentials – no secrets to manage.

Network Security Groups#

AKS automatically manages NSG rules on the subnet for load balancer health probes and node communication. Do not add deny-all rules to the AKS subnet NSG or you will break cluster operations. If you need to restrict traffic, use Kubernetes NetworkPolicy objects instead. AKS supports Azure Network Policy (for Azure CNI) and Calico Network Policy (for both kubenet and Azure CNI).

# Enable Azure Network Policy at cluster creation
az aks create ... --network-policy azure

# Or Calico
az aks create ... --network-policy calico

Network Policy is a create-time decision and cannot be added to an existing cluster without recreating it.