Multi-Cloud Networking Patterns#

Multi-cloud networking connects workloads across two or more cloud providers into a coherent network. The motivations vary – vendor redundancy, best-of-breed service selection, regulatory requirements – but the challenges are the same: private connectivity between isolated networks, consistent service discovery, and traffic routing that handles failures.

VPN Tunnels Between Clouds#

IPsec VPN tunnels are the simplest way to connect two cloud networks. Each provider offers managed VPN gateways that terminate IPsec tunnels, encrypting traffic between VPCs without exposing it to the public internet.

AWS to GCP VPN#

# AWS side
resource "aws_vpn_gateway" "main" {
  vpc_id = aws_vpc.main.id
}

resource "aws_customer_gateway" "gcp" {
  bgp_asn    = 65002
  ip_address = google_compute_ha_vpn_gateway.main.vpn_interfaces[0].ip_address
  type       = "ipsec.1"
}

resource "aws_vpn_connection" "to_gcp" {
  vpn_gateway_id      = aws_vpn_gateway.main.id
  customer_gateway_id = aws_customer_gateway.gcp.id
  type                = "ipsec.1"
  static_routes_only  = false  # use BGP
}

# GCP side
resource "google_compute_ha_vpn_gateway" "main" {
  name    = "gcp-to-aws-vpn"
  network = google_compute_network.main.id
  region  = "us-central1"
}

resource "google_compute_vpn_tunnel" "to_aws" {
  name                            = "tunnel-to-aws"
  vpn_gateway                     = google_compute_ha_vpn_gateway.main.id
  vpn_gateway_interface           = 0
  peer_external_gateway           = google_compute_external_vpn_gateway.aws.id
  peer_external_gateway_interface = 0
  shared_secret                   = aws_vpn_connection.to_gcp.tunnel1_preshared_key
  router                          = google_compute_router.vpn_router.id
}

resource "google_compute_router" "vpn_router" {
  name    = "vpn-router"
  network = google_compute_network.main.id
  bgp { asn = 65002 }
}

BGP dynamically exchanges routes between clouds. When subnets change on either side, BGP propagates without manual route table updates. Managed VPN gateways provide about 1.25 Gbps per tunnel. For higher throughput, create multiple tunnels with ECMP routing.

AWS to Azure VPN#

The Azure side uses a Virtual Network Gateway:

resource "azurerm_virtual_network_gateway" "main" {
  name     = "azure-to-aws-vng"
  type     = "Vpn"
  vpn_type = "RouteBased"
  sku      = "VpnGw2"

  bgp_settings { asn = 65003 }

  ip_configuration {
    public_ip_address_id = azurerm_public_ip.vpn.id
    subnet_id            = azurerm_subnet.gateway.id
  }
}

resource "azurerm_virtual_network_gateway_connection" "to_aws" {
  type                       = "IPsec"
  virtual_network_gateway_id = azurerm_virtual_network_gateway.main.id
  local_network_gateway_id   = azurerm_local_network_gateway.aws.id
  shared_key                 = aws_vpn_connection.to_azure.tunnel1_preshared_key
  enable_bgp                 = true
}

Transit Gateways#

Point-to-point tunnels create an O(n^2) mesh when connecting many networks. Transit gateways act as a central hub.

resource "aws_ec2_transit_gateway" "hub" {
  description                     = "multi-cloud-hub"
  default_route_table_association = "enable"
  default_route_table_propagation = "enable"
}

# Attach VPCs
resource "aws_ec2_transit_gateway_vpc_attachment" "app_vpc" {
  transit_gateway_id = aws_ec2_transit_gateway.hub.id
  vpc_id             = aws_vpc.app.id
  subnet_ids         = aws_subnet.private[*].id
}

# Attach VPN to GCP through the same gateway
resource "aws_vpn_connection" "gcp" {
  transit_gateway_id  = aws_ec2_transit_gateway.hub.id
  customer_gateway_id = aws_customer_gateway.gcp.id
  type                = "ipsec.1"
}

Route table segmentation on the transit gateway controls which networks can reach each other, enabling production/staging isolation through the same hub.

Cloud Interconnect Services#

VPN tunnels run over the public internet. Interconnect services provide dedicated physical connections with lower latency and higher bandwidth.

AWS Direct Connect: 1 Gbps or 10 Gbps dedicated, or 50 Mbps-10 Gbps hosted via partners
Google Cloud Interconnect: 10-100 Gbps dedicated, or 50 Mbps-50 Gbps via partners
Azure ExpressRoute: 50 Mbps to 100 Gbps through connectivity providers

For multi-cloud without on-premises infrastructure, use a colocation exchange like Equinix or Megaport to connect to multiple cloud providers from one location. Megaport offers virtual cross-connects provisioned programmatically.

Cost consideration: interconnect involves monthly port fees ($200-$2000+), data transfer charges, and colocation costs. VPN tunnels cost only gateway hours and data transfer. Use interconnect for bandwidth-sensitive workloads, VPN for management traffic.

Service Mesh Across Clusters#

When services in different clouds need application-layer communication, a service mesh provides mTLS, load balancing, and observability across cloud boundaries.

Istio Multi-Cluster#

# Install on primary cluster (AWS)
istioctl install --set profile=default \
  --set values.global.meshID=multi-cloud-mesh \
  --set values.global.multiCluster.clusterName=aws-cluster \
  --set values.global.network=aws-network

# Create remote secret for the secondary cluster
istioctl create-remote-secret --name=gcp-cluster \
  --context=gcp-context | kubectl apply -f - --context=aws-context

# Install on remote cluster (GCP) pointing to primary's istiod
istioctl install --set profile=remote \
  --set values.global.meshID=multi-cloud-mesh \
  --set values.global.multiCluster.clusterName=gcp-cluster \
  --set values.global.remotePilotAddress=<aws-istiod-address>

Services in either cluster can now call services in the other using Kubernetes service names, with Istio handling cross-cluster routing and mTLS.

Consul Cluster Peering#

HashiCorp Consul connects clusters through peering tokens, exporting specific services to peers:

resource "consul_config_entry" "export_api" {
  kind = "exported-services"
  name = "default"
  config_json = jsonencode({
    Services = [{
      Name      = "api-service"
      Consumers = [{ Peer = "gcp-cluster" }]
    }]
  })
}

DNS-Based Routing#

DNS is the simplest cross-cloud traffic management layer.

Failover Routing#

resource "aws_route53_record" "api_primary" {
  zone_id        = aws_route53_zone.main.zone_id
  name           = "api.example.com"
  type           = "A"
  set_identifier = "primary"
  failover_routing_policy { type = "PRIMARY" }
  health_check_id = aws_route53_health_check.aws_api.id
  alias {
    name    = aws_lb.api.dns_name
    zone_id = aws_lb.api.zone_id
    evaluate_target_health = true
  }
}

resource "aws_route53_record" "api_secondary" {
  zone_id        = aws_route53_zone.main.zone_id
  name           = "api.example.com"
  type           = "A"
  set_identifier = "secondary"
  failover_routing_policy { type = "SECONDARY" }
  health_check_id = aws_route53_health_check.gcp_api.id
  ttl     = 60
  records = ["<gcp-lb-ip>"]
}

Weighted routing enables gradual migration (90/10 split, adjusting over time). Latency routing directs users to the lowest-latency endpoint automatically. Set DNS TTLs to 60 seconds during migration for fast failback.

IP Address Planning#

Overlapping CIDRs are the most common multi-cloud networking failure. Plan allocations centrally:

AWS:     10.0.0.0/12   (10.0.0.0 - 10.15.255.255)
GCP:     10.16.0.0/12  (10.16.0.0 - 10.31.255.255)
Azure:   10.32.0.0/12  (10.32.0.0 - 10.47.255.255)
On-prem: 10.48.0.0/12  (10.48.0.0 - 10.63.255.255)

Overlapping CIDRs mean VPN tunnels and peering connections cannot route correctly, and the fix requires re-IP-addressing one side – a disruptive operation.

Choosing the Right Pattern#

Requirement	Pattern	Tradeoffs
Low bandwidth cross-cloud calls	VPN tunnels	Simple, low cost, internet latency
High bandwidth, latency-sensitive	Cloud interconnect	Higher cost, needs colocation or partner
Many networks to connect	Transit gateway	Centralized routing, avoids mesh
Application-layer service calls	Service mesh	Complex setup, rich traffic management
Active-passive failover	DNS failover	Simple, TTL limits failover speed
Gradual traffic migration	DNS weighted routing	Coarse-grained, client caching affects distribution

Most deployments combine patterns: VPN or interconnect for network connectivity, a transit gateway for routing, and DNS for external traffic management. Add a service mesh when cross-cloud service-to-service communication is frequent enough to justify the operational overhead.