SSH Key Management#

SSH keys replace password authentication with cryptographic key pairs. The choice of algorithm matters:

Ed25519 (recommended): Based on elliptic curve cryptography. Produces small keys (256 bits) that are faster and more secure than RSA. Supported by OpenSSH 6.5+ (2014) – virtually all modern systems.

ssh-keygen -t ed25519 -C "user@hostname"

RSA 4096 (legacy compatibility): Use only when connecting to systems that do not support Ed25519. Always use 4096 bits – the default 3072 is adequate but 4096 provides a safety margin.

ssh-keygen -t rsa -b 4096 -C "user@hostname"

Key rotation practices: Generate new keys periodically (annually is reasonable). Distribute the new public key to all servers before decommissioning the old one. Use different keys for different trust levels (personal servers vs. production infrastructure). Keep private keys encrypted with a passphrase and use ssh-agent so you only enter the passphrase once per session.

# Start ssh-agent and add key
eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519          # prompts for passphrase once

sshd_config Hardening#

The SSH daemon configuration at /etc/ssh/sshd_config controls how the server accepts connections. A hardened configuration:

# /etc/ssh/sshd_config

# Protocol and port
Protocol 2                        # only SSHv2 (SSHv1 is broken)
Port 2222                         # non-standard port reduces automated scan noise

# Authentication
PasswordAuthentication no          # keys only -- most impactful single change
PermitRootLogin no                 # or "prohibit-password" to allow key-based root login
PubkeyAuthentication yes
MaxAuthTries 3                     # lock out after 3 failed attempts
LoginGraceTime 60                  # 60 seconds to authenticate before disconnect
PermitEmptyPasswords no

# Access control
AllowUsers deploy admin            # whitelist specific users
# Or use groups:
# AllowGroups ssh-users

# Session management
ClientAliveInterval 300            # send keepalive every 5 minutes
ClientAliveCountMax 2              # disconnect after 2 missed keepalives (10 min idle timeout)
MaxSessions 3                      # max simultaneous sessions per connection

# Security hardening
X11Forwarding no                   # disable unless needed
AllowTcpForwarding no              # disable unless needed for tunneling
AllowAgentForwarding no            # disable -- see agent forwarding risks below

After editing, validate and reload:

sshd -t                            # test configuration syntax
systemctl reload sshd              # apply changes without disconnecting existing sessions

Using a non-standard port (like 2222) is not real security – an attacker who scans your host will find it. But it dramatically reduces noise from automated bots scanning port 22 across the entire internet. Your auth.log goes from thousands of entries per day to nearly zero, making it easier to spot real attacks.

Agent Forwarding: Convenient but Dangerous#

SSH agent forwarding (ssh -A) lets you use your local SSH keys on remote servers. When you SSH from server A to server B, server B can use your keys through the forwarded agent socket.

The security problem: anyone with root access on the intermediate server can use your forwarded agent. The agent socket is a Unix domain socket that root can access. If server A is compromised, the attacker can authenticate as you to any server your key has access to, for the duration of your connection.

# Dangerous -- your keys are exposed on the intermediate server
ssh -A bastion
ssh internal-server              # uses forwarded agent from your laptop

# Safe alternative -- ProxyJump
ssh -J bastion internal-server   # your keys never leave your laptop

ProxyJump creates a tunnel through the bastion to the internal server. Your SSH client negotiates directly with the final destination – the bastion only forwards encrypted TCP traffic. Your private keys and agent are never exposed on the bastion host.

Bastion Hosts / Jump Boxes#

A bastion host is a dedicated, hardened SSH entry point to your network. All SSH access to internal servers goes through the bastion, creating a single point for authentication, authorization, and audit logging.

Bastion Hardening#

The bastion should be minimally configured:

  • No unnecessary software installed
  • No user data or applications
  • Restrictive sshd_config (no agent forwarding, no TCP forwarding unless required)
  • All sessions logged (shell command logging via auditd or script)
  • Multi-factor authentication where possible
  • Regular security updates applied promptly

ProxyJump Configuration#

Configure ProxyJump in ~/.ssh/config to make it transparent:

# ~/.ssh/config

Host bastion
    HostName bastion.example.com
    User admin
    Port 2222
    IdentityFile ~/.ssh/id_ed25519_infra

Host internal-*
    ProxyJump bastion
    User deploy
    IdentityFile ~/.ssh/id_ed25519_infra

Host internal-web
    HostName 10.0.1.10

Host internal-db
    HostName 10.0.1.20

Now ssh internal-web automatically jumps through the bastion. The connection is transparent – your SSH client connects directly to the internal host through a tunnel in the bastion connection.

For multiple jump points (bastion chains):

ssh -J bastion1,bastion2 internal-server

SSH Certificates#

SSH certificates solve the key distribution problem at scale. Instead of copying public keys to every server’s authorized_keys file, you create a Certificate Authority (CA) and sign user keys. Servers trust the CA, and any key signed by the CA is accepted.

Setting Up a Certificate Authority#

# Generate the CA key pair (keep the private key extremely secure)
ssh-keygen -t ed25519 -f ca_key -C "SSH CA"

# Sign a user's public key
ssh-keygen -s ca_key \
    -I "jane.doe" \
    -n "deploy,admin" \
    -V +52w \
    user_key.pub

# This creates user_key-cert.pub

Parameters:

  • -s ca_key: Sign with this CA private key
  • -I "jane.doe": Key identifier (appears in logs)
  • -n "deploy,admin": Principals (usernames) the certificate is valid for
  • -V +52w: Valid for 52 weeks (time-limited access)

Configuring the Server to Trust the CA#

On each server, add one line to sshd_config:

TrustedUserCAKeys /etc/ssh/ca_key.pub

Copy the CA public key (not the private key) to /etc/ssh/ca_key.pub on every server. That is the entire server-side setup. No authorized_keys management, no key distribution scripts, no configuration management for public keys.

Advantages Over authorized_keys#

  • Scalable: Add one CA public key to servers, not hundreds of user keys.
  • Time-limited: Certificates expire. Revocation is automatic when the certificate’s validity period ends.
  • Auditable: The key ID (-I) appears in auth logs, so you can trace who logged in even if multiple people share a username.
  • No key distribution: New users get their key signed by the CA. No need to update authorized_keys on every server.

Revoking Certificates#

Before a certificate’s natural expiration:

# Create or update a revocation list
ssh-keygen -k -f /etc/ssh/revoked_keys -s ca_key user_key.pub

In sshd_config:

RevokedKeys /etc/ssh/revoked_keys

SSH Tunneling#

SSH tunnels securely access services that are not directly reachable.

Local forwarding (-L): Access a remote service as if it were local. Your machine listens on a local port and forwards traffic through the SSH connection to the remote destination.

# Access remote PostgreSQL (port 5432) via localhost:5432
ssh -L 5432:db.internal:5432 bastion

# Access remote web app via localhost:8080
ssh -L 8080:webapp.internal:80 bastion

Remote forwarding (-R): Make a local service accessible on the remote machine. The remote machine listens on a port and forwards traffic back through the SSH connection to your local machine.

# Expose local dev server (port 3000) on the remote machine's port 3000
ssh -R 3000:localhost:3000 remote-server

Dynamic forwarding (-D): Creates a SOCKS proxy. All traffic routed through the SOCKS proxy goes through the SSH tunnel.

# Create SOCKS proxy on localhost:1080
ssh -D 1080 bastion

# Configure browser or application to use SOCKS5 proxy at localhost:1080

SSH Config File#

The SSH config file (~/.ssh/config) eliminates typing long commands and standardizes connection settings:

# ~/.ssh/config

# Global defaults
Host *
    ServerAliveInterval 60
    ServerAliveCountMax 3
    AddKeysToAgent yes
    IdentitiesOnly yes

# Production infrastructure
Host prod-*
    User deploy
    IdentityFile ~/.ssh/id_ed25519_prod
    ProxyJump bastion-prod

# Staging infrastructure
Host staging-*
    User deploy
    IdentityFile ~/.ssh/id_ed25519_staging
    ProxyJump bastion-staging

# Specific hosts
Host bastion-prod
    HostName bastion.prod.example.com
    User admin
    Port 2222
    IdentityFile ~/.ssh/id_ed25519_prod

Host prod-web-1
    HostName 10.0.1.10

Host prod-db-1
    HostName 10.0.1.20
    LocalForward 5432 localhost:5432

IdentitiesOnly yes prevents SSH from trying every key in your agent – it only offers the key specified by IdentityFile. This avoids accidentally triggering lockout policies on servers that count failed key attempts.

Modern Access Management Tools#

For organizations that outgrow manual SSH management:

Teleport: Open-source SSH access platform with SSO integration (OIDC, SAML), role-based access control, session recording (video playback of terminal sessions), and certificate-based authentication. It acts as a certificate authority and issues short-lived certificates after SSO authentication, eliminating long-lived keys entirely.

HashiCorp Boundary: Zero-trust access management that brokers connections without exposing network details. Users authenticate through an identity provider and Boundary creates a tunneled connection to the target. No SSH keys, no bastion hosts, no VPN required.

Common Gotchas#

File permissions: SSH is strict about permissions. If permissions are wrong, SSH silently falls back to password authentication (or fails entirely with key-only auth).

chmod 700 ~/.ssh                   # directory
chmod 600 ~/.ssh/id_ed25519        # private key
chmod 644 ~/.ssh/id_ed25519.pub    # public key
chmod 644 ~/.ssh/authorized_keys   # authorized keys
chmod 644 ~/.ssh/config            # config file

The private key permission check is a hard fail – SSH refuses to use a private key with permissions more open than 600. The authorized_keys and directory checks are enforced by StrictModes yes in sshd_config (enabled by default).

Agent forwarding through untrusted servers: As discussed above, use ProxyJump instead. If you must use agent forwarding, limit it to specific hosts in your SSH config:

Host trusted-bastion
    ForwardAgent yes

Host *
    ForwardAgent no

Key accumulation: Over time, servers accumulate stale public keys in authorized_keys from former employees and decommissioned systems. This is a significant security risk. SSH certificates with expiration dates solve this structurally. For authorized_keys-based environments, audit and prune regularly.