SSHwatch Insights Blog

SSH Jump Servers: Building a Secure Access Gateway for Your Infrastructure

In today’s complex network environments, direct SSH access to multiple servers creates unnecessary security risks. Each exposed SSH endpoint represents a potential attack vector for malicious actors scanning the internet for vulnerable systems. Enter the SSH jump server (also known as a bastion host)—a dedicated gateway that serves as the single entry point to your infrastructure, dramatically reducing your attack surface while providing enhanced logging and access control capabilities.

What Is an SSH Jump Server?

A jump server is a hardened system that sits between your external network and your internal infrastructure. Instead of allowing direct SSH connections to all your servers from potentially untrusted networks, users connect first to the jump server, which then provides controlled access to internal systems.

Think of it as a security checkpoint where all visitors must pass through and be validated before gaining access to the rest of your facility. Just as a physical security checkpoint reduces the number of entry points that need to be monitored, a jump server creates a single choke point for SSH access that can be heavily secured and audited.

This creates several immediate security benefits:

  • Reduced attack surface with only one exposed SSH endpoint
  • Centralized authentication and access control
  • Comprehensive logging of all SSH sessions
  • Simplified firewall configuration
  • Consistent application of security policies

Without a jump server, each system in your infrastructure that’s directly accessible via SSH becomes a potential target for attackers. If any one of those systems has inadequate security configurations or unpatched vulnerabilities, your entire network could be at risk.

Why Traditional SSH Access Methods Fall Short

Before we dive into setting up a jump server, it’s worth understanding why traditional direct SSH access can be problematic in modern environments:

  1. Inconsistent Security: When administrators configure each server independently, security standards can drift over time, creating inconsistencies.
  2. Difficult Auditing: With direct access to multiple servers, tracking who accessed what and when becomes a challenging task spread across many systems.
  3. Complex Firewall Rules: Each server requiring external SSH access means more firewall rules to manage and maintain.
  4. Key Management Challenges: Distributing and rotating SSH keys across numerous servers becomes an operational burden.
  5. No Central Visibility: Security incidents may go unnoticed without centralized logging and monitoring.

A jump server addresses all these challenges by creating a centralized access point with standardized security controls.

Setting Up Your Linux Jump Server

Let’s walk through the process of configuring a secure SSH jump server on Linux. We’ll cover each step in detail with clear explanations of the commands used.

Step 1: Choose the Right System

Your jump server should be a dedicated system with minimal services running. A lean Linux distribution like Ubuntu Server or Debian is ideal. The hardware requirements are modest – even a small virtual machine with 1-2 CPUs, 2GB RAM, and 20GB storage is sufficient for most use cases.

Step 2: Harden Your Jump Server

Since your jump server will be a critical security component and potentially exposed to the internet, it needs to be especially hardened against attacks:

# Update system packages to ensure all security patches are applied
sudo apt update && sudo apt upgrade -y

This command updates the package list and applies all available updates to ensure your system has the latest security patches.

# Install fail2ban to protect against brute force attacks
sudo apt install -y fail2ban

Fail2ban monitors authentication logs and temporarily blocks IP addresses that show malicious signs, such as multiple failed login attempts. This is critical for a server that will be accessible from the internet.

Now, let’s configure the SSH server for enhanced security:

# Configure SSH to use only key-based authentication
sudo sed -i 's/#PasswordAuthentication yes/PasswordAuthentication no/' /etc/ssh/sshd_config

This command edits the SSH configuration file to disable password authentication. By allowing only key-based authentication, you significantly reduce the risk of brute force attacks.

# Ensure public key authentication is enabled
sudo sed -i 's/#PubkeyAuthentication yes/PubkeyAuthentication yes/' /etc/ssh/sshd_config

This ensures that public key authentication is explicitly enabled, which is the secure authentication method we want to enforce.

# Disable root login
sudo sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin no/' /etc/ssh/sshd_config

Disabling direct root login is a security best practice. Users should always log in with their own accounts and use sudo for administrative tasks, which provides better accountability.

# Set a more restrictive set of allowed ciphers and key exchange algorithms
echo "Ciphers [email protected],[email protected],[email protected],aes256-ctr,aes192-ctr,aes128-ctr" | sudo tee -a /etc/ssh/sshd_config
echo "KexAlgorithms curve25519-sha256,[email protected],diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha256" | sudo tee -a /etc/ssh/sshd_config
echo "MACs [email protected],[email protected],[email protected]" | sudo tee -a /etc/ssh/sshd_config

These commands restrict SSH to use only strong, modern cryptographic algorithms, enhancing the security of all SSH connections to the jump server.

# Restart SSH service to apply changes
sudo systemctl restart sshd

This command restarts the SSH service to apply all the configuration changes we’ve made.

Step 3: Configure a Dedicated Jump Server User

It’s a best practice to create dedicated user accounts for access to the jump server:

# Create a new user group for jump server access
sudo groupadd jumpusers

# Create a new user with restricted shell access
sudo useradd -m -s /bin/rbash -G jumpusers jumpuser

# Set up the authorized_keys file for this user
sudo mkdir -p /home/jumpuser/.ssh
sudo touch /home/jumpuser/.ssh/authorized_keys
sudo chmod 700 /home/jumpuser/.ssh
sudo chmod 600 /home/jumpuser/.ssh/authorized_keys
sudo chown -R jumpuser:jumpuser /home/jumpuser/.ssh

This sequence creates a dedicated user with a restricted bash shell (rbash) that limits what commands can be executed, and sets up the proper directory structure for SSH authentication.

Now you can add user public keys to the authorized_keys file. For improved security, you can restrict what each key can do:

# Example of a restricted SSH key entry
echo 'from="10.0.0.0/24",command="ssh internaluser@server1" ssh-rsa AAAAB3Nz...' | sudo tee -a /home/jumpuser/.ssh/authorized_keys

This example only allows connections from the 10.0.0.0/24 network and restricts the key to only executing the specific command to connect to server1.

Step 4: Configure Internal Servers

Your internal servers should be configured to only accept SSH connections from the jump server. This creates a secure perimeter where only the jump server can initiate connections to your infrastructure.

On each internal server, implement the following firewall rules:

# On each internal server, configure the firewall to only allow SSH from the jump server's IP
sudo ufw allow from JUMP_SERVER_IP to any port 22
sudo ufw enable

Replace JUMP_SERVER_IP with the actual IP address of your jump server. This UFW (Uncomplicated Firewall) command creates a rule that only allows incoming SSH connections from your jump server, blocking all other SSH access attempts.

Additionally, it’s a good idea to harden the SSH configuration on internal servers as well:

# Edit the SSH config to only allow key-based authentication
sudo sed -i 's/#PasswordAuthentication yes/PasswordAuthentication no/' /etc/ssh/sshd_config

# Restart SSH service
sudo systemctl restart sshd

These commands follow the same pattern as on the jump server – disabling password authentication in favor of more secure key-based authentication.

Step 5: Set Up User Access Methods

Now, let’s set up SSH configurations to streamline user access through the jump server. Users can connect through a jump server in several ways, each with different security implications and convenience trade-offs.

Method 1: SSH ProxyJump (Modern and Recommended)

The ProxyJump directive (available in OpenSSH 7.3+) is the cleanest and most secure way to use jump servers. It handles the multi-hop connection automatically and securely.

Users should configure their SSH client by editing their ~/.ssh/config file:

# In the user's ~/.ssh/config file
Host jumpserver
    HostName jump.example.com
    User jumpuser
    IdentityFile ~/.ssh/jump_key
    IdentitiesOnly yes

Host internal-server
    HostName 10.0.0.5
    User internaluser
    IdentityFile ~/.ssh/internal_key
    ProxyJump jumpserver
    IdentitiesOnly yes

Let’s break down what each of these configuration lines does:

  • Host jumpserver – Creates a named SSH host configuration called “jumpserver”
  • HostName jump.example.com – Specifies the actual hostname or IP address of the jump server
  • User jumpuser – Sets the username to use when connecting to the jump server
  • IdentityFile ~/.ssh/jump_key – Specifies the SSH private key to use for authentication
  • IdentitiesOnly yes – Ensures only the specified key is used, preventing key leakage

The second host configuration for the internal server includes:

  • ProxyJump jumpserver – This is the key directive that tells SSH to connect to this server through the jumpserver host we defined above
  • All other options work the same as for the jump server configuration

With this configuration, users can simply run:

ssh internal-server

And SSH will automatically connect through the jump server, handling all the authentication and forwarding without any additional user input. The connection is secure because:

  1. Authentication happens separately for each hop
  2. The connection is encrypted end-to-end
  3. No agent forwarding is needed
  4. Keys never leave the user’s system

Method 2: SSH Agent Forwarding (Use with Caution)

SSH agent forwarding allows your SSH authentication agent to be accessible from the jump server, making it possible to use your local SSH keys for authentication to other servers without copying the keys to the jump server.

# In ~/.ssh/config
Host jumpserver
    HostName jump.example.com
    User jumpuser
    ForwardAgent yes

The ForwardAgent yes directive tells SSH to make your local authentication agent available on the jump server during your session.

With this configuration, you would connect to the jump server first, then to internal servers:

# First, connect to the jump server
ssh jumpserver

# Then, from the jump server, connect to internal servers
# No need to specify a key since the agent is forwarded
ssh [email protected]

Security note: Agent forwarding has significant security implications. If your jump server is compromised, an attacker could potentially use your forwarded credentials to connect to any server your keys have access to while your session is active. This effectively gives the jump server temporary access to your private keys. Use this method only in environments where you have complete trust in the security of the jump server and understand the risks involved.

Method 3: ProxyCommand (For Older SSH Versions)

For older SSH versions that don’t support ProxyJump, the ProxyCommand directive can achieve similar functionality:

# In ~/.ssh/config
Host internal-server
    HostName 10.0.0.5
    User internaluser
    ProxyCommand ssh [email protected] -W %h:%p

The ProxyCommand directive tells SSH to execute the specified command to establish the connection. The -W %h:%p option tells SSH to forward the connection to the hostname and port of the destination. The %h and %p are placeholders that SSH replaces with the destination hostname and port.

This approach works well but is slightly more complex than ProxyJump and doesn’t handle some edge cases as elegantly.

Step 6: Set Up Comprehensive Logging and Monitoring

One of the key benefits of a jump server architecture is centralized logging of all SSH activity. Proper logging is essential for security monitoring, compliance, and forensic analysis in case of incidents.

First, let’s enhance the default SSH logging:

# Enable verbose session logging
sudo sed -i 's/#LogLevel INFO/LogLevel VERBOSE/' /etc/ssh/sshd_config

This increases the detail level in SSH logs, capturing more information about connections, authentication attempts, and session activities. The logs will be written to the system’s standard log files (typically /var/log/auth.log on Debian-based systems).

For more comprehensive session logging, we can set up a forced command that wraps all SSH sessions in a logging script:

# Add a forced command for session logging
echo "ForceCommand /usr/local/bin/ssh-session-logger.sh" | sudo tee -a /etc/ssh/sshd_config

This line tells SSH to execute our custom logging script for all SSH sessions, regardless of what command the user requested. Now, let’s create that script:

# Create the /usr/local/bin directory if it doesn't exist
sudo mkdir -p /usr/local/bin

# Create the session logging script
sudo tee /usr/local/bin/ssh-session-logger.sh > /dev/null << 'EOF'
#!/bin/bash

# Define log directory and filename based on user and timestamp
SESSION_LOG_DIR="/var/log/ssh-sessions"
SESSION_LOG="$SESSION_LOG_DIR/$(whoami)-$(date +%Y%m%d-%H%M%S).log"

# Create log directory if it doesn't exist
mkdir -p "$SESSION_LOG_DIR"

# Log session start with metadata
echo "=== SSH SESSION START ===" > "$SESSION_LOG"
echo "Timestamp: $(date)" >> "$SESSION_LOG"
echo "User: $(whoami)" >> "$SESSION_LOG"
echo "Remote IP: $SSH_CLIENT" >> "$SESSION_LOG"
echo "SSH_ORIGINAL_COMMAND: $SSH_ORIGINAL_COMMAND" >> "$SESSION_LOG"
echo "=== SESSION ACTIVITY FOLLOWS ===" >> "$SESSION_LOG"

# If this is an interactive session, record all activity
if [ -z "$SSH_ORIGINAL_COMMAND" ]; then
    # Execute the user's shell with script to record activity
    script -q -f -a "$SESSION_LOG" "$SHELL"
else
    # For non-interactive sessions, log the command and its output
    echo "Executing command: $SSH_ORIGINAL_COMMAND" >> "$SESSION_LOG"
    echo "Command output:" >> "$SESSION_LOG"
    eval "$SSH_ORIGINAL_COMMAND" 2>&1 | tee -a "$SESSION_LOG"
fi

# Log session end
echo "=== SSH SESSION END ===" >> "$SESSION_LOG"
echo "Timestamp: $(date)" >> "$SESSION_LOG"
EOF

# Make the script executable
sudo chmod +x /usr/local/bin/ssh-session-logger.sh

# Set proper permissions for the log directory
sudo mkdir -p /var/log/ssh-sessions
sudo chmod 750 /var/log/ssh-sessions

This comprehensive script:

  1. Creates a unique log file for each SSH session, named with the username and timestamp
  2. Records metadata about the session (user, IP, timestamp)
  3. Uses the script command to record all terminal activity for interactive sessions
  4. For non-interactive sessions (like SCP or direct commands), logs the command and its output
  5. Records when the session ends

The logs are stored in /var/log/ssh-sessions/ where they can be reviewed later for security analysis or compliance purposes.

For proper log management, set up log rotation to prevent the logs from consuming too much disk space:

# Create a logrotate configuration file for SSH session logs
sudo tee /etc/logrotate.d/ssh-sessions > /dev/null << 'EOF'
/var/log/ssh-sessions/*.log {
    weekly
    rotate 13
    compress
    delaycompress
    missingok
    notifempty
    create 640 root adm
}
EOF

This configuration tells the system to rotate the logs weekly, keep 13 weeks of logs, and compress older logs to save space.

Advanced Features for Enterprise Environments

For larger environments with more complex security requirements, consider implementing these additional security measures:

Role-Based Access Control

In enterprise environments, you’ll want fine-grained control over which users can access which internal servers. LDAP or Active Directory integration can help implement role-based access control:

# Install LDAP authentication packages
sudo apt install -y libpam-ldap nscd

# Configure PAM to use LDAP
sudo pam-auth-update --enable ldap

These commands install the necessary packages to authenticate users against an LDAP directory. After installation, you’ll need to configure the LDAP client by editing /etc/ldap/ldap.conf with your LDAP server details:

# Example LDAP configuration
sudo tee /etc/ldap/ldap.conf > /dev/null << 'EOF'
BASE dc=example,dc=com
URI ldap://ldap.example.com
TLS_CACERT /etc/ssl/certs/ca-certificates.crt
EOF

This configuration tells the system:

  • The base DN (Distinguished Name) for LDAP searches
  • The URI of your LDAP server
  • The path to the CA certificates for secure LDAP connections

You can then use LDAP groups to control access to specific servers by configuring your SSH authorized_keys file with restrictions based on group membership.

Just-In-Time Access with Certificate-Based Authentication

Traditional SSH key management can become cumbersome at scale. Certificate-based authentication provides a more secure and manageable alternative, especially when combined with short-lived certificates for just-in-time access:

# On your CA (Certificate Authority) server, generate a CA key pair
ssh-keygen -t rsa -b 4096 -f /etc/ssh/ca_key -C "SSH Certificate Authority"

This creates a key pair that will function as your SSH Certificate Authority.

On your jump server, configure it to trust certificates signed by your CA:

# Add the CA public key to the SSH configuration
echo "TrustedUserCAKeys /etc/ssh/ca_key.pub" | sudo tee -a /etc/ssh/sshd_config
sudo cp /path/to/ca_key.pub /etc/ssh/ca_key.pub

Now, instead of distributing user public keys to all servers, you can sign their keys with your CA:

# Sign a user's public key with a 24-hour validity
ssh-keygen -s /etc/ssh/ca_key -I "[email protected]" -n "john" -V +24h /path/to/user_key.pub

This creates a signed certificate valid for 24 hours that allows the user “john” to authenticate to any server that trusts your CA.

For automated just-in-time access, you can implement a service that issues short-lived certificates upon request and approval, such as Netflix’s BLESS or HashiCorp Vault’s SSH secrets engine.

Session Recording and Playback

For compliance and security forensics, recording full SSH sessions can be invaluable:

# Install and configure tlog for session recording
sudo apt install -y tlog

Tlog is a terminal I/O recording and playback package. After installation, configure SSH to use it:

# Configure SSH to use tlog for session recording
echo "ForceCommand tlog-rec-session" | sudo tee -a /etc/ssh/sshd_config

This command forces all SSH sessions to be recorded by tlog. The recordings are stored in the system journal by default and can be played back using the tlog-play command:

# Play back a recorded session
tlog-play -r journal -M RECORD_ID

You can retrieve record IDs by searching the journal:

# List all recorded sessions
journalctl -o verbose -t tlog

For long-term storage, configure tlog to write to files instead of the journal:

# Configure tlog to write to files
sudo mkdir -p /var/log/tlog
sudo tee /etc/tlog/tlog-rec-session.conf > /dev/null << 'EOF'
{
    "writer": "file",
    "file": {
        "path": "/var/log/tlog/rec-%Y%m%d-%H%M%S.log"
    }
}
EOF

This configuration tells tlog to write session recordings to files in /var/log/tlog with filenames based on the timestamp.

Integrating with SSHwatch for Real-Time Monitoring

To maximize your jump server’s security posture, integrate it with SSHwatch for real-time monitoring and alerting:

  1. Install the SSHwatch agent on your jump server following the documentation from the SSHwatch website.
  2. Configure comprehensive log forwarding to ensure SSHwatch receives detailed information about all SSH activities:
# Configure SSHwatch agent to monitor SSH logs
sudo tee /etc/sshwatch/config.yaml > /dev/null << 'EOF'
log_sources:
  - path: /var/log/auth.log
    format: syslog
  - path: /var/log/ssh-sessions
    format: custom
    pattern: "=== SSH SESSION START ==="
    timestamp_format: "Timestamp: %Y-%m-%d %H:%M:%S"
EOF
  1. Set up custom alerts in the SSHwatch dashboard for suspicious activities like:
    • Access attempts outside business hours
    • Failed authentication attempts exceeding thresholds
    • Connections from unexpected geographic locations
    • Unusual session durations
    • Access to servers outside a user’s normal pattern
  2. Configure automated responses to potential security incidents, such as:
    • Temporarily blocking IP addresses with suspicious activity
    • Notifying security teams via multiple channels
    • Triggering additional logging for forensic purposes

SSHwatch’s ability to correlate events across your entire infrastructure makes it particularly valuable in a jump server architecture, as it can detect lateral movement attempts and other advanced attack patterns.

Best Practices for Jump Server Management

Implementing a jump server is just the beginning. To maintain a strong security posture, follow these best practices:

1. Minimize Installed Software

Every additional package installed on your jump server increases its attack surface. Follow the principle of minimalism:

# List all installed packages
dpkg -l

# Remove unnecessary packages
sudo apt purge package-name

# Prevent automatic installation of recommended packages
echo 'APT::Install-Recommends "false";' | sudo tee -a /etc/apt/apt.conf.d/99norecommends

Regularly audit installed packages and remove anything not essential for the jump server’s operation.

2. Implement Key Rotation Policies

Static SSH keys represent a security risk, especially in environments with staff turnover. Implement a regular key rotation policy:

# Script to find old SSH keys (keys older than 90 days)
find /home/*/.*ssh/authorized_keys -type f -mtime +90 -exec grep -l '^ssh-' {} \;

Consider using automation to enforce key expiration and rotation, either through custom scripts or integrated solutions like SSHwatch’s key management features.

3. Configure Session Timeouts

Unattended SSH sessions can be exploited if a user leaves their computer. Configure automatic timeouts:

# Set up SSH session timeouts
echo "ClientAliveInterval 300" | sudo tee -a /etc/ssh/sshd_config
echo "ClientAliveCountMax 0" | sudo tee -a /etc/ssh/sshd_config

These settings will disconnect any SSH session that’s inactive for 5 minutes (300 seconds). The zero value for ClientAliveCountMax means the server will disconnect immediately after the timeout period rather than sending additional keep-alive messages.

You can also configure client-side timeouts in the user’s SSH config to ensure consistent timeout behavior:

# In the user's ~/.ssh/config
Host *
    ServerAliveInterval 300
    ServerAliveCountMax 0

4. Implement File Integrity Monitoring

Detect unauthorized changes to critical system files with file integrity monitoring:

# Install AIDE (Advanced Intrusion Detection Environment)
sudo apt install -y aide

# Initialize the AIDE database
sudo aide --init

# Move the new database to the active location
sudo mv /var/lib/aide/aide.db.new /var/lib/aide/aide.db

# Set up a daily check
echo '0 3 * * * root /usr/bin/aide --check' | sudo tee -a /etc/crontab

AIDE creates a database of file checksums and can detect if critical system files are modified unexpectedly, which could indicate a security breach.

5. Regular Security Audits

Schedule periodic security audits of your jump server configuration:

# Install Lynis for security auditing
sudo apt install -y lynis

# Run a security audit
sudo lynis audit system

Lynis will provide a comprehensive security audit of your system, highlighting potential vulnerabilities and recommending hardening measures.

Additionally, perform regular manual reviews of:

  • User accounts and access rights
  • SSH authorized_keys files
  • Firewall rules
  • Log files for suspicious activities
  • System updates and security patches

6. Apply the Principle of Least Privilege

Users should only have access to the specific servers they need for their job functions:

# Example of restricted SSH key in authorized_keys
from="10.0.0.0/24",command="ssh internal-server-1",no-agent-forwarding,no-port-forwarding,no-X11-forwarding ssh-rsa AAAAB3NzaC1yc...

This example restricts the key to:

  • Only connections from the 10.0.0.0/24 network
  • Only allowing a specific command (connection to a specific server)
  • Disabling agent forwarding, port forwarding, and X11 forwarding

Regularly review access permissions and revoke access that is no longer needed.

Real-World Deployment Scenarios

Let’s explore how jump server architectures can be implemented in different environments:

Small Business Deployment

A small business might implement a simple jump server setup:

  1. One hardened jump server in a DMZ network
  2. Internal servers in a protected network
  3. Simple key-based authentication
  4. Basic logging and monitoring

This provides a significant security improvement with minimal complexity and cost.

Enterprise Multi-Region Deployment

Larger enterprises might implement:

  1. Regional jump servers in each geographic location
  2. Hierarchical jump server architecture (tier 1 and tier 2 jump servers)
  3. Integration with enterprise identity management systems
  4. Certificate-based authentication with short-lived certificates
  5. Comprehensive monitoring and alerting with SSHwatch
  6. Full session recording and audit trails

This advanced architecture supports complex compliance requirements while providing scalability across multiple regions.

Cloud-Native Deployment

For cloud environments, consider:

  1. Jump servers as ephemeral infrastructure (rebuilt regularly from infrastructure-as-code definitions)
  2. Integration with cloud IAM services for authentication
  3. Jump server auto-scaling groups for high availability
  4. Cloud-native monitoring and logging solutions integrated with SSHwatch

This approach leverages cloud capabilities while maintaining the security benefits of a jump server architecture.

Conclusion

Implementing an SSH jump server is a powerful way to enhance your infrastructure’s security posture. By funneling all SSH access through a single, hardened point, you gain better control, visibility, and security across your entire environment.

The centralized nature of a jump server creates operational efficiencies as well:

  • Simplified firewall management with fewer open ports
  • Consistent security policy enforcement
  • Easier compliance with regulatory requirements
  • Reduced attack surface for your internal infrastructure
  • Comprehensive audit trails for all administrative access

Remember that a jump server is not a silver bullet—it’s one component of a comprehensive security strategy. Combine it with strong authentication, regular patching, comprehensive logging, and active monitoring like that provided by SSHwatch for a truly robust security posture.

By following the detailed implementation steps and best practices outlined in this guide, you can significantly enhance the security of your Linux infrastructure while improving operational efficiency and compliance capabilities.

Have you implemented a jump server in your environment? What additional security measures have you found effective?

Secure Your Infrastructure Today!

Sign up now to gain comprehensive insights into your SSH access logs. Start monitoring, alerting, and analyzing your entire infrastructure effortlessly.
Get started for free

Book a demo

Fill in the form below to book a demo without obligation.
Request a demo