How to Set Up SSH Key Authentication on a Linux VPS

How SSH Key Authentication Works

When you connect with key authentication, the server checks whether the public key in its authorized_keys file matches the private key on your local machine, using a cryptographic challenge. No password is transmitted over the network. If your private key is protected with a passphrase, you enter the passphrase locally, and it never leaves your machine.

The practical benefit over passwords is twofold: key authentication is not vulnerable to brute-force attacks (there is no password to guess), and it allows automation without storing passwords in scripts or environment variables.

Prerequisites

• A Linux VPS running Ubuntu, AlmaLinux, or Debian with a root or sudo user
• SSH access with password authentication (you’ll switch to key auth by the end of this guide)
• A local machine running Linux, macOS, or Windows with OpenSSH client installed (comes pre-installed on macOS and Windows 10/11)

InMotion’s Cloud VPS plans include Ubuntu 22.04 LTS, AlmaLinux 9, and Debian 12 with root SSH access from provisioning. Managed VPS plans also support SSH access.

Step 1: Generate the SSH Key Pair on Your Local Machine

Run the following on your local machine, not the server.

ssh-keygen -t ed25519 -C "your_identifier"

ssh-keygen -t ed25519 -C "your_identifier"

The Ed25519 algorithm is the current recommended standard. When prompted for a file name, press Enter to use the default (~/.ssh/id_ed25519 on Linux/macOS, %USERPROFILE%\.ssh\id_ed25519 on Windows). Adding a passphrase is strongly recommended for any key used to access production infrastructure.

Two files are created: id_ed25519 (your private key, never share this) and id_ed25519.pub (your public key, this goes on the server).

If your server or organization requires RSA keys, generate a 4096-bit RSA key instead:

ssh-keygen -t rsa -b 4096 -C "your_identifier"

ssh-keygen -t rsa -b 4096 -C "your_identifier"

Step 2: Copy the Public Key to the Server

Option A: Using ssh-copy-id (Linux/macOS)

ssh-copy-id -i ~/.ssh/id_ed25519.pub username@your-server-ip

ssh-copy-id -i ~/.ssh/id_ed25519.pub username@your-server-ip

This copies the public key to ~/.ssh/authorized_keys on the server and sets the correct permissions automatically.

Option B: Manually (Windows or when ssh-copy-id is unavailable)

Display your public key:

cat ~/.ssh/id_ed25519.pub

cat ~/.ssh/id_ed25519.pub

Copy the entire output. Then on the server:

mkdir -p ~/.sshnano ~/.ssh/authorized_keys

mkdir -p ~/.sshnano ~/.ssh/authorized_keys

Paste the public key on a new line, save, and exit. Then set the correct permissions:

chmod 700 ~/.sshchmod 600 ~/.ssh/authorized_keys

chmod 700 ~/.sshchmod 600 ~/.ssh/authorized_keys

Permission settings matter. If authorized_keys is world-readable, SSH will refuse to use it.

Step 3: Test Key Authentication Before Disabling Passwords

Open a new terminal window and connect using the key:

ssh -i ~/.ssh/id_ed25519 username@your-server-ip

ssh -i ~/.ssh/id_ed25519 username@your-server-ip

If this succeeds without prompting for a password (or after entering your key passphrase if you set one), key authentication is working. Do not close your existing SSH session until you confirm this. Losing both authentication methods simultaneously locks you out of the server.

Step 4: Harden sshd_config

Once key authentication is confirmed working, open the SSH daemon configuration file:

sudo nano /etc/ssh/sshd_config

sudo nano /etc/ssh/sshd_config

Make the following changes:

PasswordAuthentication no disables password login. Only key-authenticated connections are permitted.

PermitRootLogin no (or prohibit-password) prevents direct root login. Use a standard user with sudo instead.

PubkeyAuthentication yes confirms public key authentication is enabled (usually already set to yes by default).

AuthorizedKeysFile .ssh/authorized_keys specifies the key file location (verify this is not commented out).

Optionally, changing the default SSH port from 22 to a non-standard port (such as 2222 or any port above 1024) reduces the volume of automated scanning traffic. This is security through obscurity, not a substitute for proper authentication, but it significantly reduces log noise.

Port 2222

Port 2222

Save and close the file.

Step 5: Restart the SSH Service

sudo systemctl restart sshd

sudo systemctl restart sshd

On some distributions, the service name is ssh rather than sshd:

sudo systemctl restart ssh

sudo systemctl restart ssh

Important: test connectivity from a new terminal session before closing your existing connection. On AlmaLinux and CentOS-based systems, SELinux may block non-standard SSH ports. If you changed the port, allow it through the firewall:

sudo firewall-cmd --permanent --add-port=2222/tcpsudo firewall-cmd --reload

sudo firewall-cmd --permanent --add-port=2222/tcpsudo firewall-cmd --reload

On Ubuntu using UFW:

sudo ufw allow 2222/tcpsudo ufw delete allow 22/tcp

sudo ufw allow 2222/tcpsudo ufw delete allow 22/tcp

Step 6: Add Your Key to SSH Agent for Convenience

If you set a passphrase on your key (which you should), entering it on every connection becomes tedious. The SSH agent caches the decrypted key in memory for the duration of your session.

eval "$(ssh-agent -s)"ssh-add ~/.ssh/id_ed25519

eval "$(ssh-agent -s)"ssh-add ~/.ssh/id_ed25519

On macOS, add -K to store the passphrase in your keychain:

ssh-add --apple-use-keychain ~/.ssh/id_ed25519

ssh-add --apple-use-keychain ~/.ssh/id_ed25519

On Windows, the OpenSSH Authentication Agent service can be configured to start automatically via Services or PowerShell.

Managing Multiple Keys and Servers

A ~/.ssh/config file simplifies connecting to multiple servers without specifying the key file and user on every command.

Host production-vps  HostName 192.0.2.100  User deploy  IdentityFile ~/.ssh/id_ed25519  Port 2222Host staging-vps  HostName 192.0.2.101  User deploy  IdentityFile ~/.ssh/id_ed25519_staging  Port 22

Host production-vps  HostName 192.0.2.100  User deploy  IdentityFile ~/.ssh/id_ed25519  Port 2222Host staging-vps  HostName 192.0.2.101  User deploy  IdentityFile ~/.ssh/id_ed25519_staging  Port 22

With this config in place, connecting to production is simply:

ssh production-vps

ssh production-vps

Related: How to Setup a VPS Server covers the full VPS provisioning workflow including initial SSH connection.