5 steps to secure your SSH servers

Some say that the good ol’ Linux and Unix (e.g. FreeBSD, OpenBSD, etc.) servers we used to connect to are out of style and that the way to go is running serverless workloads on cloud providers. This is definitely not the case for quite a lot of scenarios though. There are many valid reasons why it makes sense to own or rent a bare metal dedicated server. From running predictable workloads (e.g. development and staging environments or B2B back-office dashboards) to maintaining a legacy system that needs to stay on-premise.

In all of these cases you should take care of a few security best pratices when connecting to your servers via SSH — the de facto protocol for remote access.

Restrict to private network only

SSH connections should not be accessible from anywhere in the world. SSH connections should be allowed only from an IP range that is controlled and belongs to a private network. This means that in order for anyone to connect to your server, they need to enter the private network first. This can be achieved either configuring the operating system of the server (e.g., with iptables or ufw) or by configuring the firewall rules on the network interface of the server on your hosting provider.

Set up a bastion server

A great and easy way to allow access through the internet when needed, while maintaining the requirement for private network access is to set up a “bastion” server. It will only work as an intermmediate SSH server, accessible via the internet itself, but with no data on it.

Forward SSH keys

Since the bastion server has no keys on it, you should set up SSH key forwarding so that it can forward the SSH keys from your local computer to the server you would like to eventually connect to. This can be achieved in two steps:

1. Configure the host you would like to connect to in ~/.ssh/config, to connect via the bastion server and forward SSH keys

   Host bastion-00
       HostName 192.168.0.11
       User logic
       ForwardAgent yes
   
   Host swarm-11
       HostName 192.168.0.42
       User logic
       ForwardAgent yes
       ProxyCommand ssh -W %h:%p bastion-00
   

2. Start the SSH agent and add your private key

   ssh-agent
   ssh-add -k
   

Now you can connect to the server swarm-11 via bastion-00, without storing any private keys on the bastion server by just running the following command in your terminal:

ssh swarm-11

Now, although we are not storing any private keys on any of those server, if your SSH public key (stored by default in ~/.ssh/id_rsa.pub) is not stored in your server by your hosting provider, you should copy it key to both of those servers, before you connect for the first time. This can be achieved by running the following command in your terminal:

ssh-copy-id bastion-00
ssh-copy-id swarm-1

This part will ask for password authentication.

Restrict to key-only authentication

Connecting to your SSH server should only take place via key authentication. Password authentication, if enabled and allowed by the hosting provider, should be disabled as soon as you set up a successful SSH connection via key authentication.

To disallow password authentication, edit the SSH daemon configuration (by default stored in /etc/ssh/sshd_config), look for the line with the PasswordAuthentication setting and set its value to no:

PasswordAuthentication no

Now, reload your SSH server. On environments using systemd (e.g. Ubuntu) you can do this by running:

systemctl restart ssh

Disallow SSH access to root user

SSH access should only be allowed to non-root users. Privilege escalation to the root user should only happen using su or sudo, by the user connected via SSH, requiring authentication via a strong password.

To disallow root login via SSH, update the PermitRootLogin SSH daemon setting to no:

PermitRootLogin No

A few more considetations

• 🙌 Make sure you can connect after changing SSH daemon settings As updates to the SSH daemon configuration can prevent you from connecting to your server in the future, after restarting the SSH service attempt to reconnect to your server from another terminal window, without disconnecting the previous one. If you connect successfully, you can disconnect from both windows without worries. If you have trouble reconnecting, we suggest you reset the SSH daemon’s settings to their previous state, restarting the SSH service again and then verify you can connect again with the old settings, before conducting further troubleshooting.
• 📤 Set up email notifications It’s important to be able to audit access to your servers. We suggest setting up email notifications, both when a user connects to a server via SSH and when a user escalates privileges to root.
• ⚙️ Consider Infrastructure as Code As all those settings can be quite a lot and overwhelming to manage, you can consider Infrastructure as Code solutions like Terraform or Ansible to automate the provisioning of your servers based on your desired settings, without requiring any manual steps.

Avoid the hassle

If this sounds important, yet a bit "too much" given that you are busy developing your business, we would love to help.

Contact us here to learn how we can help keep your infrastructure nice and neat.