Hardening a Linux server

What hardening is

A fresh server is set up to be easy to use, not hard to attack. Hardening is the work of taking away everything an attacker could use that the server doesn't need: ways to log in, accounts, privileges, open ports, running services, missing patches. The fewer of these there are, the smaller the attack surface.

Most hardening guides, including the CIS Benchmarks (the Center for Internet Security's detailed checklists for each operating system), come down to the same few areas, and this lesson takes them in order:

1. Logins: who can log in, and how.
2. Accounts and privileges: who exists, and who can become root.
3. Services and ports: what's running and reachable.
4. The firewall: what's allowed in.
5. Updates: known holes closed, automatically.

An inherited server

The server for this lesson is built by Multipass from a cloud-init file: the same tool that sets up a new server on AWS, DigitalOcean and most other clouds the first time it boots. This one sets it up the way plenty of real servers end up, a few years and a few people in:

ssh_pwauth: true              # passwords allowed over SSH
chpasswd:
  users:
    - name: root
      password: lesson3-root  # root has a password
users:
  - default
  - name: deploy              # "the old deploy account"
    groups: [sudo]
    sudo: "ALL=(ALL) NOPASSWD:ALL"
    plain_text_passwd: deploy123
write_files:
  - path: /etc/ssh/sshd_config.d/40-legacy.conf
    content: |
      PermitRootLogin yes     # "to make deploys easier"
  - path: /etc/systemd/system/files-share.service   # python3 -m http.server 8000 --directory /var/log

(An extract; it also turns automatic updates off.)

multipass launch 24.04 --name sec-server --cpus 1 --memory 1G --disk 6G --cloud-init inherited-server.yaml
multipass info sec-server | grep -E 'Release|IPv4'

IPv4:           192.168.252.3
Release:        Ubuntu 24.04.5 LTS

Its address is on a small private network only your Mac can reach. To the server, your Mac is 192.168.252.1.

From the outside, before

Start where an attacker starts: what answers, and how does it let you in?

for p in 22 80 443 8000; do printf 'port %-5s ' $p; nc -z -G 3 192.168.252.3 $p && echo open || echo closed; done

• nc -z tries to connect to a port and reports whether it could, without sending anything. -G 3 gives up after 3 seconds.

  port 22    open
  port 80    closed
  port 443   closed
  port 8000  open

Two ports: SSH on 22, and something on 8000. Next, how SSH would let you in. This never sends a password: asked to log in with no method at all, SSH replies with the list of methods it would accept.

ssh -o BatchMode=yes -o PreferredAuthentications=none deploy@192.168.252.3 true

• PreferredAuthentications=none offers nothing, so the server's refusal lists what it would take.
• BatchMode=yes never stops to ask you for anything.

  deploy  deploy@192.168.252.3: Permission denied (publickey,password).
  root    root@192.168.252.3: Permission denied (publickey,password).

password for both, and root among them. Anyone who can reach port 22 can try passwords for root, as fast as the network allows. And port 8000:

curl -s http://192.168.252.3:8000/ | grep -o 'href="[^"]*"' | head -8
curl -s http://192.168.252.3:8000/auth.log | tail -3

href="alternatives.log" href="apport.log" href="apt/" href="auth.log" href="btmp" href="cloud-init-output.log" href="cloud-init.log" href="dist-upgrade/"

2026-11-07T14:06:23.939162+00:00 sec-server sshd[1843]: Connection closed by 192.168.252.1 port 51192
2026-11-07T14:06:26.197456+00:00 sec-server sshd[1845]: Connection closed by authenticating user deploy 192.168.252.1 port 51196 [preauth]
2026-11-07T14:06:26.397390+00:00 sec-server sshd[1847]: Connection closed by authenticating user root 192.168.252.1 port 51197 [preauth]

The server's own logs, to anyone, with no login. auth.log records every login attempt, and here it's recorded ours from a second earlier: the user names tried, from where, and when. An attacker reading it learns which accounts exist and whether anyone is watching. The service runs as root, so it can read every log. Nobody remembers setting it up, which is how these usually go.

From the inside, before

Now with the console, multipass exec, which works even if SSH is broken: the same as a cloud provider's web console. The checks:

sudo sshd -T | grep -E '^(permitrootlogin|passwordauthentication|kbdinteractiveauthentication|pubkeyauthentication|maxauthtries|x11forwarding|allowusers) '
sudo ss -tlnp
awk -F: '$7 !~ /(nologin|false)$/ {print $1}' /etc/passwd
sudo passwd -S root; sudo passwd -S deploy
getent group sudo; sudo grep -rh NOPASSWD /etc/sudoers /etc/sudoers.d/
sudo ufw status
systemctl is-enabled unattended-upgrades
apt list --upgradable 2>/dev/null | grep -c -- -security

• sshd -T prints the settings SSH is actually using, after reading every file, not what any one file says.
• ss -tlnp lists every listening TCP port, and the program behind it.
• The awk lists accounts whose shell lets them log in (anything but nologin or false).
• passwd -S gives each account's password status: P has a usable password, L is locked.
• NOPASSWD in a sudoers file means that account becomes root without typing anything.

maxauthtries 6
permitrootlogin yes
pubkeyauthentication yes
passwordauthentication yes
kbdinteractiveauthentication no
x11forwarding yes

0.0.0.0:22 sshd
0.0.0.0:8000 python3
127.0.0.53%lo:53 systemd-resolve
127.0.0.54:53 systemd-resolve
[::]:22 sshd

root sync deploy ubuntu

root P
ubuntu L
deploy P

sudo:x:27:deploy,ubuntu
deploy ALL=(ALL) NOPASSWD:ALL
ubuntu ALL=(ALL) NOPASSWD:ALL

Status: inactive

disabled

19

The full picture: root logins and passwords on; port 8000 open to every address (0.0.0.0); root and deploy with usable passwords; deploy in sudo with no password needed; no firewall; automatic updates off; 19 security updates waiting. (systemd-resolve on 127.0.0.53 is the server's own name lookup, listening only to itself; that's normal.)

1. An account of your own, with a key

Before taking anything away, make sure you can still get in. That's an account of your own, ops, which logs in with an SSH key instead of a password.

A key is a pair of files. The private key stays on your computer; the public key goes on the server. When you log in, the server sends a challenge only the private key can answer, and the private key itself never leaves your computer. There's nothing to guess and nothing to reuse from another site's leak, which is why keys beat passwords.

ssh-keygen -t ed25519 -f ~/.ssh/sec-lesson-ed25519 -C "sec-lesson"
ssh-keygen -l -f ~/.ssh/sec-lesson-ed25519.pub

• -t ed25519 makes a modern, short, fast key type. Use it unless something you connect to can't.
• -l shows the key's fingerprint, a short summary to compare when checking you have the right one.

256 SHA256:VRW6FYG1A0ajokicYc86LqUMdoitvmnisNkzF8cZ7js sec-lesson (mayowababatola@KwamiB) (ED25519)

The lesson's key has no passphrase, so the scripts can use it. A real key should have one: then a copied key file is useless without it, and ssh-agent (or the macOS keychain) remembers it so you type it once a session.

On the server, the account and the public key:

sudo useradd -m -s /bin/bash -G sudo ops
sudo install -d -m 700 -o ops -g ops /home/ops/.ssh
sudo install -m 600 -o ops -g ops /tmp/ops.pub /home/ops/.ssh/authorized_keys
sudo passwd ops

• useradd -m -G sudo creates the account with a home folder, in the sudo group.
• install -m 700 and -m 600: SSH refuses keys in folders and files other people can write to, so the folder is readable only by ops, and authorized_keys too.
• passwd ops sets a password, used only for sudo, never for logging in. (The lesson's script made a random one and kept it on the Mac, readable only by you.)

(The public key got onto the server with multipass transfer, Multipass's file copy. Piping it through multipass exec hung on a Mac, waiting for an end of input that never came; copying the file is the reliable way.)

uid=1002(ops) gid=1002(ops) groups=1002(ops),27(sudo)
-rw------- 1 ops ops 116 Nov  7 09:06 /home/ops/.ssh/authorized_keys

$ ssh -i ~/.ssh/sec-lesson-ed25519 -o IdentitiesOnly=yes ops@192.168.252.3 'echo logged in as $(whoami) on $(hostname), with a key'
logged in as ops on sec-server, with a key

• IdentitiesOnly=yes offers only the named key, not every key you have.

Logged in with the key, before anything else changes. That order matters: never take away a way in until another one is proven.

2. SSH: keys only, no root, named users only

SSH's settings live in /etc/ssh/sshd_config, and in files in /etc/ssh/sshd_config.d/, which it reads in name order:

$ ls /etc/ssh/sshd_config.d/
40-legacy.conf
50-cloud-init.conf
60-cloudimg-settings.conf

Here's the trap: for most settings, SSH uses the first value it finds, not the last. 40-legacy.conf says PermitRootLogin yes; 50-cloud-init.conf, from the image, says PasswordAuthentication yes. A new file named 90-hardening.conf would be read last, and ignored. So the new file is named to sort first:

# /etc/ssh/sshd_config.d/10-hardening.conf
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
# ubuntu stays because Multipass uses it for multipass exec; ops is yours.
AllowUsers ops ubuntu
MaxAuthTries 3
X11Forwarding no

• PermitRootLogin no: nobody logs in as root; you log in as yourself and use sudo, so there's a record of who.
• PasswordAuthentication no and KbdInteractiveAuthentication no: the two ways SSH can ask for a password, both off.
• AllowUsers: only these accounts may log in at all, whatever else exists.
• MaxAuthTries 3: three tries per connection, then it's dropped.
• X11Forwarding no: no forwarding of graphical programs, which a server doesn't need.

Then test before restarting:

$ sudo sshd -t && echo "sshd -t: the new settings are valid"
sshd -t: the new settings are valid

$ sudo systemctl restart ssh && systemctl is-active ssh
active

$ sudo rm -f /etc/ssh/sshd_config.d/40-legacy.conf && sudo sshd -t && sudo systemctl restart ssh && echo removed 40-legacy.conf
removed 40-legacy.conf

• sshd -t checks the settings without applying them. A typo in an SSH setting can stop SSH starting at all, and with it your only way in. Always test, and keep a working session open while you restart, so you can undo a mistake.

The old 40-legacy.conf went too: the new file already wins, but a setting that says the opposite of what's in force is a trap for the next person.

3. Accounts: lock what isn't used

$ sudo passwd -l root; sudo passwd -l deploy; sudo usermod -s /usr/sbin/nologin deploy; sudo gpasswd -d deploy sudo
passwd: password changed.
passwd: password changed.
Removing user deploy from group sudo

• passwd -l locks a password: it can't be used to log in, at the console or anywhere else. Root needs no password on a server where admins use sudo.
• usermod -s /usr/sbin/nologin gives deploy a shell that refuses logins.
• gpasswd -d deploy sudo takes it out of the sudo group.

deploy is locked, not deleted. Deleting an account frees its user number for reuse, and the files it owned would then belong to the next account created; locking keeps the history clear. Delete it later, once you're sure nothing uses it.

Its password-free sudo was written separately, by cloud-init:

$ sudo grep -H NOPASSWD /etc/sudoers.d/*
/etc/sudoers.d/90-cloud-init-users:deploy ALL=(ALL) NOPASSWD:ALL
/etc/sudoers.d/90-cloud-init-users:ubuntu ALL=(ALL) NOPASSWD:ALL

$ sudo sed -i '/^deploy[[:space:]]/d' /etc/sudoers.d/90-cloud-init-users && sudo visudo -c -q && echo 'deploy removed from sudoers; visudo -c: still valid'
deploy removed from sudoers; visudo -c: still valid

• visudo -c checks every sudoers file. A broken sudoers file stops sudo working for everyone, so check after every change, as with sshd -t.

The ubuntu line stays, because Multipass uses that account for multipass exec. That's a real trade-off, written down: on a cloud server, the image's default account is often set up exactly like this, and it's worth removing once your own account works.

4. Remove what nobody needs

$ sudo systemctl disable --now files-share && sudo rm -f /etc/systemd/system/files-share.service && sudo systemctl daemon-reload
Removed "/etc/systemd/system/multi-user.target.wants/files-share.service".
files-share stopped, disabled and removed

• disable --now stops it and stops it starting at the next boot.
• Removing the unit file and daemon-reload means it can't be started again by accident.

A service that isn't running can't be attacked. That's the cheapest hardening there is.

5. A firewall: only SSH in

Ubuntu's firewall front end is ufw, the Uncomplicated Firewall:

$ sudo ufw default deny incoming && sudo ufw default allow outgoing && sudo ufw limit OpenSSH && sudo ufw --force enable
Default incoming policy changed to 'deny'
Default outgoing policy changed to 'allow'
Rules updated
Rules updated (v6)
Firewall is active and enabled on system startup

$ sudo ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)

To                         Action      From
--                         ------      ----
22/tcp (OpenSSH)           LIMIT IN    Anywhere
22/tcp (OpenSSH (v6))      LIMIT IN    Anywhere (v6)

• default deny incoming: nothing gets in unless a rule says so. Port 8000's service is gone, but if anything starts listening again, the firewall keeps it unreachable.
• limit OpenSSH: SSH is allowed, but one address may open only 6 new connections in 30 seconds. Past that, it's refused. That slows down anyone guessing.
• The order matters: allow SSH before enabling a firewall that denies everything, or the next SSH connection is the last.

The firewall that locked out its own administrator

Then the "after" checks ran, and every one failed:

exec failed: ssh connection failed: 'Connection refused'

Section 5 of the script had just opened five SSH connections from the Mac in a few seconds: a port check, three login probes and the ops login. multipass exec reaches the VM over SSH too, from the same address, so it was the sixth. The rate limit doesn't know a guesser from an administrator; it counts connections. A second script, which waited out the window and then made its own five connections, hit the same wall. The firewall's own log:

$ sudo grep -h 'UFW LIMIT BLOCK' /var/log/ufw.log | tail -4
UFW LIMIT BLOCK from 192.168.252.1 to port 22
UFW LIMIT BLOCK from 192.168.252.1 to port 22
UFW LIMIT BLOCK from 192.168.252.1 to port 22
UFW LIMIT BLOCK from 192.168.252.1 to port 22

The fix real teams use: an allow rule for the management address, above the limit. ufw checks rules top to bottom and stops at the first match, so a connection from the Mac matches the allow and never reaches the limit. Everyone else still does.

$ sudo ufw insert 1 allow proto tcp from 192.168.252.1 to any port 22 comment 'management: this Mac'
Rule inserted

$ sudo ufw status numbered
     To                         Action      From
     --                         ------      ----
[ 1] 22/tcp                     ALLOW IN    192.168.252.1              # management: this Mac
[ 2] OpenSSH                    LIMIT IN    Anywhere
[ 3] OpenSSH (v6)               LIMIT IN    Anywhere (v6)

blocked so far: 10

• insert 1 puts the rule at the top, rather than the end, where it would never be reached.
• comment says why it's there, for the next person.

Then all eight checks, eight connections back to back, and the count afterwards:

blocked so far: 10

Unchanged: none refused. The Mac's connections matched rule 1. In a real setup, the management address would be a bastion (one hardened server admins connect through) or a VPN's address, never a laptop on home Wi-Fi.

6. Updates that install themselves

Most attacks use holes that already have a fix. On Ubuntu, unattended-upgrades installs security updates every day by itself:

$ sudo systemctl enable --now unattended-upgrades && cat /etc/apt/apt.conf.d/20auto-upgrades
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";

• Update-Package-Lists "1": refresh the list of available updates every day.
• Unattended-Upgrade "1": install the security ones every day.

And the 19 waiting, now:

$ sudo unattended-upgrade && echo unattended-upgrade finished
/usr/bin/unattended-upgrade:567: DeprecationWarning: This process (pid=4087) is multi-threaded, use of fork() may lead to deadlocks in the child.
  pid = os.fork()
...
unattended-upgrade finished

It printed that warning 13 times, which looks alarming and isn't. Ubuntu 24.04's Python warns whenever a program running several threads starts a child process; unattended-upgrade does that for each batch of packages. The updates installed normally. Still, warnings like this are worth looking up rather than ignoring: one day one of them won't be cosmetic.

Some updates, the kernel's for instance, only take effect after a restart. Ubuntu writes /var/run/reboot-required when one is needed; check for it after updates, and plan the restart.

After

From the outside:

Which ports answer:
  port 22    open
  port 80    closed
  port 443   closed
  port 8000  closed

How the server lets you log in now:
  deploy  deploy@192.168.252.3: Permission denied (publickey).
  root    root@192.168.252.3: Permission denied (publickey).
  ops     ops@192.168.252.3: Permission denied (publickey).

$ ssh -i ~/.ssh/sec-lesson-ed25519 -o IdentitiesOnly=yes ops@192.168.252.3 'whoami; sudo -n true 2>&1 | head -1'
ops
sudo: a password is required

publickey only, for everyone. Without the key, even ops gets nothing to try. With it, ops is in, but sudo wants a password (-n means "fail rather than ask"): a stolen key alone isn't root.

And from the inside, side by side with before:

Two lines still need explaining. root and sync keep login shells, but root's password is locked and SSH refuses it, and sync is a system account with no password. And ubuntu keeps password-free sudo for Multipass, as decided in step 3.

Spot the problem

Work out what's wrong before you read the explanation.

The protection that took the server "down". A team adds ufw limit OpenSSH to every server, as this lesson did. The next deploy, run by an automation tool that opens a new SSH connection for each step, fails halfway on twenty servers at once, with Connection refused. The on-call engineer sees SSH refusing connections and assumes the servers are down or under attack, and starts restarting them. What happened? The rate limit counted the deploy tool as an attacker. limit allows six new connections from one address in 30 seconds, and doesn't know a guesser from an administrator, exactly as it refused multipass exec in this lesson, ten times. From outside, a firewall refusal looks the same as SSH being down. Three fixes, any of which would have worked: exempt the management address with an allow rule above the limit, as step 5 did; reuse one connection for many commands (SSH's ControlMaster setting, which Ansible uses for exactly this); or drop the limit, because with keys only there's nothing to guess. And one habit: when SSH suddenly refuses, check the firewall's log (UFW LIMIT BLOCK) before assuming the worst.

Where you are now

You can take a server as you find it, see its weak spots from outside and in, and harden it one checked step at a time: your own account with a key, SSH that only takes keys, retired accounts, fewer services, a firewall that doesn't lock you out, and updates that install themselves. But hardening is a moment; servers drift. The next lesson keeps the same server and asks how you'd know if it changed: scoring it against a benchmark, recording who changed what, and finding a change someone made without telling you.

Check yourself

Key terms

• Hardening: removing what a server doesn't need, so there's less for an attacker to use.
• Attack surface: every way in to a system: logins, ports, services, accounts.
• cloud-init: the tool that sets up a new cloud server the first time it boots.
• SSH key pair: a private key you keep and a public key the server holds; logging in proves you have the private one.
• sshd -T and sshd -t: the SSH settings in force, and a check of new settings before applying them.
• sudoers: the files saying who may use sudo, and whether they need a password.
• ufw: Ubuntu's Uncomplicated Firewall.
• Rate limit: a cap on how many new connections one address may open in a time window.
• Bastion: one hardened server administrators connect through to reach the rest.
• unattended-upgrades: Ubuntu's daily automatic installer of security updates.

Go deeper

• The CIS Benchmarks (opens in a new tab), including Ubuntu Linux 24.04 LTS (free to download with registration).
• Ubuntu's documentation on OpenSSH (opens in a new tab), ufw (opens in a new tab) and automatic updates (opens in a new tab).
• Try: run ssh-audit (in Homebrew) against the server, before and after changing SSH's ciphers. What does it flag that this lesson didn't change?
• Try: add a passphrase to the lesson's key with ssh-keygen -p -f ~/.ssh/sec-lesson-ed25519, then add it to the agent with ssh-add --apple-use-keychain. Does logging in still need you to type anything?