SSH and Remote Work

Secure Shell (SSH), the host key check on first connect, and how an administrator hardens the server (sshd) are covered in Remote Access and Management. This page is about the client side: the tools and habits you use every day when your work happens on machines you never physically touch.

Key pairs

An SSH key pair is two matching files. The private key stays on your computer; the public key goes into ~/.ssh/authorized_keys on every server you want to log in to. At login the server asks your client to sign some data with the private key and checks the signature with the public key. The private key never crosses the network, and there is no password for an attacker to guess.

Creating a key

$ ssh-keygen -t ed25519 -C "ana@laptop"

Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/ana/.ssh/id_ed25519):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/ana/.ssh/id_ed25519
Your public key has been saved in /home/ana/.ssh/id_ed25519.pub
The key fingerprint is:
SHA256:mL04ZstZlBtgBgzS625PxQe2MKQRTe063ceyJYBysZg ana@laptop

• -t ed25519 chooses the key type. Ed25519 keys are short, fast and secure, and every current OpenSSH supports them. Use -t rsa -b 4096 only for old systems that do not.
• -C sets a comment, usually who and where. It ends up at the end of the public key line, so an administrator reading authorized_keys can tell whose key is whose.
• Press Enter to accept the default file name. Then type a passphrase: it encrypts the private key on disk, so a stolen laptop or backup does not hand out your logins. The passphrase is never sent to the server.

The public key is one line of text, safe to paste into a ticket or an email:

$ cat ~/.ssh/id_ed25519.pub

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICz6ynQ2ZQz3aC9hJ3/GpkduAInneBkH893y50mpFgBs ana@laptop

Installing the key on a server

ssh-copy-id logs in once with your password and appends the public key to ~/.ssh/authorized_keys on the server, creating the directory with the right permissions if needed:

$ ssh-copy-id ana@srv01.example.com

/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/ana/.ssh/id_ed25519.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
ana@srv01.example.com's password:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'ana@srv01.example.com'"
and check to make sure that only the key(s) you wanted were added.

Log in again: you are now asked for the key's passphrase (or nothing, if an agent holds it) instead of the account password. Once key logins work for everyone who needs them, the server can refuse passwords altogether; the steps are in Key-based authentication.

File permissions in ~/.ssh

SSH checks permissions and refuses files that other users could read or change. A private key that others can read is no longer private; an authorized_keys file that others can write lets them add their own key.

ssh-keygen and ssh-copy-id set these for you. Problems start when keys are copied by hand, restored from a backup, or unpacked from a zip file. Then you see:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for '/home/ana/.ssh/id_ed25519' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.

The fix is to tighten the permissions, not to loosen the check:

$ chmod 700 ~/.ssh
$ chmod 600 ~/.ssh/id_ed25519 ~/.ssh/config

ssh-agent: type the passphrase once

A passphrase on every connection quickly tempts people to remove it. The SSH agent (ssh-agent) solves that: it is a small background program that keeps your unlocked private keys in memory and signs login challenges for the ssh, scp and rsync commands you run. You type the passphrase once, with ssh-add; the key stays encrypted on disk.

$ eval "$(ssh-agent -s)"
$ ssh-add
$ ssh-add -l

Agent pid 4127
Enter passphrase for /home/ana/.ssh/id_ed25519:
Identity added: /home/ana/.ssh/id_ed25519 (ana@laptop)
256 SHA256:mL04ZstZlBtgBgzS625PxQe2MKQRTe063ceyJYBysZg ana@laptop (ED25519)

• ssh-agent -s starts the agent and prints shell commands that set SSH_AUTH_SOCK, the path of the socket where clients find it; eval runs those commands in your current shell. On a Linux or macOS desktop an agent usually runs already, started at login.
• ssh-add loads the default keys (add a path for another key). ssh-add -t 8h makes the agent forget the key after eight hours; ssh-add -D removes all keys at once, for example before you walk away.
• ssh-add -l lists the fingerprints of the loaded keys, which is the quickest check when a login unexpectedly asks for a password.

The client config file

Typing ssh -p 2222 -i ~/.ssh/id_work ana@web01.example.com every time is slow and error-prone. The per-user file ~/.ssh/config stores those settings under a short alias. ssh, scp, sftp, rsync and ssh-copy-id all read it.

Host web01
    HostName web01.example.com
    User ana
    Port 2222
    IdentityFile ~/.ssh/id_work

Host bastion
    HostName bastion.example.com
    User ana

Host db01
    HostName 10.20.0.15
    User ana
    ProxyJump bastion

Host *
    ServerAliveInterval 60
    AddKeysToAgent yes

Now ssh web01 is enough, and scp report.pdf web01: uses the same port, user and key. For every setting, SSH uses the first value it finds, reading the file from the top. That is why specific hosts come first and the Host * defaults come last. Running one command on many servers from a script is on Remote Execution.

Jump servers with ProxyJump

Many networks allow SSH to internal servers only from a jump server (bastion host). With the db01 block above, ssh db01 first connects to bastion, then opens a tunnel through it to 10.20.0.15. You authenticate to both machines with the keys on your computer. Without a config file, the same is ssh -J ana@bastion.example.com ana@10.20.0.15.

known_hosts and the changed-key warning

The first time you connect to a server, SSH shows its host key fingerprint and asks whether to trust it. When you answer yes, the key is saved in ~/.ssh/known_hosts, your personal list of servers and their host keys. On every later connection SSH compares the key the server presents with the saved one. If they differ, it stops with a warning that is hard to miss:

$ ssh srv01.example.com

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ED25519 key sent by the remote host is
SHA256:q7Vd2R0cLk9sXe4mB1nTz8yPwH3aJ6uF5gK0iN2oQrE.
Please contact your system administrator.
Add correct host key in /home/ana/.ssh/known_hosts to get rid of this message.
Offending ED25519 key in /home/ana/.ssh/known_hosts:3
  remove with:
  ssh-keygen -f "/home/ana/.ssh/known_hosts" -R "srv01.example.com"
Host key for srv01.example.com has changed and you have requested strict checking.
Host key verification failed.

It means exactly one thing: the machine answering at that name holds a different host key than last time. There are innocent reasons: the server was reinstalled or rebuilt from an image, or the name or IP address now belongs to a different machine (common with cloud instances and the Dynamic Host Configuration Protocol, DHCP). There is also the reason the warning exists: someone between you and the server is intercepting the connection, a man-in-the-middle attack, and hopes you will type your password into their machine.

The safe way to handle it:

1. Do not connect and do not delete the entry yet.
2. Find out whether a change was expected: a ticket, a change notice, a colleague who rebuilt the server.
3. Get the real fingerprint through another channel, for example from the cloud provider's console or from someone logged in at the server's own console, who runs ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub.
4. Only when it matches the fingerprint in the warning, remove the old entry and connect again; SSH then asks to save the new key.

$ ssh-keygen -R srv01.example.com

# Host srv01.example.com found: line 3
/home/ana/.ssh/known_hosts updated.
Original contents retained as /home/ana/.ssh/known_hosts.old

Debian and Ubuntu save the host names in known_hosts in hashed form (HashKnownHosts yes), so you cannot simply search the file for a name. ssh-keygen -F srv01.example.com finds the entry, and ssh-keygen -R removes it, whether hashed or not.

Copying files: scp, sftp and rsync

All three run over SSH, so they use the same keys, agent and config file, and the data is encrypted on the way.

scp: quick one-off copies

scp (secure copy) works like cp, with host:path for the remote side. A path after the colon without a leading slash is relative to your home directory on that host.

$ scp report.pdf ana@srv01.example.com:/tmp/
$ scp web01:/var/log/nginx/error.log .
$ scp -r -P 2222 site/ ana@web01.example.com:backup/

• The first copies a local file to /tmp/ on the server.
• The second fetches a file from the host alias web01 (from ~/.ssh/config) into the current directory (.).
• -r copies a directory recursively. -P sets the port, with a capital P, unlike ssh -p.

Since OpenSSH 9.0, scp uses the SSH File Transfer Protocol (SFTP) under the hood; the command line stayed the same.

sftp: an interactive file session

sftp opens a prompt where you browse both sides and move files, handy when you do not know the exact remote path yet. Graphical clients such as FileZilla and WinSCP speak the same protocol.

$ sftp web01
sftp> cd /var/www/site
sftp> ls
sftp> get index.html
sftp> put new-logo.png
sftp> bye

get downloads, put uploads, lcd and lls change and list the local directory, and bye ends the session.

rsync: synchronize efficiently

rsync copies only what differs between source and destination: new files, changed files, and within large changed files only the changed blocks. A second run after a small change takes seconds. It must be installed on both machines.

$ rsync -avz --dry-run site/ web01:/var/www/site/

sending incremental file list
./
index.html
css/
css/style.css

sent 172 bytes  received 28 bytes  400.00 bytes/sec
total size is 6,114  speedup is 30.57 (DRY RUN)

The trailing slash

For rsync, a slash at the end of the source changes the result:

• rsync -a site backup-a copies the directory itself: you get backup-a/site/….
• rsync -a site/ backup-b copies what is inside it: you get backup-b/index.html directly.

A slash at the end of the destination makes no difference. cp ignores a trailing slash on the source, but you can reproduce both layouts with it: cp -r site copies the directory, cp -r site/. copies its contents. This example shows the two results that rsync would produce:

mkdir -p site/css backup-a backup-b
printf '<h1>Hello</h1>\n' > site/index.html
printf 'body { margin: 0; }\n' > site/css/style.css

cp -r site backup-a      # the directory itself   (like: site  backup-a)
cp -r site/. backup-b    # only what is inside it (like: site/ backup-b)

find backup-a backup-b | sort

backup-a
backup-a/site
backup-a/site/css
backup-a/site/css/style.css
backup-a/site/index.html
backup-b
backup-b/css
backup-b/css/style.css
backup-b/index.html

Port forwarding

SSH port forwarding (tunneling) carries other Transmission Control Protocol (TCP) connections inside an encrypted SSH session. The classic use case: a database server db01 accepts PostgreSQL connections on port 5432 only from itself (localhost), which is good security. You still want to query it from your laptop.

$ ssh -N -L 5433:localhost:5432 db01

-L 5433:localhost:5432 means: listen on port 5433 of my computer, and send everything that arrives there through the tunnel to localhost:5432 as seen from db01. -N opens the tunnel without starting a remote shell. While it runs, a database tool on your laptop connects to localhost:5433 and reaches the database, and port 5432 never has to be opened in a firewall. Press Ctrl+C to close the tunnel.

By default forwarded ports listen on the loopback address only, so other machines on the network cannot use your tunnel — unless you give a bind address (-L 0.0.0.0:5433:…) or the server sets GatewayPorts yes for -R. Then anyone who can reach that port uses your tunnel without logging in. Tunnels can carry traffic around firewalls; many organizations only allow them for approved purposes, and administrators can switch forwarding off in the server configuration.

Keeping work alive: tmux and screen

When an SSH connection drops because of a Wi-Fi hiccup, a closed laptop lid or a virtual private network (VPN) that reconnects, the shell on the server receives a hangup signal and normally ends, together with the command you were running. A two-hour database restore can die halfway. A terminal multiplexer prevents that: it runs your shells inside a session on the server that keeps running when you disconnect, and you reattach later, from the same or another computer.

tmux

$ tmux new -s restore
  … start the long job, then press Ctrl+b, then d …
[detached (from session restore)]
$ tmux ls
restore: 1 windows (created Thu Oct  1 09:12:44 2026)
$ tmux attach -t restore

All tmux keys start with a prefix, Ctrl+b by default: press and release it, then press the command key.

screen

GNU screen is the older multiplexer and works the same way with a different prefix, Ctrl+a: screen -S restore starts a session, Ctrl+a d detaches, screen -ls lists sessions and screen -r restore reattaches. You will find it on older systems. Since Red Hat Enterprise Linux (RHEL) 8 screen is only available from Extra Packages for Enterprise Linux (EPEL), and Red Hat recommends tmux.