A Simple and (Rather) Secure Mirroring Using rsync Over ssh

When it comes to remote backups, there are certainly many ways to skin a cat. In this particular case, I will show a little known, yet rather simple, approach that is able to exactly mirror all files together with permissions and ownership information; and isvery secure (as in it is hard for a third party to hack into your stream and capture or alter data, or gain access to your systems).

Characteristics of this approach

  • All readable files in the system can be copied (/dev, /proc and similar can't, but since we are not doing block-level backup, the OS cannot be restored directly from this backup anyway but has to be reinstalled. This only stores data and configuration files.).
  • File ownership and permissions are preserved (even extended permissions and ACLs if you wish so).
  • All traffic is encrypted with ssh.
  • No passwords are stored anywhere.
  • If an attacker takes control of one of the machines, he cannot easily enter the other one or do much more damage except messing up the backups. If he takes control of the source machine, he can only overwrite backups on the destination machine. If he takes control of the destination machine, he can only prevent the source from writing the backup.

The method

The gist of it is to use rsync-over-ssh (with the rsync -e 'ssh' option) to mirror the entire filesystem in "/". In order to be able to access everything possible, and keep permissions and ownership, the process needs to have root permissions on both source and destination machines. And this is where it gets tricky. How can we run as root on both sides without jeopardizing security of both systems? Thankfully, sshd supports authorization via private/public keys and a configuration parameter command= which will make sshd always run the same command when connected using this key, no matter what the remote host requests. Using that, we can prevent anything else except this exact backup command to be ran.

Step-by-step procedure

We have two machines: source and destination. The former is the one that needs to be backed up, and the latter will be holding the backup. You will need to have root access on both of those machines. On the source machine it is required if you want to backup the entire system, and on destination if you want to preserve ownership and permissions. If one or both of these features are not needed, you can use a normal user on that machine. But for the purpose of making the backup complete, the steps below assume root user is used on both sides. On the destination machine, there will be a /mnt/source_backup/ folder where the backup will be stored.

First, do a test run of the rsync-over-ssh command (with same options we will want to run in the end!) to capture the remote command executed through ssh. Make sure to replace the someuser below with a valid user on the remote machine that you can log in with. It is easiest to use root for that, but for systems where root user doesn't have a valid password (e.g. Ubuntu), you cannot use root, but any user will do for this.

rsync -e 'ssh -v -l someuser' -av / destination:/mnt/source_backup/

You will have to input the password and wait until it starts copying (or at least attempting to do so). Then stop it with Ctrl+C. Now examine the output looking for a line like this:

debug1: Sending command: rsync --server -vlogDtpre.iLsf . /mnt/source_backup/

Ignore the part I grayed above and copy the rest of the line somewhere. We will need it later.

Then, generate a key pair for root user access on the source machine. When asked for a passphrase, just hit enter, as we don't want to use a passphrase when running from batch scripts.

root@source:/root/# cd .ssh
root@source:/root/.ssh# ssh-keygen -t dsa -f backup_source_to_destination

Now copy the public key file backup_source_to_destination.pub to the destination machine in whichever way is appropriate. Put it in the /root/.ssh/ directory. Then add it to authorized_keys there using these commands. Note that the orange command text is the text we copied away a few steps back.

root@destination:/root/.ssh# echo -n 'command="rsync --server -vlogDtpre.iLsf . /mnt/source_backup/" ' >>authorized_keys
root@destination:/root/.ssh# cat backup_source_to_destination.pub >>authorized_keys

You will want to also prepare a list of directories to exclude from backup. On the source machine, edit the file excludelist with these contents:

/proc/*
/tmp/*
/lost+found/*
/sys/*
/dev/*

And make a shell script that will run your backup:

#!/bin/sh
rsync -e 'ssh -i /root/.ssh/backup_source_to_destination -l root' -a --exclude-from=/root/excludelist --log-file=/var/log/backup.log / destination:/mnt/source_backup/

Running this as a root user on the  source machine will back it up to the  destination machine, without producing any output on stdout unless there are errors (what makes it perfect for use with cron), and will log all copying into the /var/log/backup.log file.

Additional considerations

  • Make sure the ownership and permissions on key files and authorized_keys are set up correctly so that no one but root can read or write those files.
  • If you need to keep multiple backups for safety, you can rotate several disks on target mounts.

 References