OpenSSH/Cookbook/Automated Backup


Using OpenSSH with keys can facilitate secure automated backups. rsync(1)[1], tar(1), and dump(8) are the foundation for most backup methods. It's a myth that remote root access must be allowed. sudo(8) works just fine -- if properly configured. Remember, that until the backup data has been tested and shown to restore reliably, it does not count as a backup copy.

Backup with rsyncEdit

rsync(1) now defaults to using SSH. But it still can be specified explicitly:

$ rsync --exclude '*~' -avv \
        -e 'ssh' \ \

For some types of data, transfer can be speeded up greatly by using rsync(1) with compression, -z, if the CPUs on both ends can handle the extra work.

Rsync with keysEdit

rsync(1) uses SSH by default and can even authenticate using SSH keys by using the -e option to specify alternatives for the remote shell command. Thus it is possible through it to point to the specific SSH key file for the SSH client to use.

$ rsync --exclude '*~' -avv \
        -e 'ssh -i ~/.ssh/key_rsa' \ \

If the key is first added to an agent, then the passphrase only needs to be entered once. Other configuration options can also be sent to the SSH client in the same way if needed, or via the SSH client's configuration file.

Backup with rsync(1) and sudo(8)Edit

rsync(1) is often used to back up both locally and remotely. It is fast and flexible and copies incrementally so only the changes are transferred, thus avoiding wasting time re-copying what is already at the destination. It does that through use of its now famous algorithm. When working remotely, it needs a little help with the encryption and the usual practice is to tunnel it over SSH.

Say you're backing up from server to client. rsync(1) on the client uses ssh(1) to make the connection to rsync on the server. rsync(1) is invoked from client with -v passed to the SSH client to see exactly what parameters are being passed to the server. Those details will be needed in order to configure sudo(8) on the server.

$ rsync \
  -e 'ssh -v \
          -i ~/.ssh/key_bkup_rsa  \
          -t             \
          -l bkupacct'   \
  --rsync-path='sudo rsync' \ 
  --delete   \
  --archive  \
  --compress \
  --verbose  \
  bkupacct@server:/var/www/ \

The argument --rsync-path tells the server what to run in place of rsync(1). In this case it runs sudo rsync. The argument -e says which remote shell tool to use. In this case it is ssh(1). For the SSH client being called by the rsync(1) client, -i says specifically which key to use. That is independent of whether or not an authentication agent is used for ssh keys. Having more than one key is a possibility, since it is possible to have different keys for different tasks.

It will most likely be an iterative process. Keep making adjustments to /etc/sudoers on the server until it works as it should.

%autobackup ALL=(ALL) NOPASSWD: /usr/local/bin/rsync --server \
--sender -vlogDtpre.if . /var/www/

You can find the exact settings(s) to use in /etc/sudoers by running the SSH in verbose mode (-v) on the client. Be careful when working with patterns not to match more than is safe.

Preparation: create an account to use for the backup, create a pair of keys to use only for backup, then make sure you can log in to that account with ssh(1) with and without those keys.

$ ssh -i ~/.ssh/mybkupkey

The account on the server is named 'bkupacct' and the private RSA key is ~/.ssh/key_bkup_rsa on the client. On the server, the account 'bkupacct' is a member of the group 'autobackup'.

The public key, ~/.ssh/, has been copied to the account bkupacct on server and placed in ~/.ssh/authorized_keys there.

The following directories on server are owned by root and belong to the group bkupacct and not group readable, but not group writeable, and definitely not world readable: ~ and ~/.ssh. Same for the file ~/.ssh/authorized_keys there. (This assumes you are not also using ACLs) This is one way of many to set permissions on the server:

$ sudo chown root:bkupacct ~
$ sudo chown root:bkupacct ~/.ssh/
$ sudo chown root:bkupacct ~/.ssh/authorized_keys
$ sudo chmod u=rwx,g=rx,o= ~
$ sudo chmod u=rwx,g=rx,o= ~/.ssh/
$ sudo chmod u=rwx,g=r,o=  ~/.ssh/authorized_keys

Step 1: Configure sudoers(5) and test rsync(1) with sudo(8) on the remote host. In this case data is staying on the remote machine.

Step 2: Test rsync(1) with sudo(8) over ssh(1) to make sure it can run with elevated privileges on the remote system.

$ ssh -l bkupacct sudo rsync -av:/var/www/ /tmp/

It will be necessary to tune /etc/sudoers a little at this stage. More refinements may come later. Note that there is an rsync(1) user and an ssh(1) user. The data in this case gets copied from the remote machine to the local /tmp directory.

$ rsync -e 'ssh -t -l bkupacct' --rsync-path='sudo rsync' \
-av /tmp/

Step 3: Do the same transfer again but using the key for authentication to make sure that the key works.

$ rsync -e 'ssh -i ~/.ssh/key -t -l bkupacct' --rsync-path='sudo rsync' \
-av /tmp/

Step 4: Adjust /etc/sudoers so that the backup account has just enough access to run rsync(1) but only in the directories it is supposed to run in and without free-rein on the system. Use the first debugging level to see the actual parameters getting passed to the remote host.

$ rsync -e 'ssh -t -v' --rsync-path='sudo rsync' \
-av /tmp/
debug1: Sending command: sudo rsync --server --sender -e.iLs . /var/www

That provides the basis of what /etc/sudoers will need configured. Be sure that the backed up data is not accessible to others. At this point you are almost done, although the process can be automated much further.

Step 5: Test rsync(1) with sudo(8) over ssh(1) to verify that the settings made in /etc/sudoers are correct.

$ rsync -e 'ssh -t' --rsync-path='sudo rsync' \
-av /tmp/

Then if the settings are correct it is possible to set up a single-purpose key for just this task.

Backup Using tar(1)Edit

A frequent choice for creating archives is tar(1). But since it copies whole files and directories, rsync(1) is usually much more efficient for updates or incremental backups.

The following will make a tarball of the directory /var/www/ and send it via stdout into sdtin via a pipe into ssh(1) where, on the remote machine it is directed into the file called backup.tar. Here tar(1) runs on a local machine and stores the tarball remotely:

$ tar cf - /var/www/ | ssh -l fred "cat > backup.tar"

There are almost limitless options for that recipe:

$ tar zcf - /var/www/ /home/*/www/ \
	|  ssh -l fred "cat > $(date +'%Y-%m-%d').tar.gz"

That example does the same, but also gets user WWW directories, compress the tarball using gzip(1), and label the resulting file according to the current date. It can be done with keys, too:

$ tar zcf - /var/www/ /home/*/www/ \
	|  ssh -i key_rsa -l fred "cat > $(date +"%Y-%m-%d").tgz

And going the other direction is just as easy for tar(1) to find what is on a remote machine and store the tarball locally.

$ ssh "tar zcf - /var/www/" >  backup.tgz

Or here is a fancier example of running tar(1) on the remote machine but storing the tarball locally.

$ ssh -i key_rsa -l fred "tar jcf - /var/www/ /home/*/www/" \
	> $(date +"%Y-%m-%d").tar.bz2

So in summary, the secret to using tar(1) for backup is the use of stdout and stdin to effect the transfer through pipes and redirects.

Backup of Files With tar(1) But Without Making A TarballEdit

Sometimes it is necessary to just transfer the files and directories without making a tarball at the destination. In addition to writing to stdin on the source machine, tar(1) can read from stdin on the destination machine to transfer whole directory hierarchies at once.

$ tar zcf - /var/www/ | ssh -l fred "cd /some/path/; tar zxf -"

Or going the opposite direction, it would be the following.

$ ssh 'tar zcf - /var/www/' | (cd /some/path/; tar zxf - )

However, these still copy everything each time they are run. So rsync(1) described above in the previous section might be a better choice in many situations, since on subsequent runs it only copies the changes. Also, depending on the type of data network conditionsm, and CPUs available, compression might be a good idea either with tar(1) or ssh(1) itself.

Backup using dumpEdit

Using dump(8) remotely is like using tar(1). One can copy from the remote server to the local server.

$ ssh -t 'sudo dump -0an -f - /var/www/ | gzip -c9' > backup.dump.gz

Note that the password prompt for sudo(8) might not be visible and it must be typed blindly.

Or one can go the other direction, copying from the locate server to the remote:

$ sudo dump -0an -f - /var/www/ | gzip -c9 | ssh 'cat > backup.dump.gz'

Note again that the password prompt might get hidden in the initial output from dump(8). However, it's still there, even if not visbile.



  1. "How Rsync Works". Samba.