Automated backups on Linux

In this article I will talk you through how to set up automatic backups from your Linux computer, to another Linux/Unix computer of some description. This works for OS X, too. I have tested these instructions on Fedora and Ubuntu.

These backups will use rsync, a handy program for copying only the files that have changed. The first backup will obviously copy all of your stuff, but after then if you only create/change one text file, it only copies one text file.

Your receiving computer (let’s call it the server) needs to be running an SSH server. This is standard on almost every Linux/Unix/OS X computer. If not, consult your OS documentation for guidance on how make it go. Don’t forget you need to poke a hole in your firewall (port 22/tcp) as well as setting up the SSH server.

Setting up key authentication

Your sending computer (the client) needs to be set up with a private/public key pair. This is so it can communicate with the server without having to ask for a password. First check if you have keys by doing the following:

jonathan@hyperion:~$ ls ~/.ssh

If this command results in displaying the files id_rsa and id_rsa.pub then you are all set. Skip ahead to the step about copying the key to the server.. If those files or that directory don’t exist then you’ll need to create them, like so:

jonathan@hyperion:~$ ssh-keygen

Accept the default options and it will create a private/public key pair for you. Now we set permissions on your keys to keep them secure, and to ensure that the key authentication works properly:

jonathan@hyperion:~$ chmod 600 ~/.ssh
jonathan@hyperion:~$ chmod 600 ~/.ssh/id_rsa*

Copy the public key (identified by its .pub extension) to the server. Don’t whatever you do copy the private key to any other computer. You should treat the private key as securely as your password Copying is easiest using scp, but you can use a USB memory stick, email attachment or any other method of copying data.

jonathan@hyperion:~$ scp ~/.ssh/id_rsa.pub jonathan@server.com:~

Now you need to log onto the server and tell it to trust your key. Be doubly sure to use a double >>, otherwise you will overwrite the authorized_keys file rather than appending to it.

jonathan@server:~$ cat id_rsa.pub >> .ssh/authorized_keys
jonathan@server:~$ chmod 600 ~/.ssh/authorized_keys

Your key authentication should now be fully set up. You can test it by connection to the server from the client using ssh – if the key authentication is set up properly you will get automatically logged in without being asked for a password.

The backup script

Consider what you actually want to back up. Most likely just your home directory, i.e. /home/jonathan. Also think about where the backups will be stored on the server. If this is your home directory again, then no problem. If you wanted to store it in a different directory, you would probably need to grant write access to that directory to your user account.

In this article, we will assume that I want to back up /home/jonathan on my client to /media/private/Backup/hyperion on the server. So let’s make a script to do this. Call it backup.sh and save it in your home directory. Change the paths, usernames and server name in this example to suit your setup. You can also use an IP address instead of a server name if you don’t have DNS running on your LAN.

#!/bin/sh
rsync -rutvz --delete --exclude=".*" /home/jonathan jonathan@server.example.com:/media/private/Backup/


Note: the line starting with rsync is one long line – don’t put a linebreak in
Note: this excludes hidden files (ones starting with a dot, such as .test). If this is not what you want, remove --exclude=".*" from the rsync line.

Set it as executable, and run it for the first time

jonathan@hyperion:~$ chmod +x backup.sh
jonathan@hyperion:~$ ./backup.sh

Depending on the size of your home folder, the first run could take ages. I strongly recommend having a gigabit LAN in your home for copying large amounts of data. After it has completed, check on the server that your files have indeed made it across to the place you intended.

Now run it again. It should take only a few seconds to run, as no files have changed since you last ran the script.

Create a text file in your home directory with a few words on it. Run your script again. Check the new file got copied to the server.

Delete the text file from your home directory. Run your script again. Check the the file got deleted from the server.

Setting the script to run regularly

If your server and client are on the same LAN, the server is always on, and it’s a fast LAN, the best option will probably be to set this to run regularly.

For example, my desktop PC is set to sync with my server every hour. They are both on gigabit so even if I’ve got loads of new data to copy it rarely takes longer than a few minutes.

My server then syncs with an offsite server every night at 3am, when I don’t notice if my broadband is running slowly due to the traffic. (Yes, maybe you consider this to be OTT, but I have lots of irreplacable photos and recordings, and had you considered what might happen to my data if I was burgled and both PC and server were taken, or if a fire destroyed my home and its contents?)

For scheduling regular jobs, cron is your friend. The syntax can be a bit odd but if you open up /etc/crontab in an editor, you can add some comments to the start as a reminder of what each field means. When you’re done, add a new entry to the bottom, like I have here.

# +---------------- minute (0 - 59)
# |  +------------- hour (0 - 23)
# |  |  +---------- day of month (1 - 31)
# |  |  |  +------- month (1 - 12)
# |  |  |  |  +---- day of week (0-6) (Sun=0 or 7)
# |  |  |  |  |
# *  *  *  *  *  command to be executed

# run-parts
01 * * * * root run-parts /etc/cron.hourly
02 3 * * * root run-parts /etc/cron.daily
22 4 * * 0 root run-parts /etc/cron.weekly
42 4 1 * * root run-parts /etc/cron.monthly
30 * * * * jonathan /home/jonathan/backup.sh

This command tells my script to run at 30 minutes past the hour, every hour of every day of every month. It is vitally important that you run the script as the same user you configured the key authentication for, otherwise it won’t be able to authenticate. Finally fill in the full path to the backup.sh script we prepared earlier.

Setting up an icon in Ubuntu Netbook Remix

If you are running Ubuntu Netbook Remix (UNR), I assume you are also using a netbook. The hourly (or daily, etc) backups described above might not be what you want – you probably don’t have a permanent network. And what if you’re connected via a 3G USB modem, or the slow wireless at friend’s house? Unlikely you’d want to copy several gigabytes of data over such a connection. So the obvious choice here is to put you in charge of when the script runs.

You already know how to run the script manually from the command line, but here we will set up a pretty icon for your desktop. While these instructions are particularly aimed at UNR (because I have it on my EeePC), a similar method will also work in any GNOME environment.

• Click on Preferences, and then Main Menu
• Under the Favourites category, click New Item
• Give it a sensible title, like Backup to server
• Fill in the path to your script
• Pick your favourite icon

That’s it! Now you can just click the icon to start a backup when you know you’re in a position to make a backup.