sebflipper

Backing up data is normally something most people usually neglect (unless your on a Mac), and even if you do backup what happens if you get your stuff stolen or lost by hard drive failure or worst case scenario fire?

Enter the cloud, online backup is nothing new the only difference these days is you normally get more storage for your money. Over the last few weeks I have been keeping an eye out for different solutions:

Service	Pros	Cons
Drop Box	Backs up all your documents and photos online incrementally Keeps things synced across machines (and cross platform) Keeps revision history (so you can go back to previous versions of a document) Allows you to make files or folders public and gives a link to download or generates a photo gallery for you Works on Windows, Mac and Linux	Currently in beta (no public signup) 2GB for free account
	Unlimited storage (paid accounts) Social network for sharing with friends/family and tagging Easy to manage and setup galleries	Photos and videos only 100MB upload per month on free accounts Manual backup process
Mozy	Unlimited automatic incremental backups Set it up and forget it	2GB free account Doesn’t support Linux
Self Hosted	Can be used for backup as well as hosting websites, blogs and galleries etc. Backups work with open-source software like rsync Most hosting companies normally provide 100GB+ of storage space	Manual setup Complex to initially setup

Find out which one I chose after the jump.

As I have already have a hosting plan with Site5 who provide 110GB of storage it only really made sense to make the most of my hosting package.

The process:

1. Create a public/private key to connect over SSH
2. Write a bash script to perform the backup
3. Setup a con job to run it automatically
4. Sit back and let rsync do its job, once the initial backup has succeeded it will then start doing incremental backups (e.g. only uploading changed files/folders)

This process works on Mac and Linux out of the box, but if you are using Windows you will need to tinker around with Cygwin to run bash scripts, rsync and cron jobs etc or you could always use SyncBack in FTP mode.

Step 1: SSH into your web host and enter the following commands:

ssh-keygen -t rsa

When prompted for a path and password just press enter twice. Then:

cat ~/.ssh/id_rsa.pub > ~/.ssh/authorized_keys

Next download ~/.ssh/id_rsa into your document folder and rename to your-site.pk this will be used in the bash script below

Step 2: Write a bash script to backup your data, this step involves playing around with rsync to upload/backup your data. You will need to change values in the config section to match your setup. (Extra bonus if you are using a Mac with growlnotify installed (in the Growl Extras folder) as you will get a notifcation when it starts and finishes backing up)

backup.sh

#! /bin/bash
 
# rsync backup
# Sebastian Flippence
# http://sebflipper.co.uk
# You are free to modify and distribute this code,
# so long as you keep my name and URL in it.
 
# Must be run with sudo (e.g. sudo ./backup.sh)
 
# For some reason rsync doesn't like tilde (home directory) being used in a passed in path variable
# Please use full paths for source and private key
 
# Config
SOURCE="/Users/Seb/" # Your home directory
TARGET="~/backup/Sebs-Mac-Mini" # Where should we save on the remote server
USER="ssh username"
HOST="domain or ip"
PRIVKEY="/Users/Seb/Documents/your-site.pk"
# Uncomment to exclude your Downloads folder
#EXCLUDE="Downloads"
 
# Log Files
TIMESTAMP=`date +%Y-%m`
LOG="/var/log/sf-"$TIMESTAMP"-rsync-backup.log"
 
if [ `id -u` != '0' ] ; then
    echo "You must use sudo to run this command"
    exit 1
fi
 
GROWL=$(which growlnotify)
 
if [ $GROWL ]
then
	$GROWL -st 'rsync' -m 'Backup process has started'
fi
 
if [ -z "$EXCLUDE" ]
then
	rsync -rltDzh --log-file=$LOG --chmod=a+r --delete $SOURCE -e "ssh -p 22 -i $PRIVKEY" $USER@$HOST:$TARGET
else
	rsync -rltDzh --log-file=$LOG --chmod=a+r --delete --delete-excluded --exclude=$EXCLUDE $SOURCE -e "ssh -p 22 -i $PRIVKEY" $USER@$HOST:$TARGET
fi
 
if [ $GROWL ]
then
	$GROWL -st 'rsync' -m 'Backup process has finished'
fi

Step 3: On your machine enter the following command:

sudo crontab -e

Then paste the following:

0 20 * * * /Users/your-username/Documents/backup.sh >/dev/null 2>&1

This will now run at 8pm everyday backing up your precious data.

This entry was posted on Sunday, August 17th, 2008 at 10:18 pm and is filed under Automator Scripts, Development. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.