My backup and disaster recovery system

As I mentioned briefly a few weeks ago, one of the drives in Cairo, our desktop G4, died recently. This was the main system drive, the one the OS ran off, which meant that while I had to reinstall everything from scratch, our backups were unaffected. As any form of insurance, backups aren't useful until you need them, and only then do you find out if they're sufficient. I'm happy to say that, for the most part, my backup routine worked out pretty well. Almost no important data was lost and once the system was reinstalled I was able to restore everything else pretty painlessly.

Observant readers will notice that I used the word "almost" in that last line. As it turns out, I had neglected to backup a couple minor things that would have been handy to have, and the precautions I took for other things turned out not to be sufficient. In the first case, I was able to get things back to normal, but it just took a little more time and work than it should have. In the second case, I did irretrievably lose data, but it was mostly unimportant and amounts to a minor inconvenience more than anything else. I've modified my procedure a bit and both these things are now covered. Live and learn.

My backup system is based mostly around rsnapshot, which provides multiple "snapshots" of your system at specified intervals. Using rsync and hard links, it approximates the .snapshot backups you get on NetApp file servers. A "snapshot" is a copy of your data as it existed at the time the snapshot was taken. With these snapshots available, deleted files are quickly recovered, and modified files can be easily rolled back to a previous state. I have snapshots taken six times per day, and keep daily, weekly, and monthly copies as well, which are stored for six months.

This is less useful on my Powerbook which, obviously, does not have a second drive to back things up to. For this, I use Unison to sync my home directory on my laptop with that on the desktop every day or two, when I remember to do it. This serves two purposes: It makes my files available to me whether I'm using the Powerbook or the desktop, and it ensures that they are included in the rsnapshot jobs on Cairo.

Some types of files I want to have more fine-grained control over, notably code and other web stuff. I keep Subversion repositories on Cairo and check code in/out from my laptop as needed, which provides not only backups, but file versioning. The repositories themselves are backed up as part of the rsnapshot job, above, as are MySQL dumps.

This system is still missing a couple things. Our 50 GB MP3 library is not backed up anywhere, as the drive our backups are stored on is only 80 GB and it needs to hold all our other data as well. I really need to back this up on DVDs until I buy a larger backup drive. The second major failing is that, while these backups are all stored on a second drive to protect against hardware failure, that second drive is still sitting in the same machine, so we may still be vulnerable to large power surges, fire, or theft. Ideally, I'd periodically copy these to an external firewire drive that's kept off-site, or at least somewhere else in the house. Services like Strongspace are interesting, but they're a little expensive and not really well suited to my needs.

Update

I've written a brief rsnapshot howto.

Update

The music library is now being backed up via rsync to an additional internal hard drive. I'm not including this in the regular snapshots for two reasons. First, this data only changes when we buy more music, so it doesn't require the regular snapshots that normal data does. Second, and more important, the drive the snapshots are stored on isn't big enough to hold all the music in addition to everything else.

Comments

Eric Scalf says:

Kenn, I'm currently beta testing (in private beta) an online data storage service. Although to get any decent amount of space (free is 1gb), you have to pay, it's still an option, and likely cheaper than Strongspace. They're getting ready to roll out a desktop sync tool that will sync your filespace online with your PC/laptop. Whether or not it will work with Mac is unknown at this point, but the base site works with all systems.

The non-disclosure agreement for testing the service states that I can't reveal specifics until Feb 22nd, but at that time, I'll be glad to send you an invitation to the service. (I'll be posting screenshots and a review on my blog on the 22nd as well, so if you want to give it a look-see, first?)

Just thought you might be interested.
-Eric

Kenn Christ says:

Sure, sounds interesting. I'd like to know all my options for this type of thing.

I suppose your NDA prevents your talking much about it now, but for what it's worth, a desktop client isn't important to me, as long as I can use rsync or Unison.

[...] Voices from Catland >> Archives >> My backup and disaster recovery system (tags: backup software) [...]