Backups save the day after data loss

About 13GB of data was accidentally deleted from my account on my Mac yesterday. Fortunately, due to an aggressive backup system, I was able to recover all of the files.

I was trying to test out a tip sent to the TUAW team, which led me to log into (and out of) the "Guest" account several times. One of the logins seemed to hang, which triggered a memory about a data loss bug in Snow Leopard which I had read about (and which Apple is now acknowledging in "extremely rare" cases, per Cnet). I immediately reached around the back of my iMac and shut it down.

At first I thought everything was OK, but then Dropbox started asking me to configure settings, as if it was the first time the app was run. Fortunately, due to the new "LAN sync" feature in recent Dropbox betas, the iMac copied my Dropbox files from my MacBook Pro on the same network in a matter of seconds, rather than having to download almost 2GB of data from Dropbox's servers.

I thought that was the end of the problems, until I saw that my Terminal customizations were missing. Then I noticed that a few other applications were not functioning correctly. Suddenly I realized that I had most likely suffered more data loss than was immediately obvious.

Read on to see what had to be done to recover after the data loss occurred.

SuperDuper is my main backup tool, and it runs every day at 9:30 a.m. on my computer to clone the entire hard drive, so I mounted the drive that it targets and compared the size of my home directory. The current version was 13GB smaller than the backed up version, even though I had not purposefully deleted any files.

Being a command-line nerd, I opened Terminal.app and issued these commands:

cd /Users
find luomat -print | sort > /tmp/luomat.1
cd /Volumes/iMac_Backup/Users
find luomat -print | sort > /tmp/luomat.2
bbdiff /tmp/luomat.1 /tmp/luomat.2

(Note that "luomat" is my username on the iMac.) What did these commands do? They printed out a complete list of all of the files found in my Home directory on my iMac, and then it printed out a complete list of all the files found in the Home directory on my backup drive (/Volumes/iMac_Backup/Users). The "bbdiff" command then opened the two files and compared the differences using BBEdit, which made it easy to see what was gone.

Originally I had planned to manually copy the files from the backup drive, but there were just too many of them. So I fired up ChronoSync 4.0 (and made sure that I had updated to the latest version). I selected the two "luomat" folders in ChronoSync, chose "Synchronize Bidirectional" and made sure that I did NOT check "Synchronize deletions" (since I wanted to make sure that the files on the backup drive were not deleted). Then I ran a "Trial Sync" which showed me what would be done if I chose to do a complete sync.

After the "Trial Sync" ran, I opted to just copy files from the backup drive to the iMac drive. I did this for two reasons: 1. there was nothing on the iMac drive I wanted to copy to the backup drive, which I wanted to keep as-is. When SuperDuper runs again the next time it will take care of updating the backup drive. 2. Dropbox had already updated the files on the iMac drive to the latest version, so I did not want to mess that up either.

ChronoSync ran fairly quickly, and copied the missing data as fast as it could be transferred over USB. I re-ran the Terminal commands to see if anything else was missing, and determined that I had recovered everything that I needed.

Whew.

I can't stress enough how important it is to have a regular, automatic, complete backup of your entire hard drive. Yes, Time Machine (which I do not use, for reasons I will explain below) can also do this, but I was looking at an estimated 14,000 files. Time Machine would have been completely impractical. I didn't even know which files were missing or from where!

You've heard it said many times, but I'll say it again: backup, backup, backup. I have several layers of backups which run every day. Here is how I protect my data:

1. Apple's Backup.app runs on both the iMac and MacBook Pro every day. I use it to backup to my .Mac (sorry, "MobileMe") account, but you can use it to backup elsewhere. This backs up my iCal and Address Book information. This information is also synced via MobileMe, but keeping backups is important in case you realize that corruption has snuck in.

2. SuperDuper (or CarbonCopyCloner) runs every day to backup the entire hard drive.

3. Dropbox is where I keep my "daily" files that I am working with on a regular basis. With 2GB available for free (up to 5GB with referrals), this gives me plenty of space for documents, spreadsheets, the occasional picture, etc. It's not going to hold your entire iPhoto or iTunes library, but it gives you two very important things (even if you don't need to syncing across computers): online backups, and the ability to "undo" changes made in the past 30 days. Also, it works like a regular folder, unlike iDisk. You can upgrade your storage with Dropbox for $9.99/month, which gives you a 50 GB allocation.

4. Gmail. "What? Gmail isn't a backup system." You're right, and what I really mean more than Gmail specifically is IMAP, which maintains your email on the remote server instead of having the only copy of it residing on your hard drive. Gmail is a good choice (recent outages notwithstanding) but most decent ISPs will give you IMAP support. The nice thing about Gmail is the excessive amount of space they give you so you don't have to worry about deleting email if you don't want to. The big take-away here is: if email is important to you, don't leave the only copy of it on your hard drive. Functionally, IMAP is an off-site backup system for email.

If I was to add another part to the system, it would be Time Machine (although maybe not a Time Capsule!) to give an even more well-rounded backup coverage. And someone is bound to mention off-site backups to protect against theft/fire/etc, which is another area that could be improved upon.

That said, what I have is probably more than 75% of the computing world, and yesterday was a reminder for me of why backups are so essential.