Imagine yourself waking up one fine morning with a crashed hard disk. How would you feel, all your precious documents, photos and videos all lost in one shot without any warning. That’s where data backups come into the picture and with data growing like never before it is vital not just for enterprises but also for the normal users.
First I will be mentioning the applications I am using to scheduling the auto backup, then I will be mentioning my backup methods.
For the auto backup I am using Cron and RSync. The choice of the applications is a personal one, I choose RSync for making the actual backup because it is a excellent file copying application which finds files that need to be transferred using a “quick check” algorithm which looks for files that have changed in size or in last-modified time.
$ crontab -e05 22 * * 0 rsync -av --delete --progress -exclude=**/*cache*/ --exclude=**/*Cache*/ ~/.thunderbird/ ~/Dropbox/backup/thunderbird/Explanation:
The structure is :
[minute] [hour] [day] [month] [day of Week] [command]where, all the parameters are numbers except the [command]. Hence in the above example cron is scheduled to run rsync at 10:05pm every Sunday (0=Sunday, 1=Monday,…).
Rsync has it own parameters, one can check what each parameter means by giving ‘man rsync’. Basically in the above, it will backup all the contents of the .thunderbird folder in the logged in user’s home directory to the the directory ‘Dropbox/backup/thunderbird’ in the logged in user’s home. While taking the backup it will exclude the cache and Cache folders and delete any files which was present in the earlier backup but not present in the current backup.
Ideally the backups should go to a separate external backup source, but as I am yet to buy an external HDD, I am saving it to the same HDD. This is where the cloud services come into the picture, I will be using Dropbox and Ubuntu One to ensure that I have at least one copy some place other than my HDD. Even in the above example if you notice I have copied the files into the Dropbox folder from where it will be automatically picked up by the Dropbox client and uploaded to the Dropbox servers. This will ensure that the files are accessible from anywhere.
Now that the backup method is in place the next step is identification of the files which require backup. For me that would be:
1. Documents : All of these are present in one main folder, so this makes backup very easy.
2. Mails and profiles : Having used Thunderbird since early 2005 it has lots of mails so this has to be backed up. Other than the Thunderbird mails, I am looking at backing up the Firefox profile, the Liferea profile and the Pidgin profile folder.
3. Photos and Videos: 25 GB of photos in the past three years.
The first two are easily taken care by Dropbox and Ubuntu One. Only thing to note here is that don’t directly sync your profile folders of Thunderbird or for that matter any application as it will result in a strange situation where at the time profile is getting updated the cloud application also tried to upload. Ultimately give a sluggish performance. So to avoid this always move it to a separate folder. For the photos and video, there is no free cloud service which will provide me 25GB so for the moment only the important ones will go to the cloud servers, rest will be move to the external HDD which I will soon be buying.
There are still a few other things which need backup like my blog which I am still looking at.
This was my backup plan, please comment and let me know about any better options if you know about or if you do it differently.
For reference reason I am putting the whole contents of the crontab below:
05 22 * * 0 rsync -av --delete --progress -exclude=**/*cache*/ --exclude=**/*Cache*/ ~/.thunderbird/ ~/Dropbox/backup/thunderbird/
07 22 * * 0 rsync -av --delete --progress -exclude=**/*cache*/ --exclude=**/*Cache*/ ~/.mozilla/ ~/Dropbox/backup/firefox/
30 23 * * 6 rsync -av --delete --progress -exclude=**/*cache*/ --exclude=**/*Cache*/ ~/.liferea_1.6/ ~/Dropbox/backup/liferea/
05 22 * * 6 rsync -av --delete --progress -exclude=**/*cache*/ --exclude=**/*Cache*/ ~/.purple/ ~/Dropbox/backup/pidgin/
Atul Chitnis introducing Philip Tellis at FOSS.in 2009
Lately I have been looking at improving my command line skills in Linux, and here is a quick tip to truncate a file to zero byte file, helpful to empty huge log files quickly.