In this Recipe, I will show you how to build a Linux bootable clone that you can plug in place of a crashed hard disk and run. I will also present a method for creating a DVD-R based archival offsite backup that is capable of being conveniently shipped.
The good news is, you don't need to be a Linux expert to get these backup procedures to work. The procedures described here are at the simple and specific "what to do" level. Though for those of you who want more detail, I will also explain how these procedures work.
By the way, if you need a complete disk data backup, see the simple backup procedure described in my earlier TechBuilder Recipe, Teach Linux to Do Windows.
For this Recipe, please note that specific command-line instructions are encapsulated in command-line scripts, even if they're single-line commands. Each of these scripts can be cut-and-pasted into script files as directed.This reduces the odds of an end user making a mess by incorrectly entering a command-line switch.
Most of the processes described in this article will be executed within a terminal at the command line. What you will see in monospace type within this article will usually refer to processes or programs or files you will see within a terminal.
Most of the commands described here must be executed by the root user. To log in as root:
su %96 root
Building a bootable clone
First, this method is suitable only if you are backing up to an identical hard drive; that is, the same capacity, make, and model. This is the only way to ensure that the head/cylinder hard drive internal arrangements are identical.
Assuming you have an identical drive, let's create a bootable clone with all the hard drive partitions faithfully copied.
The following script unmounts the backup drive if mounted, bit-copies the boot sector onto the designated mirror drive, then bit copies the rest of the hard drive -- "hdb" in my example -- onto the backup hard drive. Change this to fit your own setup, if necessary. The formatting and partition data is copied along with everything else. First, log in as root:
echo "cloning main drive to drive mirror"
dd if=/dev/hdb of=/dev/hdd bs=446 count=1
dd if=/dev/hdb of=/dev/hdd
Next, open a text editor. nano will do, unless you like something better:
Highlight everything between "start" and "end" above (that is, the first character highlighted should be '#', and last should be the final D in hdd). Copy to clipboard, and paste to the text editor. Change the main (/dev/hdb in this example) and backup (/dev/hdd in this example) to fit your actual setup, if necessary. Save. This will put it in the PATH Linux searches for command files.
The following is a backup procedure for use once this script is installed and the hardware is set up. The procedure can either be cut-and-pasted into an alert/reminder program or used as a quick-start printed script for an end user. The DO NOT CLOSE/CLOSE and "Commands can be copied. . ." text relates to its use with a reminder program:
DO NOT CLOSE THIS ALERT.
- Shut down the computer.
- Plug in the hard drive.
- Restart the computer.
- Open a terminal window.
- Log in as root:
su - root
- Once the # prompt reappears -- this could take hours -- type exit.
- Type exit again. Terminal should disappear. CLOSE THIS ALERT.
- When you shut down the computer again, unplug the backup drive.
Put keys and drive in safe place.
The following screen shot shows how a successful drive clone should look in either the Hardware Browser or whatever tool your distribution uses to show information on the hardware (drives, peripherals, etc.) connected to your system. In this case, hdb is the primary Linux drive, and hdd is the clone drive. Note that the partition sizes are identical, as they are supposed to be:
Important: This process copies everything: the boot sector, swap partition, and boot sector. It's time-consuming, but creating the drive partitions needs to be done only once.
To get access to the backup drive content, mount it:
- Login as root.
mount -t ext3 /dev/hdd2 /mnt/linux_backup
- Change ext3 to match the backup drive filesystem if not ext3.
- Change /dev/hdd2 to match your actual backup drive / partition assignment if different.
- Change /mnt/linux_backup to match your actual mount point (directory).
Differential Rsync Backup Script
A differential backup only changes files changed since the last backup. Only one level of increment is needed: "write in changes since last full backup" for a drive-to-drive backup. Your clients can do this daily, since it shouldn't take them more than a few minutes.
rsync_backup.sh is a simple bash script that uses rsync (Remote Sync) to synchronize the backup directory to the source.
Each line updates both a main directory and all subdirectories under it. Anywhere will do, so long as you know where it is. This is one case where permissions and file ownership don't matter, since you will execute the script as root.
This script will mount the backup drive if it isn't already mounted. Each line starting with "rsync" is identical except for the directory names. Other information about the script are in the lines starting with # in the file itself.
-------------- start here
echo "Mounting backup drive."
mount -t ext3 /dev/hdd2 /mnt/linux_backup
#change ext3 to match the backup drive filesystem
#if not ext3
#change /dev/hdd2 to match your actual backup drive
#/ partition assignment if different
#change /mnt/linux_backup to match your actual mount
#delete or comment out mount line above if backup drive
echo "Executing incremental backup script"
echo "Backing up /bin"
rsync -aHv --delete-after --progress /bin/ /mnt/linux_backup/bin/
echo "Backing up /etc"
rsync -aHv --delete-after --progress /etc/ /mnt/linux_backup/etc/
echo "Backing up /home"
rsync -aHv --delete-after --progress /home/ /mnt/linux_backup/home/
echo "Backing up /lib"
rsync -aHv --delete-after --progress /lib/ /mnt/linux_backup/lib/
echo "Backing up /misc"
rsync -aHv --delete-after --progress /misc/ /mnt/linux_backup/misc/
echo "Backing up /opt"
rsync -aHv --delete-after --progress /opt/ /mnt/linux_backup/opt/
echo "Backing up /root"
rsync -aHv --delete-after --progress /root/ /mnt/linux_backup/root/
echo "Backing up /sbin"
rsync -aHv --delete-after --progress /sbin/ /mnt/linux_backup/sbin/
echo "Backing up /usr"
rsync -aHv --delete-after --progress /usr/ /mnt/linux_backup/usr/
echo "Backing up /var"
rsync -aHv --delete-after --progress /var/ /mnt/linux_backup/var/
#see rsync man file for more information (type man rsync from the
To install this script, log in as root, and open a text editor (nano is shown in this case): nano /usr/local/bin/rsync_backup.sh
Highlight every line between "start here" and "end" from this article (that is, the first copied character should be '#', and the last should be the final E in "line".) Copy/paste into your text editor. Save.
In each line of the script, the source and destination directory change. That change aside, each rsync command line in the script is identical to the others.
If you have other directory trees you need to archive, add a line for each as above, with /[directoryname] inserted as above. But don't back up this:
/proc/ /sys/ /dev/ /mnt/
Want to know more? The following explains how the above script works:
#!/bin/sh indicates a bash script. The commands are executed a line at a time. # is a non-executable comment line.
Command-line switches modify the operation of the preceding command. They work in *nix basically the same way in which they work in DOS.
Here are the command-line switches used above to modify rsync as applied to each directory:
- -a archive combines several commands into a function that transfers the files recursively with file permissions intact.
- -H preserves symlinks and hardlinks. This ensures that file aliases continue to function.
- -v verbose: Provides extra information needed to debug and help verify operation.
- --delete-after: After the backup files are transferred, delete any file on your backup drive that does not exist on the main drive you are copying. That way, files deleted from the main drive will be deleted on backup. If you delete a file by mistake, retrieve it from backup before running backup next.
- --progress: Displays information on screen that tells what rsync is doing.
The following is a backup procedure to use once the script is installed and the hardware is set up. It can be cut-and-pasted into an alert/reminder program the same way the COMPLETE BACKUP procedure above can be.
DO NOT CLOSE THIS ALERT.
Commands can be copied and pasted to the terminal.
- Shut down the computer.
- Plug in the hard drive.
- Restart the computer.
- Open a terminal window
- Log in as: root: su %96 root
- Enter: sh rsync_backup.sh
- Once the # prompt reappears, type Exit.
- Type Exit again. Terminal should disappear. CLOSE THIS ALERT.
- Shut down the computer again. Unplug the backup drive.
- Put keys and drive in safe place.
Other rsync and shell script options are in the application manuals made accessible via the man (get manual for) command:
The Unix program manuals that the man command invokes are written with the presupposition that the user already knows *nix. The basic resources mentioned in my earlier TechBuilder Recipe, Teach Linux To Do Windows, will help with the parts that don't make sense.
Every month, do a complete backup procedure, as shown above. This will catch any files that have become corrupt on the backup.
Next, do the archival DVD-R backup, which follows. This is sufficiently time-consuming that you might want to consider doing archival backups only once every other month or even just four times a year.
To restore from clone (mirror) drive:
- Unplug problem hard drive.
- Plug mirror drive where the problem hard drive used to be.
- Boot. Use the dd-based clone procedure above to copy the backup (now main) content to a new backup drive.
How To Archive to DVD-R
There are many backup programs available for this task. But, unfortunately, most require complex installation or configuration. Instead, we need something that is suitable for a Linux novice, a tool that separates creating DVD-sized files from burning them to DVD-R, and that compresses individual files, not directory trees. This way, a disk-media defect will kill just one file, not the entire backup.
Archiving, compressing, and burning-in separate operations avoids the need to spend hours baby-sitting the computer, waiting for it to demand yet another DVD-R. Instead, this procedure first makes DVD-sized archives. Then you burn them to DVD-R as quickly as the computer can create the DVD images. This is still time-consuming, but the only faster alternative is to use a DVD autoloader or DVD library system that costs thousands of dollars.
dar is a Linux hard-drive backup program that creates archives the way described above. It's easy to install and highly functional.
But using dar requires that you have enough space on the hard drive you are backing up for all the DVD ISOs plus some working room to allow for the creation of the temporary files associated with the process. For example, if you're backing up 20 GB of actual disk usage, you're going to need at least 15 extra gigabytes of hard drive space to make this work. When in doubt, get a bigger drive. Hard drives are still cheaper than your time.
Note that the backup and compression process will be extremely time-consuming. This is one of those processes you want to start, then let run overnight. It's also CPU-intensive; while it runs, you can't do anything else with the machine.
As the next screenshot shows, the process labeled 3 (dar) -- indicated by the arrow -- in the terminal window displayed by the top command was only running at 72.2% CPU utilization when the shot was taken. At other times, however, it frequently went over 90%.
I'm assuming a DVD-R drive in the following. Even a minimal personal workstation configuration is too large to record onto any reasonable number of CD-Rs -- especially if you have transitioned an old Windows box onto Win4Lin and ported all personal/business files into the Linux environment. If you are using a tape drive, then there are many tutorials available on the Web, and Google can find them for you. Let's get started:
- Download dar: You can get it here. Find a binary that'll work with your distribution at the links from the download page.
- Install dar:
rpm -i dar-2.1.1-0.2.i686.rpmIt appears that dependencies aren't a problem for it.
- Create a backup directory:
-------------- start here
echo "Executing archival backup script"
dar -z5 -s 1900M -D -R / -c 'date %5C
-I'__data -Z "*.gz" -Z "*.bz2" -Z "*.zip" -Z "*.png" -P tmp %5C
-P dev -P proc -P mnt -P sys -P selinux -P usr/local/backups
- Next, open a text editor (nano will do):
- Highlight everything between "start" and "end" above. (That is, the first highlighted character should be '#', and the last character should be the S in "backups".) Copy the above script to the clipboard, and paste it to the text editor.
How the DVD-R Archive Works
The script is a version of a sample dar command line to be found in the third-party dar tutorial DAR differential backup mini-how-to. I've modified it to fit a workstation environment. It's a good idea to grab a copy of this page to add to your backup documentation for your DVDs.
Each line portion of the dar command above is ended with a backslash.
That's because the dar command is a single line; the backslash tells the terminal that what follows is part of the same command line.
The main command line switches are:
- -z compress - warning: The default is maximum compression, and running it this way pushed my CPU utilization to over 90%.
- -s 1900M: This splits the backup file length into 1900 MB. This means that you'll be burning two archive files per disk. While 4000 MB to fill the DVD-R with one backup file would be preferable, not all Linux software deals reliably with 2000 MB plus individual files. If you have radically different sized backup media (for example, if you are burning to CD), then change this number to fit.
- -Z: Do not compress this file (or file type). Running a second compression over an already compressed file burns CPU time and often increases the file size.
- [-c 'date -I'_data tells the program to create a backup filename based on the current date plus _data. This is where the 2004-10-13_data master backup data name comes from, as you see it below, the current date++ _data.
- -P means "exclude this directory tree." Add directories to exclude as necessary.
The following is a backup procedure for use once the program is installed and the hardware is set up. It can be copied-and-pasted into an alert/reminder program the same way the COMPLETE BACKUP procedure above can be.
Your backup files will appear in /usr/local/backups in compressed 1.9G "slices" with a filename based on date/time:
These instructions are for a stand-alone workstation.
To burn CD/DVD backup to write-once (CD-R, DVD-R) or reusable (CD-RW / DVD-RW), you need to install a CD/DVD burner program if you don't already have one. And, of course, you'll need a CD or DVD burner.
If you do have a burner program, skip to the section "How to Burn Archives to DVD-R" below, and copy the files specified below into your DVDs. I used K3b as a GUI interface to cdrecord, which is included in most Linux distributions.
Install CD/DVD burner software
For users of distributions that support either yum (Fedora Core) or apt-get (Debian) automated installation methods, this is for you.
This is how a modern application is supposed to be installed. The K3b team recommends the use of yum or apt-get and has their own repositories on-site to support these installers.
Next, login as root, and then:
yum install k3b
apt-get install k3b
When the first and second text prompts come from the yum installation program -- "Do you want me to install (or download) ? y/n" -- type Y for yes. Otherwise, the program you want won't install.
If you're running Fedora Core 2 and you see a lot of error messages referring to the /selinux directory tree, ignore them. It's a "not implemented yet" part of that distribution; they generally indicate that the program is installed normally.
After installation, open k3b in KDE menu > Sound & Video > CD & DVD Burning. If it doesn't show up there, open a terminal and type: k3b
How to Burn Archives to DVD-R
You will generally be burning a pair of 1.9 GB archive files to each DVD-R. If you are not using k3b, the interface will differ, but you will be able to do substantially the same things described here.
The Currents Projects subwindow is shown in the following DVD burner program screenshot. It shows what you should be burning to the first DVD-R in your backup set: the first two backup files, backup software, and backup documentation:
Here are the steps for burning archives to DVD-R:
- Open New Project by selecting it from the top toolbar. Via the file window, change directories to /usr/local/backups. Drag in the first (2004-10-13_data.1.dar) and second (2004-10-13_data.2.dar) backup file. Get the dar rpm installer file from wherever you downloaded it to.
I recommend that you save a copy of this article, too. The dar documentation is in /usr/share/dar, and it's quite good; grab the TUTORIAL and the README. Drag-and-drop-in whatever docs or programs you want to make life easier when you're ready to re-install.
Save the project as BACKUP1, then burn it. Label the volume backup1, and burn it. You'll want to put the dar docs into disk 1 of every backup, replacing via drag-and-drop the actual backup files. If the program complains that you are not running as root, log in as root and rerun k3b
Once you are done, close the BACKUP1 project.
- Open New Project: Drag. Drop the third and fourth backup file, with a filenames 2004-10-13_data.3.dar and 2004-10-13_data.4.dar into the second disk. Note the 3 number; subsequent backup files will be numbered 4, 5, etc. Save the project as BACKUP2.
Burn and label it backup 2 (3, 4, etc.). Repeat Step 2 until all backup files are burned to DVD-Rs.
Next, the following is a backup procedure for use with the program once it's installed and the hardware is set up. This procedure can be copied-and-pasted into an alert/reminder program the same way the COMPLETE BACKUP procedure above can be.
CREATE PERMANENT BACKUP ARCHIVE SET
DO NOT CLOSE THIS ALERT
Commands can be cut and pasted to the terminal.
- Open a terminal window.
- Log in as root: su - root
- Enter in terminal (processing will take several hours): sh dar_archive.sh
- Your backup archive files are in /usr/local/backups.
- Open k3b (or other CD/DVD-R burner program).
- Open BACKUP1 from your CD/DVD-R burner program.
- Substitute first two ( .1.dar , .2.dar) files ending in dar
for original dar extension backup files. Do not change other included files.
- Burn the first DVD-R.
- Open BACKUP2. Substitute first two (............003.dar ............ 004.dar) files ending in dar for original dar extension backup files - do not change other included files. Continue to burn pairs of .dar files (005,006. . .) to DVD-R until you run out.
- Erase dar files : cd /usr/local/backups and enter: rm -rf *.dar
- Remove DVD-R backup set to safe place or offsite.
- Close this alert.
Of course, the best time to make a backup is when the machine is loaded as lightly as possible. Also, if a file is changed during the course of a backup, the likely result will be a corrupted file. To avoid this problem, instruct your clients to either back up their files during lunch or run a full backup overnight, whether to backup drive or by creating the files for an archival DVD-R.
But no backup program will work if the user forgets to run it. Most Linux distributions come with an alarm/timer program called KAlarm. This will put a reminder based on text you entered at the time/date specified, or at daily / monthly / weekly / yearly intervals.
To use reminders, in the KDE Menu, go to Accessories > More Accessories > KAlarm (Personal Alarm Scheduler) . If it isn't in the menu, check Add/Remove Applications or your distribution's equivalent to see if it can be automatically installed from your distribution's ISOs. If all else fails, go to this KAlarm page, and look for a version that works with your distribution.
From the Settings > Configure KAlarm, click Run Continuously in System Tray / Autostart at Login / Warn before Quitting as in the image below. This will ensure that the reminders will be available whenever your user starts her computer.
To set up an alert, either click New from the top toolbar or right-click the alarm icon in the System tray and then click New. Next, enter the text of the message. In this case, you will see the Incremental Backup procedure as cut-and-pasted from article. Next, enter the date and time. Click on Confirm Acknowledgement.
To set recurrence, click the Recurrence top tab:
Incremental backups are set here on a weekly basis: Monday, Wednesday, and Friday. This screen shot shows how the alert will look to the user:
Finally, let's restore to a test directory. For a bare-metal post-crash install, do a minimal install of a Linux distribution -- preferably the one you were previously running -- so you have an environment to run dar in. Print copies of the backup documentation from the first disk, including a copy of this article. This is in case you want to do something other than a full restore with overwrite of all files from your DVD backup set.
Next, copy the compressed backups from DVD and the documentation:
Install dar as above. If that doesn't work, try copying this:
/etc/dar_staticin place of dar in the line below:
dar -x /usr/local/backups/2004-10-13_data -R /usr/local/backups/test
The above command is a single line you can copy/paste into a terminal window. Substitute the directory name wherever you want to put the restore for /test and your actual filename for 2004-10-13_data. Add -w to the end of the command line if you wants to overwrite without warning. That is, if you are restoring straight to / ... add -b to the end of the command line "do not overwrite files later than archive being restored".
Once the restore is complete, copy the files to the respective directories for your home directory:
cp -au /usr/local/backups/home /home
This lets you be more selective about copying files and directories. But if space doesn't permit, you can simply copy it to / and overwrite. But do read the tutorial docs before doing anything that hasn't been described here. It's only your client's data (or your own) that's at stake!
A. LIZARD is an Internet consultant in the San Francisco Bay Area. He has been writing for technology magazines and Web sites since 1987.
What did you think? Discuss this Recipe with other system builders in the TechBuilder Recipe Forum.