Howto: Backup and restore your system!

[m0rphex]

Just succesfully extracted everything on to fresh formatted harddrive from a usb stick with the help of knoppix (it was the only livecd i had lying around). However, I had excluded /dev and that turned out to be a bit hard to fix later since grub-install didn't work. Had to ask for help from a friend Maybe someone could make a howto for this empty /dev scenario.

[SpookyET]

Hello. I'm new to linux and bash scripting, but I do have programming knowledge. I've been looking for an easy to use backup program for Ubuntu. I am yet to find one. I'm hopping for the time machine clones to mature (Flyback and TimeVault).

I want to backup my laptop (running Ubuntu) to a desktop machine which is running Windows XP. I cannot use tools like rsnapshot and rdiff-backup because they cannot create symbolic links on NTFS, even though NTFS supports symbolic links (google for a tool called junction by Sys Internals).
It's best to create a tar archive since it can preserve permissions and symbolic links, and it can be stored on any file system.

Since a backup requires twice the disk space, it's best to write it directly to the desktop machine. So, the script mounts the machine using fuse and sshfs and creates the backup transparently over the network.

The script is by no means finished. It need to account for network failures, support incremental backups and restorations, but it's a start.

Since, I'm new to bash scripting, I'm looking for the community to help improve it.

Requirements:
SSH client, FUSE, SSHFS, etherwake on client (just apt-get them).
SSH server on backup server.

#! /bin/bash

################################################################################
# CONFIGURATION
################################################################################

# Backup server user name
USER=user

# Backup server password
PASSWORD=password

# Backup server host/IP
BACKUP_SERVER_HOST=192.168.1.100

# Backup server MAC address
BACKUP_SERVER_MAC=xx:xx:xx:xx:xx:xx

# Backup server boot wait time (seconds)
BACKUP_SERVER_BOOT_TIME=0

# Backup path on the backup server
BACKUP_DESTINATION="../All\ Users/Documents/Backup"

# This is the backup subdirectory, dictated by date and time
BACKUP_FILE_PREFIX=`date +%Y-%m-%dT%H:%M:%S`

# Backup server mount point
BACKUP_SERVER_MOUNT=/var/backup

# Log file
LOGFILE="/var/log/$BACKUP_FILE_PREFIX.backup.log"

# Include directories or files to backup
INCLUDES=(
	"/"
)

# Exclude directories or files from backup
EXCLUDES=( 
	"/**/.thumbnails/" 
	"/**/.mozilla/**/Cache/"
	"/**/.cache/tracker/"
	"/**/.beagle/"
	"/**/.Trash/"
	"/**/.emerald/themecache/"
	"/**/.fontconfig/*.cache*"
	"/**/.java/deployment/cache/"
	"/**/amarok/albumcovers/cache/"
	"/**/amarok/albumcovers/large/"
	"/**/.liferea*/mozilla/liferea/Cache/"
	"/**/.liferea*/cache/"
	"/**/.macromedia/Flash_Player/*SharedObjects/"
	"/**/.macromedia/Macromedia/Flash\ Player/*SharedObjects/"
	"/**/.metacity/sessions/"
	"/**/.nautilus/saved*"
	"/**/.mythtv/osdcache/"
	"/**/.mythtv/themecache/"
	"/**/var/cache/"
	"/**/workspace/.metadata/"
	"/**/.openoffice.org2/user/registry/cache/"
	"/**/.openoffice.org2/user/uno_packages/cache/"
	"/**/.grails/*/scriptCache/"
	"/**/.wine/drive_c/windows/temp/"
	"/**/lost+found"
	"/cdrom/"
	"/dev/"
	"/media/"
	"/mnt/"
	"/proc/"
	"/sys/" 
	"/tmp/"
	"/var/log"
	$BACKUP_SERVER_MOUNT
)

# Tar options
OPTIONS=(
	"--create"
	"--gzip"
	"--verbose"
	"--preserve"
	"--file=${BACKUP_FILE_PREFIX}.backup.tar.gz"
)

# Populate the tar command
TAR_COMMAND="tar "

for OPTION in ${OPTIONS[@]};
do
	TAR_COMMAND="$TAR_COMMAND $OPTION"
done

for EXCLUDE in ${EXCLUDES[@]};
do
	TAR_COMMAND="$TAR_COMMAND --exclude=$EXCLUDE"
done

for INCLUDE in ${INCLUDES[@]};
do
	TAR_COMMAND="$TAR_COMMAND $INCLUDE"
done

echo $TAR_COMMAND

################################################################################
# Commands
################################################################################

# Dump output to log
echo >> $LOGFILE 1>&2
echo >> $LOGFILE 1>&2
echo "Starting Backup On `date`" >> $LOGFILE 1>&2

# Wake up the backup server
echo "Waking up backup server with HOST ${BACKUP_SERVER_HOST} and MAC ${BACKUP_SERVER_MAC}" >> $LOGFILE 1>&2
etherwake $BACKUP_SERVER_MAC

# Wait an some time to allow the backup server to boot
echo "Waiting ${BACKUP_SERVER_BOOT_TIME} seconds for the backup server to boot" >> $LOGFILE 1>&2
sleep $BACKUP_SERVER_BOOT_TIME

# Test if the backup server is awake.
echo "Pinging backup server to check if it is awake" >> $LOGFILE 1>&2
ping -c1 -W2 $BACKUP_SERVER_HOST > /dev/null 2>&1
PING_RETURN=$?

if [ $PING_RETURN = "0" ] ; then
	echo "The backup server is awake" >> $LOGFILE 1>&2
else
	echo "The backup server is not awake" >> $LOGFILE 1>&2
	echo "BACKUP FAILED"  >> $LOGFILE 1>&2
	exit;
fi

# Mount backup server
echo "Mounting backup server" >> $LOGFILE 1>&2
mkdir $BACKUP_SERVER_MOUNT
sshfs ssh://$USER:$PASSWORD@$BACKUP_SERVER_HOST:$BACKUP_DESTINATION $BACKUP_SERVER_MOUNT

# Back up files
echo "Backing up files" >> $LOGFILE 1>&2
exec `$TAR_COMMAND`  >> $LOGFILE 1>&2
echo "Finished backing up files"

# Unmount backup server
echo "Unmounting backup server." >> $LOGFILE 1>&2
fusermount -u -z $BACKUP_SERVER_MOUNT
rm -R $BACKUP_SERVER_MOUNT

[ephman]

When restoring your system from your backup (using this method) I recommend that you do a fresh reinstall of the OS and then extract the backup.

Regards UUID stuff, see this post:
http://ubuntuforums.org/showthread.p...95#post3860695

Did you not exclude the /boot and fstab ?

yes i did exclude the boot. didn't matter including cause it'll be recreated when i reinstall. it's been a couple months since i had this issue, and since have been running with a non-encrypted drive. but i do remember modifying the fstab for the new uuid's. then i had a grub error. no worries i see now reading the thread you posted i might have to redo my grub. i'll try that in a few days.

thanks,
ephman

[mattforni]

The HOWTO as written with the edits worked beautifully for me. To all those thinking about searching for other backup methods, I don't recommend it; this is too easy and effective. Thanks!

[rugbert]

OK! This is great but for some reason I cant it to run daily even tho I added it to the cron.daily directory.

[Hopworks]

Originally Posted by Hopworks
Just wanted to say that I appreciate your post on backup and restore. I just have a question or two. I wouldn't even bother you except I'm really afraid I'll lose all my work over the past several months. This is my first trip through the wonderful world of Linux, and I had to jump through a lot of hoops to get my LAMP working.

I noticed the command line solution you offered...
tar cvpzf backup.tgz --exclude=/etc/fstab --exclude=/boot --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys /

Just wanted to put that backup file on my second drive. Would I just append to the front of that first part of that line? Like /video/backups/backup.tgz? The 'video' is because that drive was set up to handle my PVR files.

So then I pull the hard drives and put them into my p4 system, and what... install feisty fresh and then unarchive the file to that drive and get back where I was?

Sorry again to bother you. I'm so appreciative of your post and the help it is offering me during this very tense procedure for me. =)

Hop
Hi Hop,

Actaully all you need to do is change directories to /video/backups/ on your second drive and run the command. The backup file is created in whichever directory your in. By default the terminal loads you to /home, so that is where the backup is saved unless you specify otherwise... example cd /dev/sda2//video/backups/. (not sure if sda2 would be your second drive - just a guess- use the fdisk -l command to figure that out)

Anyway I hope this helps, but for future reference please asks these kinds of questions inside the post, so that everyone can benefit.

Take Care,

T9

Done =)

And I'm researching TAR now to find out how to span the archive over multiple files of a set size in case I go over the size of a DVD, and so I can include par files for recovery if needed.
EDIT: I got it. -M or --multi-volume. Just don't know how to specify the size of the spanned files yet. My Linux box is in transition atm, so I can't play with TAR right now.

Thanks again!

EDIT2: Could someone PM me or reply with a good ubuntuforums thread on using TAR? My search of TAR TUTORIAL turned up a lot of hits that are unrelated. =\

[rkillcrazy]

I'm using this to backup my 7.10 AMD64 system. It seems to work well. I'll give the script I use below...

# To use this open a terminal window and become root via:
# sudo su
# Then call this script via:
# sh backup.sh


# This will create a variable of today's date.
NOW=$(date +"%b-%d-%y")


# This will run the backup and name the file with today's date.
tar cvpzf /media/data/backup/backup-$NOW.tgz --exclude=/proc --exclude=/media --exclude=/lost+found --exclude=dvd.iso --exclude=/tmp --exclude=/home/rob/DVDFab --exclude=/mnt --exclude=/sys /

You'll note I excluded several files and directories. Some of them are just junk files I don't want or need to back up. However, I excluded /media because if I didn't exclude it, the script would backup the backup.tgz file which would be redundant and huge. My /media folder has mount points that point to my DVD drives, my second HDD and my file server (used to store other backups) so I wasn't worried about backing that stuff up. However, if you have important things listed there, you my feel different and remove that exclusion.

I do have a question. If one backs their system up and it has sensitive data on it, it would be bad if that tar file got out of the office. It's nice to have these things on DVDs but it also makes it easy to lose. Is there a way to set up the tar file with a password? I know I've done this with 7-Zip in Windows but I'm unsure how to do it with this setup.

01-01-08
1556 EST

[Kache]

Just wondering, why not just do the tar archiving through the use of a live CD? that way, wouldn't all the --exclude stuff not matter anymore (except backing up the archive itself, if needed), since you're no longer running off the hard drive at all?

I was also trying to add empty folders /media, /proc, /lost+found, etc to the backup archive so that I wouldn't have to deal with making them later, but can't seem to figure out how to do that. A quick search online and some experimentation didn't work, so just in case anyone knew....

[lilies34]

thanks for the tip guys! really learned a lot from reading this thread!

[cdtech]

This is a script I wrote you might want to check out.

.:: backup script ::.

:Note the --files-from and --exclude-from, these are text files that tell tar which directories and files to backup and not to backup.

I run this from cron and use a 4G USB stick for the backup (I leave it plugged in). I also backup my laptop with the same script, just changed the mount point to a storage drive.

Hope this helps you..