calraith
January 24th, 2008, 12:46 PM
edit history
2008-1-25: modified backup script to move variables to the top
2008-1-26: updated tarp wrapper to final version, modified backup script to incorporate requested "exclude" ability and to fix --progress-bar flag
2008-1-28: backup script checks whether b[] and ex[] are contiguous (so no more errors if elements are commented out)
2008-1-28: backup script accepts "/" to back up the entire filesystem recursively, automatically skipping some system directories better left as excluded
2008-4-2: no clue why I forgot to exclude /dev when backing up /. Fix'd.
known problems
It seems that symlinks throw off the accuracy of the progress bar. This is only cosmetic -- the backup still works.
Instructions
I use a home-grown script to make my backups. The nice thing about this is that rather than blindly backing up "everything" excluding the directories I don't want or having the script guess what I want backed up as other tutorials show, I can instead specify exactly what I would like to include. This generally only includes conf files I've modified; settings kept in my home directory for apps such as Firefox, irssi, GIMP, etc; the contents of /usr/local/bin; and more obvious stuff such as pictures I have stored in my home directory.
My script uses a perl wrapper for tar that provides a progress bar, called tarp. This wrapper, written by Michal Kwiatkowski (http://joker.linuxstuff.pl/projects/completed), is published under a BSD license (http://www.opensource.org/licenses/bsd-license.php), and is attached at the bottom of this post.
After you save tarp_0.8.3-1_all.deb, double-click it and enter your password to let gdebi install it.
Anyway, as follows is my script. I've updated it so that adding new files and directories to back up is a bit more intuitive. The script will automatically append a / to the end of directory names, escape spaces in file names, and check for files that don't exist. As franestepona requested below, I also added the ability to specify patterns to exclude from the backup. I left my backup list untouched because, I don't know, maybe something in it will remind you to backup a file you would have forgotten about otherwise.
#!/bin/bash
# destination directory
BACKUP_TO=/media/storage
# p = preserve permissions; P = store absolute path; c = create; j = bzip2; f = overwrite
TAR_FLAGS=pcjf
# array of files and directories to be included in the backup -- Enclose any paths
# containing spaces in quotation marks. If / (as the file system root) is specified,
# then /proc, /lost+found, /mnt, /sys, /tmp, /dev, and /media are skipped. If you
# wish to re-include any of those ignored directories, simply specify each as an
# additional item to be included in the backup.
b[0]=/home/rojo/.centericq/
b[1]=/home/rojo/.conkyrc
b[2]=/home/rojo/.gimp-2.4/
b[3]=/home/rojo/Pictures/
b[4]=/home/rojo/se-code.tar
b[5]=/home/rojo/.irssi/
b[6]=/home/rojo/.kiba-dock/
b[7]=/home/rojo/.ssh/
b[8]=/home/rojo/.vnc/
b[9]=/home/rojo/.wacom
b[10]=/home/rojo/.mozilla/
#b[11]=/home/rojo/REISUB.txt
b[12]=/home/rojo/backupscript.sh
b[13]=/home/w00t/eggdrop/
b[14]=/etc/X11/xorg.conf
b[15]=/etc/default/locale
b[16]=/etc/environment
b[17]=/etc/rc.local
b[18]=/etc/bluetooth/hcid.conf
b[19]=/etc/network/interfaces
b[20]=/etc/wpa_supplicant/
b[21]=/etc/nanorc
b[22]=/etc/fail2ban/
b[23]=/etc/fstab
b[24]=/etc/qemu-ifup
b[25]=/etc/samba/smb.conf
b[26]=/etc/hosts.deny
b[27]=/etc/vpnc/
b[28]=/usr/local/
#b[29]=/boot/grub/splashimages/
b[30]=/var/www/
b[31]=/var/lib/locales/
b[32]=/root/.vnc/
b[33]=/root/.ssh/
b[34]=/usr/share/applications/screensavers/
## array of patterns matching files and folders to exclude from the backup
ex[0]=*Cache*
ex[1]=*.bak
ex[2]=*.old
ex[3]=quota.*
############################## nothing else needs to be edited ################################
function make_backup {
local timestamp=$(date | sed -r -e 's/^\S+ //' -e 's/([0-9]+:){2}[0-9]+ //g' -e 's/[A-Z]{3} //' -e 's/ +/-/g')
local bakfile=$BACKUP_TO/backup-$timestamp.tar.bz2
# Unless the P flag is included in $TAR_FLAGS, tar will brag about removing leading slashes.
# By the time this bit executes, we will have already silently removed them. cd / to keep sane.
cd / >/dev/null
time sudo tarp -$TAR_FLAGS $bakfile $INCLUDES --progress-bar
cd $OLDPWD >/dev/null
ls -sh --color $bakfile
}
EXCLUDES=
exLen=${#ex[@]}
for (( x=0; x<$exLen; x++ )); do {
if [ "${ex[$x]}" != "" ]; then
# prepend --exclude= to the element and escape spaces
ex[$x]=`echo '--exclude='${ex[$x]} | sed -r -e 's/ /\\\\ /g'`
EXCLUDES=`echo ''$EXCLUDES ${ex[$x]}`
else
((exLen=$exLen+1))
fi
}; done
INCLUDES=
bLen=${#b[@]}
for (( x=0; x<$bLen; x++ )); do {
if [ "${b[$x]}" != "" ]; then
# if array element is the name of a directory
if [ -d "${b[$x]}" ]; then
# append a trailing /, but only if not already existing
b[$x]=`echo ${b[$x]} | sed -r -e 's/([^\/])$/\1\//'`
else
# otherwise, try removing trailing slash in case this is a regular file
b[$x]=`echo ${b[$x]} | sed -r -e 's/\/$//'`
# if array element *still* can't be found in the filesystem
if [ ! -f "${b[$x]}" ]; then
# don't bother adding it to the backup list -- insult user instead
read -p "Dropping ${b[$x]} -- file does not exist. Replace user and strike any key to continue." -n 1
echo;
b[$x]=" "
fi
fi
# escape spaces in the filename and remove leading slashes (so tar won't make a
# big deal about doing it for us, otherwise ruining our pretty pretty output)
if [ "${b[$x]}" = "/" ]; then
b[$x]=`ls -Amp --ignore=proc --ignore=lost+found --ignore=mnt --ignore=sys --ignore=media --ignore=tmp --ignore=dev / | sed -r -e 's/,//g'`
else
b[$x]=`echo ${b[$x]} | sed -r -e 's/ /\\\\ /g' -e 's/^\///'`
fi
INCLUDES=`echo $INCLUDES ${b[$x]}`
else
((bLen=$bLen+1))
fi
}; done
if [ "$EXCLUDES" != "" ]; then
INCLUDES=`echo ''$EXCLUDES $INCLUDES`
fi
make_backup
It looks like this when run:
http://xs123.xs.to/xs123/08044/backupimg311.gif
2008-1-25: modified backup script to move variables to the top
2008-1-26: updated tarp wrapper to final version, modified backup script to incorporate requested "exclude" ability and to fix --progress-bar flag
2008-1-28: backup script checks whether b[] and ex[] are contiguous (so no more errors if elements are commented out)
2008-1-28: backup script accepts "/" to back up the entire filesystem recursively, automatically skipping some system directories better left as excluded
2008-4-2: no clue why I forgot to exclude /dev when backing up /. Fix'd.
known problems
It seems that symlinks throw off the accuracy of the progress bar. This is only cosmetic -- the backup still works.
Instructions
I use a home-grown script to make my backups. The nice thing about this is that rather than blindly backing up "everything" excluding the directories I don't want or having the script guess what I want backed up as other tutorials show, I can instead specify exactly what I would like to include. This generally only includes conf files I've modified; settings kept in my home directory for apps such as Firefox, irssi, GIMP, etc; the contents of /usr/local/bin; and more obvious stuff such as pictures I have stored in my home directory.
My script uses a perl wrapper for tar that provides a progress bar, called tarp. This wrapper, written by Michal Kwiatkowski (http://joker.linuxstuff.pl/projects/completed), is published under a BSD license (http://www.opensource.org/licenses/bsd-license.php), and is attached at the bottom of this post.
After you save tarp_0.8.3-1_all.deb, double-click it and enter your password to let gdebi install it.
Anyway, as follows is my script. I've updated it so that adding new files and directories to back up is a bit more intuitive. The script will automatically append a / to the end of directory names, escape spaces in file names, and check for files that don't exist. As franestepona requested below, I also added the ability to specify patterns to exclude from the backup. I left my backup list untouched because, I don't know, maybe something in it will remind you to backup a file you would have forgotten about otherwise.
#!/bin/bash
# destination directory
BACKUP_TO=/media/storage
# p = preserve permissions; P = store absolute path; c = create; j = bzip2; f = overwrite
TAR_FLAGS=pcjf
# array of files and directories to be included in the backup -- Enclose any paths
# containing spaces in quotation marks. If / (as the file system root) is specified,
# then /proc, /lost+found, /mnt, /sys, /tmp, /dev, and /media are skipped. If you
# wish to re-include any of those ignored directories, simply specify each as an
# additional item to be included in the backup.
b[0]=/home/rojo/.centericq/
b[1]=/home/rojo/.conkyrc
b[2]=/home/rojo/.gimp-2.4/
b[3]=/home/rojo/Pictures/
b[4]=/home/rojo/se-code.tar
b[5]=/home/rojo/.irssi/
b[6]=/home/rojo/.kiba-dock/
b[7]=/home/rojo/.ssh/
b[8]=/home/rojo/.vnc/
b[9]=/home/rojo/.wacom
b[10]=/home/rojo/.mozilla/
#b[11]=/home/rojo/REISUB.txt
b[12]=/home/rojo/backupscript.sh
b[13]=/home/w00t/eggdrop/
b[14]=/etc/X11/xorg.conf
b[15]=/etc/default/locale
b[16]=/etc/environment
b[17]=/etc/rc.local
b[18]=/etc/bluetooth/hcid.conf
b[19]=/etc/network/interfaces
b[20]=/etc/wpa_supplicant/
b[21]=/etc/nanorc
b[22]=/etc/fail2ban/
b[23]=/etc/fstab
b[24]=/etc/qemu-ifup
b[25]=/etc/samba/smb.conf
b[26]=/etc/hosts.deny
b[27]=/etc/vpnc/
b[28]=/usr/local/
#b[29]=/boot/grub/splashimages/
b[30]=/var/www/
b[31]=/var/lib/locales/
b[32]=/root/.vnc/
b[33]=/root/.ssh/
b[34]=/usr/share/applications/screensavers/
## array of patterns matching files and folders to exclude from the backup
ex[0]=*Cache*
ex[1]=*.bak
ex[2]=*.old
ex[3]=quota.*
############################## nothing else needs to be edited ################################
function make_backup {
local timestamp=$(date | sed -r -e 's/^\S+ //' -e 's/([0-9]+:){2}[0-9]+ //g' -e 's/[A-Z]{3} //' -e 's/ +/-/g')
local bakfile=$BACKUP_TO/backup-$timestamp.tar.bz2
# Unless the P flag is included in $TAR_FLAGS, tar will brag about removing leading slashes.
# By the time this bit executes, we will have already silently removed them. cd / to keep sane.
cd / >/dev/null
time sudo tarp -$TAR_FLAGS $bakfile $INCLUDES --progress-bar
cd $OLDPWD >/dev/null
ls -sh --color $bakfile
}
EXCLUDES=
exLen=${#ex[@]}
for (( x=0; x<$exLen; x++ )); do {
if [ "${ex[$x]}" != "" ]; then
# prepend --exclude= to the element and escape spaces
ex[$x]=`echo '--exclude='${ex[$x]} | sed -r -e 's/ /\\\\ /g'`
EXCLUDES=`echo ''$EXCLUDES ${ex[$x]}`
else
((exLen=$exLen+1))
fi
}; done
INCLUDES=
bLen=${#b[@]}
for (( x=0; x<$bLen; x++ )); do {
if [ "${b[$x]}" != "" ]; then
# if array element is the name of a directory
if [ -d "${b[$x]}" ]; then
# append a trailing /, but only if not already existing
b[$x]=`echo ${b[$x]} | sed -r -e 's/([^\/])$/\1\//'`
else
# otherwise, try removing trailing slash in case this is a regular file
b[$x]=`echo ${b[$x]} | sed -r -e 's/\/$//'`
# if array element *still* can't be found in the filesystem
if [ ! -f "${b[$x]}" ]; then
# don't bother adding it to the backup list -- insult user instead
read -p "Dropping ${b[$x]} -- file does not exist. Replace user and strike any key to continue." -n 1
echo;
b[$x]=" "
fi
fi
# escape spaces in the filename and remove leading slashes (so tar won't make a
# big deal about doing it for us, otherwise ruining our pretty pretty output)
if [ "${b[$x]}" = "/" ]; then
b[$x]=`ls -Amp --ignore=proc --ignore=lost+found --ignore=mnt --ignore=sys --ignore=media --ignore=tmp --ignore=dev / | sed -r -e 's/,//g'`
else
b[$x]=`echo ${b[$x]} | sed -r -e 's/ /\\\\ /g' -e 's/^\///'`
fi
INCLUDES=`echo $INCLUDES ${b[$x]}`
else
((bLen=$bLen+1))
fi
}; done
if [ "$EXCLUDES" != "" ]; then
INCLUDES=`echo ''$EXCLUDES $INCLUDES`
fi
make_backup
It looks like this when run:
http://xs123.xs.to/xs123/08044/backupimg311.gif