Ubuntu Forums ubuntu.com - launchpad.net - ubuntu help  

Go Back   Ubuntu Forums > The Ubuntu Forum Community > Other Community Discussions > Tutorials & Tips
Register Reset Password Forum Help Forum Council Search Today's Posts Mark Forums Read

Tutorials & Tips
The place to find Ubuntu related Tips & Tricks.

 
Thread Tools Display Modes
Old December 13th, 2007   #1
altonbr
Fresh Brewed Ubuntu
 
altonbr's Avatar
 
Join Date: Mar 2006
Location: Canada
Beans: 1,216
Ubuntu 9.10 Karmic Koala
HOWTO: Backup nightly via rsync

Preamble
As a Linux System Administrator, I've had to keep two servers in synchronization, either nightly or every couple of hours. The term is 'mirror' and can be seen in use when downloading Ubuntu .iso files from localized servers, say from a Cambodian server when the main server is in Britain. (You may not know this, but that's how it works, using pretty much the same script you see below).

Why should I use this script?
You don't have to be a Linux System Administrator to enjoy the beautiful interoperability benefits of Linux, you can put it to use right at home!

Use this script to backup your entire family's data, every night, by using a central server and making the server 'pull' everyone's documents to it.

How do I use this script?
How? Well if you purchase a 500GB server, then you could potentially backup five family member's computers if they each had a 100GB hard drive.

Simply use this script with the 'install' argument five times on the server and it will create five keys, one for each of your family members.

You need to setup either DNS names for the computers or static IP addresses for each computer.

I won't go into great detail on how to do this in every situation, but for a home network, you can change the IP address of one of your computers to a static IP address like this:
Code:
gksu gedit /etc/network/interfaces
Before:
Quote:
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet dhcp
After:
Quote:
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
#iface eth0 inet dhcp

iface eth0 inet static
address 192.168.1.10 # computer1 = 192.168.1.10, computer2 = 192.168.1.11, computer3 = 192.168.1.12, etc.
netmask 255.255.255.0
gateway 192.168.1.1
As for the Linux System Administrators, I don't need to tell you how useful this script is

Any alternatives?
Of course! I'm not the first person to need file-synchronization or mirroring, but I feel most comfortable writing my own scripts so I know what's going on and can control the encryption strength.

GUI (Graphical User Interface)
unison: a file synchronization program used to sync files between two directories, either on one computer, or between a computer and another storage device. (I've never used it, but I heard its reliable)
Code:
sudo aptitude install unison unison-gtk
CLI (Command Line)
rsnapshot: a filesystem snapshot utility for making backups of local and remote systems. The disk space required is just a little more than the space of one full backup, plus incrementals. (My program does not use incrementals).
Code:
sudo aptitude install rsnapshot
Vocabulary
  • ssh: a network protocol that allows data to be exchanged using a secure channel (encryption) between two computers
  • rsync: a software application for Linux which synchronizes files and directories from one location to another using minimal bandwidth (only transfers files (or parts of files) that don't exist)
  • mirror: an exact copy of a data set
  • push: to send (or give) data from one computer to another
  • pull: to receive (or ask) for data from one computer to another

Let's get started
It has several flags and options, but some are hard-coded, so you may have to edit the script by hand.

Options
General usage:
Quote:
./rsync.sh ['uninstall' | 'install' | 'run'] ['push' | 'pull'] [local_dir] [remote_user] [remote_host] [remote_dir] [remote_ssh_port]
Note: '[remote_ssh_port]' is usually 22 unless you change it from the default. If you don't know how to change the ssh port, its more than likely 22.

install
Use this option if you want to setup an automated, nightly backup between one computer and another.
Arguments:
Quote:
./rsync.sh install [push | pull] [local_dir] [remote_user] [remote_host] [remote_dir] [remote_ssh_port]
Example:
Quote:
./rsync.sh install pull /home/brett/ brett 192.168.1.2 /home/brett 2222
  • Creates the following directories: $HOME/bin, $HOME/cron, $HOME/logs
  • Creates 4096-bit RSA encryption key (illegal in the USA I think)
  • Installs file to directories above

uninstall
Use this option if you want to remove a previously installed setup.
Arguments:
Quote:
./rsync.sh uninstall [remote_user] [remote_host]
Example:
Quote:
./rsync.sh uninstall brett 192.168.1.2
  • Removes cron-job
  • Removes RSA key

run
Use this option if you want to run the program without installing anything. Good to use as a test and on a single-needs basis.
Arguments:
Quote:
./rsync.sh run [push | pull] [local_dir] [remote_user] [remote_host] [remote_dir] [remote_ssh_port]
Example:
Quote:
./rsync.sh run pull /home/brett/ brett 192.168.1.2 /home/brett 2222
  • Runs rsync with your parameters

rsync.sh
Code:
#!/bin/bash

# rsync between two different servers - optimized for multiple cron jobs
# rsync.sh
# Copyright (C) 2008  Brett Alton (Alton Labs)

# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.

# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.

# You should have received a copy of the GNU General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses/>.


# INFO:
# 	* you can either run this script using 'run' then 'push' or 'pull', you can install it using 'install' then 'push' or 'pull'  or you can choose the 'uninstall' option to remove previously installed files
#	* if rsync/ssh are both prompting you for your password and it should be automatic, run 'rsync.sh uninstall' and then try again

# DIRECTORIES CREATED:
# 	* $HOME/
#		* bin
#		* cron
#		* logs

# USAGE: #!/bin/bash

# rsync between two different servers - optimized for multiple cron jobs
# rsync.sh
# Copyright (C) 2008  Brett Alton (Alton Labs)

# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.

# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.

# You should have received a copy of the GNU General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses/>.


# INFO:
# 	* you can either run this script using 'run' then 'push' or 'pull', you can install it using 'install' then 'push' or 'pull'  or you can choose the 'uninstall' option to remove previously installed files
#	* if rsync/ssh are both prompting you for your password and it should be automatic, run 'rsync.sh uninstall' and then try again

# DIRECTORIES CREATED:
# 	* $HOME/
#		* bin
#		* cron
#		* logs

# USAGE: rsync.sh ['uninstall' | 'install' | 'run'] ['push' | 'pull'] [local_dir] [remote_user] [remote_host] [remote_dir] [remote_ssh_port]


# === FUNCTIONS ===
# --- print and exit functions ---
function print_info()
{
	echo " -- $1, continuing..."
}

function print_warn()
{
	echo " ** $1. You may want to look into this, continuing..."
}

function force_exit()
{
	echo " !! $2, exiting..."
	echo " !! USAGE: $0 ['uninstall' | 'install' | 'run'] ['push' | 'pull'] [local_dir] [remote_user] [remote_host] [remote_dir] [remote_ssh_port]"
	cleanup # cleanup rsync log files
	exit $1
}

function safe_exit()
{
	echo " -- Safely exiting..."
	cleanup # cleanup rsync log files
	exit 0
}

# --- helper functions ---
function check_dir()
{
	if [ ! -d $1 ]; then
		print_info "Creating directory: $1"

		mkdir -p $1 # if $1 doesn't exist, create it
		if [ $? -ne 0 ]; then
			force_exit 1 "Could not create $1"
		fi
	fi
}

function check_key()
{
	# make sure the local key has been created
	if [ ! -f $1 ]; then
		create_key
	fi
}

function create_key()
{
	# create key
	print_info 'Creating encryption key (this may take some time)'
	ssh-keygen -q -t rsa -b $ENCRYPTION_STRENGTH -f $LOCAL_KEY_FILE -N '' # quiet, type (rsa), encryption strength, filename, passphrase (empty)
	if [ $? -ne 0 ]; then
		uninstall # no point having the program installed with no encryption key
		force_exit 1 'Could not create encryption key'
	fi

	# upload key
	print_info 'Sending encryption key'
	ssh-copy-id -i $LOCAL_KEY_FILE "-p $REMOTE_SSH_PORT $REMOTE_USER@$REMOTE_HOST"
	if [ $? -ne 0 ]; then
		uninstall # no point having the program installed without the encryption key uploaded
		force_exit 1 'Could not upload encryption key'
	fi
}

function uninstall()
{
	#if [ -f $LOCAL_BIN_FILE ]; then
	#	print_info 'Uninstalling program'
	#	rm -f $LOCAL_BIN_FILE
	#	if [ $? -ne 0 ]; then
	#		print_warn "Could not remove $LOCAL_BIN_FILE"
	#	fi
	#fi

	if [ -f $LOCAL_CRON_FILE ]; then
		print_info 'Uninstalling cron file'

		crontab -u $LOCAL_USER -r # deletes user entire crontab
		if [ $? -ne 0 ]; then
			print_warn "Could not uninstall $LOCAL_USER's crontab"
		fi

		rm -f $LOCAL_CRON_FILE
		if [ $? -ne 0 ]; then
			print_warn "Could not remove $LOCAL_CRON_FILE"
		fi
	fi

	if [ -f $LOCAL_KEY_FILE ] || [ -f $LOCAL_KEY_FILE.pub ]; then
		print_info 'Uninstalling encryption keys'

		rm -f $LOCAL_KEY_FILE $LOCAL_KEY_FILE.pub
		if [ $? -ne 0 ]; then
			print_warn "Could not remove $LOCAL_KEY_FILE and/or $LOCAL_KEY_FILE.pub"
		fi
	fi
}

function cleanup()
{
	# gunzip the logfile
	if [ -f $LOCAL_LOG_FILE ]; then
		print_info 'Compressing log file'

		gzip -c $LOCAL_LOG_FILE > $LOCAL_LOG_FILE.gz
		if [ $? -ne 0 ]; then
			print_warn "Could not compress $LOCAL_LOG_FILE"
		fi
	fi

	# delete the original, uncompressed logfile
	if [ -f $LOCAL_LOG_FILE ]; then
		print_info 'Removing uncompressed log file'

		rm $LOCAL_LOG_FILE
		if [ $? -ne 0 ]; then
			print_warn "Could not remove $LOCAL_LOG_FILE"
		fi
	fi
}

# === VARIABLES ===
# %Y     year
# %m     month (01..12)
# %d     day of month (e.g, 01)
# %s     seconds since 1970-01-01 00:00:00 UTC
THEDATE=`date '+%Y%m%d-%s'` # 20071010-1192044000
ENCRYPTION_STRENGTH=4096 # bits in length for rsa key (1024,2048,4096,8172,10240,20480,etc)

# parameters
if [ "$1" == "install" ] || [ "$1" == "run" ]; then
	ACTION=$1 # install? uninstall? run?
	METHOD=$2 # push? pull?
	LOCAL_DIR=$3
	REMOTE_USER=$4 # ie: brett, root
	REMOTE_HOST=$5 # ie: 192.168.1.2, example.com, sub.domain.example.com
	REMOTE_DIR=$6
	REMOTE_SSH_PORT=$7 # default is 22
else # uninstall or unknown
	ACTION=$1
	REMOTE_USER=$2
	REMOTE_HOST=$3
fi

# environment
LOCAL_USER=$USER # ie: brett, root
LOCAL_HOME=$HOME # ie: /home/brett, /root

# executable
LOCAL_BIN_NAME='rsync.sh'
LOCAL_BIN_PATH=$LOCAL_HOME/bin
LOCAL_BIN_FILE=$LOCAL_BIN_PATH/$LOCAL_BIN_NAME # ie: /home/brett/bin/rsync.sh

# unique key
KEY="$REMOTE_HOST-$REMOTE_USER" # 192.168.1.2-brett, example.com-root

# cron
LOCAL_CRON_NAME="rsync-$KEY.cron"
LOCAL_CRON_PATH=$LOCAL_HOME/cron
LOCAL_CRON_FILE=$LOCAL_CRON_PATH/$LOCAL_CRON_NAME # ie: /home/brett/cron/rsync-ssh-example.com-root.cron
LOCAL_CRON_TIME='0 2 * * *' # 2am # minute, hour, day of month, month, day of week

# rsa/dsa key
LOCAL_KEY_NAME="rsync-$KEY" # using local username to avoid collisions
LOCAL_KEY_PATH=$LOCAL_HOME/.ssh
LOCAL_KEY_FILE=$LOCAL_KEY_PATH/$LOCAL_KEY_NAME # ie: /home/brett/.ssh/altonlabs-rsync-ssh

# log
LOCAL_LOG_NAME="rsync-$KEY-$THEDATE.log"
LOCAL_LOG_PATH=$LOCAL_HOME/logs
LOCAL_LOG_FILE=$LOCAL_LOG_PATH/$LOCAL_LOG_NAME # ie: /home/brett/logs/rsync-20071010-1192044000.log
LOCAL_LOG_GZ_FILE=$LOCAL_LOG_FILE.gz # ie: /home/brett/logs/rsync-20071010-1192044000.log.gz


# === LOGIC ===
# --- UPLOADING (push) / DOWNLOADING (pull) ---
if [ "$ACTION" == "run" ]; then

	# need 7 parameters to continue
	if [ $# -ne 7 ]; then
		force_exit 1 "Improper number of parameters ($#)"
	fi

	check_dir $LOCAL_LOG_PATH

	# run rsync
	# -a, --archive               archive mode; equals -rlptgoD (no -H,-A,-X)
	# -v, --verbose               increase verbosity
	# -z, --compress              compress file data during the transfer
	# -e, --rsh=COMMAND           specify the remote shell to use
	#     --delete                delete extraneous files from dest dirs
	#     --log-file=FILE         log what we're doing to the specified FILE

	# TODO: apperently Red Hat/Fedora/CentOS doesn't have the --log-file option in rsync, so I must add Debian/Ubuntu vs CentOS detection

	if [ "$METHOD" == "push" ]; then # local to remote
		rsync -avz --rsh="ssh -l $REMOTE_USER -p $REMOTE_SSH_PORT -i $LOCAL_KEY_FILE" --delete $LOCAL_DIR $REMOTE_USER@$REMOTE_HOST:$REMOTE_DIR # --log-file=$LOCAL_LOG_FILE
	elif [ "$METHOD" == "pull" ]; then # remote to local
		rsync -avz --rsh="ssh -l $REMOTE_USER -p $REMOTE_SSH_PORT -i $LOCAL_KEY_FILE" --delete $REMOTE_USER@$REMOTE_HOST:$REMOTE_DIR $LOCAL_DIR # --log-file=$LOCAL_LOG_FILE
	else
		echo " ** Incorrect method selected: $METHOD (should be 'push' or 'pull')" # print to screen
		echo " ** Incorrect method selected: $METHOD (should be 'push' or 'pull')" > $LOCAL_LOG_FILE # echo to log file
	fi

# --- INSTALLING ---
elif [ "$ACTION" == "install" ]; then

	# needs 7 parameters to continue
	if [ $# -ne 7 ]; then
		force_exit 7 "Improper number of parameters ($#)"
	fi

	check_dir $LOCAL_BIN_PATH
	check_dir $LOCAL_CRON_PATH
	check_dir $LOCAL_LOG_PATH
	check_key $LOCAL_KEY_FILE

	# install program to $LOCAL_BIN_FILE
	print_info 'Installing program'
	cp -pf $0 $LOCAL_BIN_FILE # force copy to make sure this current version is the newest
	if [ $? -ne 0 ]; then
		uninstall # if you can't install it, remove anything left behind
		force_exit 1 "Could not install program at $LOCAL_BIN_FILE"
	fi

	# create cron
	# http://en.wikipedia.org/wiki/cron#Fields
	print_info 'Creating cron file'
	echo "$LOCAL_CRON_TIME $LOCAL_BIN_FILE run $METHOD $LOCAL_DIR $REMOTE_USER $REMOTE_HOST $REMOTE_DIR $REMOTE_SSH_PORT" > $LOCAL_CRON_FILE # not checking to see if cron file already exists because we want to overwrite
	if [ $? -ne 0 ]; then
		uninstall # if you can't install it, remove anything left behind
		force_exit 1 "Could not create cron file at $LOCAL_CRON_FILE"
	fi

	# install cron
	print_info 'Registering cron file'
	crontab -u $LOCAL_USER $LOCAL_CRON_FILE
	if [ $? -ne 0 ]; then
		uninstall # if you can't install it, remove anything left behind
		force_exit 1 "Could not register cron file $LOCAL_CRON_FILE to $LOCAL_USER"
	fi

# --- UNINSTALLING ---
elif [ "$ACTION" == "uninstall" ]; then
	# need only 1 parameter to continue
	if [ $# -ne 3 ]; then
		force_exit 1 "Improper number of parameters ($#)"
	fi

	uninstall # call uninstall function

# --- WHOOPS ---
else
	force_exit 1 'Unknown parameter'
fi

safe_exit # everything went fine



# === FUNCTIONS ===
# --- print and exit functions ---
function print_info()
{
	echo " -- $1, continuing..."
}

function print_warn()
{
	echo " ** $1. You may want to look into this, continuing..."
}

function force_exit()
{
	echo " !! $2, exiting..."
	echo " !! USAGE: $0 ['uninstall' | 'install' | 'run'] ['push' | 'pull'] [local_dir] [remote_user] [remote_host] [remote_dir] [remote_ssh_port]"
	cleanup # cleanup rsync log files
	exit $1
}

function safe_exit()
{
	echo " -- Safely exiting..."
	cleanup # cleanup rsync log files
	exit 0
}

# --- helper functions ---
function check_dir()
{
	if [ ! -d $1 ]; then
		print_info "Creating directory: $1"

		mkdir -p $1 # if $1 doesn't exist, create it
		if [ $? -ne 0 ]; then
			force_exit 1 "Could not create $1"
		fi
	fi
}

function check_key()
{
	# make sure the local key has been created
	if [ ! -f $1 ]; then
		create_key
	fi
}

function create_key()
{
	# create key
	print_info 'Creating encryption key (this may take some time)'
	ssh-keygen -q -t rsa -b $ENCRYPTION_STRENGTH -f $LOCAL_KEY_FILE -N '' # quiet, type (rsa), encryption strength, filename, passphrase (empty)
	if [ $? -ne 0 ]; then
		uninstall # no point having the program installed with no encryption key
		force_exit 1 'Could not create encryption key'
	fi

	# upload key
	print_info 'Sending encryption key'
	ssh-copy-id -i $LOCAL_KEY_FILE "-p $REMOTE_SSH_PORT $REMOTE_USER@$REMOTE_HOST"
	if [ $? -ne 0 ]; then
		uninstall # no point having the program installed without the encryption key uploaded
		force_exit 1 'Could not upload encryption key'
	fi
}

function uninstall()
{
	#if [ -f $LOCAL_BIN_FILE ]; then
	#	print_info 'Uninstalling program'
	#	rm -f $LOCAL_BIN_FILE
	#	if [ $? -ne 0 ]; then
	#		print_warn "Could not remove $LOCAL_BIN_FILE"
	#	fi
	#fi

	if [ -f $LOCAL_CRON_FILE ]; then
		print_info 'Uninstalling cron file'

		crontab -u $LOCAL_USER -r # deletes user entire crontab
		if [ $? -ne 0 ]; then
			print_warn "Could not uninstall $LOCAL_USER's crontab"
		fi

		rm -f $LOCAL_CRON_FILE
		if [ $? -ne 0 ]; then
			print_warn "Could not remove $LOCAL_CRON_FILE"
		fi
	fi

	if [ -f $LOCAL_KEY_FILE ] || [ -f $LOCAL_KEY_FILE.pub ]; then
		print_info 'Uninstalling encryption keys'

		rm -f $LOCAL_KEY_FILE $LOCAL_KEY_FILE.pub
		if [ $? -ne 0 ]; then
			print_warn "Could not remove $LOCAL_KEY_FILE and/or $LOCAL_KEY_FILE.pub"
		fi
	fi
}

function cleanup()
{
	# gunzip the logfile
	if [ -f $LOCAL_LOG_FILE ]; then
		print_info 'Compressing log file'

		gzip -c $LOCAL_LOG_FILE > $LOCAL_LOG_FILE.gz
		if [ $? -ne 0 ]; then
			print_warn "Could not compress $LOCAL_LOG_FILE"
		fi
	fi

	# delete the original, uncompressed logfile
	if [ -f $LOCAL_LOG_FILE ]; then
		print_info 'Removing uncompressed log file'

		rm $LOCAL_LOG_FILE
		if [ $? -ne 0 ]; then
			print_warn "Could not remove $LOCAL_LOG_FILE"
		fi
	fi
}

# === VARIABLES ===
# %Y     year
# %m     month (01..12)
# %d     day of month (e.g, 01)
# %s     seconds since 1970-01-01 00:00:00 UTC
THEDATE=`date '+%Y%m%d-%s'` # 20071010-1192044000
ENCRYPTION_STRENGTH=4096 # bits in length for rsa key (1024,2048,4096,8172,10240,20480,etc)

# parameters
if [ "$1" == "install" ] || [ "$1" == "run" ]; then
	ACTION=$1 # install? uninstall? run?
	METHOD=$2 # push? pull?
	LOCAL_DIR=$3
	REMOTE_USER=$4 # ie: brett, root
	REMOTE_HOST=$5 # ie: 192.168.1.2, example.com, sub.domain.example.com
	REMOTE_DIR=$6
	REMOTE_SSH_PORT=$7 # default is 22
else # uninstall or unknown
	ACTION=$1
	REMOTE_USER=$2
	REMOTE_HOST=$3
fi

# environment
LOCAL_USER=$USER # ie: brett, root
LOCAL_HOME=$HOME # ie: /home/brett, /root

# executable
LOCAL_BIN_NAME='rsync.sh'
LOCAL_BIN_PATH=$LOCAL_HOME/bin
LOCAL_BIN_FILE=$LOCAL_BIN_PATH/$LOCAL_BIN_NAME # ie: /home/brett/bin/rsync.sh

# unique key
KEY="$REMOTE_HOST-$REMOTE_USER" # 192.168.1.2-brett, example.com-root

# cron
LOCAL_CRON_NAME="rsync-$KEY.cron"
LOCAL_CRON_PATH=$LOCAL_HOME/cron
LOCAL_CRON_FILE=$LOCAL_CRON_PATH/$LOCAL_CRON_NAME # ie: /home/brett/cron/rsync-ssh-example.com-root.cron
LOCAL_CRON_TIME='0 2 * * *' # 2am # minute, hour, day of month, month, day of week

# rsa/dsa key
LOCAL_KEY_NAME="rsync-$KEY" # using local username to avoid collisions
LOCAL_KEY_PATH=$LOCAL_HOME/.ssh
LOCAL_KEY_FILE=$LOCAL_KEY_PATH/$LOCAL_KEY_NAME # ie: /home/brett/.ssh/altonlabs-rsync-ssh

# log
LOCAL_LOG_NAME="rsync-$KEY-$THEDATE.log"
LOCAL_LOG_PATH=$LOCAL_HOME/logs
LOCAL_LOG_FILE=$LOCAL_LOG_PATH/$LOCAL_LOG_NAME # ie: /home/brett/logs/rsync-20071010-1192044000.log
LOCAL_LOG_GZ_FILE=$LOCAL_LOG_FILE.gz # ie: /home/brett/logs/rsync-20071010-1192044000.log.gz


# === LOGIC ===
# --- UPLOADING (push) / DOWNLOADING (pull) ---
if [ "$ACTION" == "run" ]; then

	# need 7 parameters to continue
	if [ $# -ne 7 ]; then
		force_exit 1 "Improper number of parameters ($#)"
	fi

	check_dir $LOCAL_LOG_PATH

	# run rsync
	# -a, --archive               archive mode; equals -rlptgoD (no -H,-A,-X)
	# -v, --verbose               increase verbosity
	# -z, --compress              compress file data during the transfer
	# -e, --rsh=COMMAND           specify the remote shell to use
	#     --delete                delete extraneous files from dest dirs
	#     --log-file=FILE         log what we're doing to the specified FILE

	# TODO: apperently Red Hat/Fedora/CentOS doesn't have the --log-file option in rsync, so I must add Debian/Ubuntu vs CentOS detection

	if [ "$METHOD" == "push" ]; then # local to remote
		rsync -avz --rsh="ssh -l $REMOTE_USER -p $REMOTE_SSH_PORT -i $LOCAL_KEY_FILE" --delete $LOCAL_DIR $REMOTE_USER@$REMOTE_HOST:$REMOTE_DIR --log-file=$LOCAL_LOG_FILE
	elif [ "$METHOD" == "pull" ]; then # remote to local
		rsync -avz --rsh="ssh -l $REMOTE_USER -p $REMOTE_SSH_PORT -i $LOCAL_KEY_FILE" --delete $REMOTE_USER@$REMOTE_HOST:$REMOTE_DIR $LOCAL_DIR --log-file=$LOCAL_LOG_FILE
	else
		echo " ** Incorrect method selected: $METHOD (should be 'push' or 'pull')" # print to screen
		echo " ** Incorrect method selected: $METHOD (should be 'push' or 'pull')" > $LOCAL_LOG_FILE # echo to log file
	fi

# --- INSTALLING ---
elif [ "$ACTION" == "install" ]; then


	# needs 7 parameters to continue
	if [ $# -ne 7 ]; then
		force_exit 7 "Improper number of parameters ($#)"
	fi

	check_dir $LOCAL_BIN_PATH
	check_dir $LOCAL_CRON_PATH
	check_dir $LOCAL_LOG_PATH
	check_key $LOCAL_KEY_FILE

	# install program to $LOCAL_BIN_FILE
	print_info 'Installing program'
	cp -pf $0 $LOCAL_BIN_FILE # force copy to make sure this current version is the newest
	if [ $? -ne 0 ]; then
		uninstall # if you can't install it, remove anything left behind
		force_exit 1 "Could not install program at $LOCAL_BIN_FILE"
	fi

	# create cron
	# http://en.wikipedia.org/wiki/cron#Fields
	print_info 'Creating cron file'
	echo "$LOCAL_CRON_TIME $LOCAL_BIN_FILE run $METHOD $LOCAL_DIR $REMOTE_USER $REMOTE_HOST $REMOTE_DIR $REMOTE_SSH_PORT" > $LOCAL_CRON_FILE # not checking to see if cron file already exists because we want to overwrite
	if [ $? -ne 0 ]; then
		uninstall # if you can't install it, remove anything left behind
		force_exit 1 "Could not create cron file at $LOCAL_CRON_FILE"
	fi

	# install cron
	print_info 'Registering cron file'
	crontab -u $LOCAL_USER $LOCAL_CRON_FILE
	if [ $? -ne 0 ]; then
		uninstall # if you can't install it, remove anything left behind
		force_exit 1 "Could not register cron file $LOCAL_CRON_FILE to $LOCAL_USER"
	fi

# --- UNINSTALLING ---
elif [ "$ACTION" == "uninstall" ]; then
	# need only 1 parameter to continue
	if [ $# -ne 3 ]; then
		force_exit 1 "Improper number of parameters ($#)"
	fi

	uninstall # call uninstall function

# --- WHOOPS ---
else
	force_exit 1 'Unknown parameter'
fi

safe_exit # everything went fine
__________________
Tutorials: rsync and ssh
Website: brettalton.com
Articles: FCM #20, FCM #21, FCM #22, FCM #23

Last edited by altonbr; November 22nd, 2008 at 01:51 PM.. Reason: reactive --log-file=$LOCAL_LOG_FILE for Ubuntu users
altonbr is offline   Reply With Quote
Old December 24th, 2007   #2
grim4593
Just Give Me the Beans!
 
grim4593's Avatar
 
Join Date: Apr 2006
Location: Michigan
Beans: 55
Ubuntu 8.10 Intrepid Ibex
Re: HOWTO: Backup nightly via rsync

This was very useful!
I didn't actually use your script in conjunction with cron, but it let me come up with my own backup scheme. Thanks
grim4593 is offline   Reply With Quote
Old December 24th, 2007   #3
altonbr
Fresh Brewed Ubuntu
 
altonbr's Avatar
 
Join Date: Mar 2006
Location: Canada
Beans: 1,216
Ubuntu 9.10 Karmic Koala
Re: HOWTO: Backup nightly via rsync

Quote:
Originally Posted by grim4593 View Post
This was very useful!
I didn't actually use your script in conjunction with cron, but it let me come up with my own backup scheme. Thanks
Great! I was worried that this tutorial might be to complex, when really it's only a couple steps.
__________________
Tutorials: rsync and ssh
Website: brettalton.com
Articles: FCM #20, FCM #21, FCM #22, FCM #23
altonbr is offline   Reply With Quote
Old December 31st, 2007   #4
madk
First Cup of Ubuntu
 
Join Date: Apr 2006
My beans are hidden!
Send a message via MSN to madk
Re: HOWTO: Backup nightly via rsync

I haven't tried this yet (just trying to figure out what's happening in your script) so I could be wrong, but don't you need the $BACKUPDIR variabe in your rsync-line?

It looks like you only define a destination and no source directory.
madk is offline   Reply With Quote
Old December 31st, 2007   #5
altonbr
Fresh Brewed Ubuntu
 
altonbr's Avatar
 
Join Date: Mar 2006
Location: Canada
Beans: 1,216
Ubuntu 9.10 Karmic Koala
Re: HOWTO: Backup nightly via rsync

Quote:
Originally Posted by madk View Post
I haven't tried this yet (just trying to figure out what's happening in your script) so I could be wrong, but don't you need the $BACKUPDIR variabe in your rsync-line?

It looks like you only define a destination and no source directory.
Brutal. You're correct. Thank you and my apologizes.
Code:
rsync -avrz --delete --delete-excluded --exclude-from=$EXCLUDEFILE --log-file=$LOGFILE --rsh="ssh -p $SSHPORT -i /home/$USER/.ssh/backup" $BACKUPDIR $USER@$IPADDRESS:/home/$USER/backup/
__________________
Tutorials: rsync and ssh
Website: brettalton.com
Articles: FCM #20, FCM #21, FCM #22, FCM #23
altonbr is offline   Reply With Quote
Old May 18th, 2008   #6
neil.the.blue
First Cup of Ubuntu
 
Join Date: Jan 2007
Beans: 6
Re: HOWTO: Backup nightly via rsync

The ssh-copy-id command is useful for copying the public key.

ssh-copy-id -i <path to key> user@host
neil.the.blue is offline   Reply With Quote
Old May 18th, 2008   #7
altonbr
Fresh Brewed Ubuntu
 
altonbr's Avatar
 
Join Date: Mar 2006
Location: Canada
Beans: 1,216
Ubuntu 9.10 Karmic Koala
Re: HOWTO: Backup nightly via rsync

Yeah, but I needed to do logic checks and create folders as well.

I'm planning on re-writing this soon actually as it seems a bit scattered... It could really be made into one script.
__________________
Tutorials: rsync and ssh
Website: brettalton.com
Articles: FCM #20, FCM #21, FCM #22, FCM #23

Last edited by altonbr; May 18th, 2008 at 06:10 PM..
altonbr is offline   Reply With Quote
Old June 10th, 2008   #8
Mister X
Gee! These Aren't Roasted!
 
Mister X's Avatar
 
Join Date: Aug 2007
Location: Olavarria, Argentina
Beans: 197
Ubuntu 8.10 Intrepid Ibex
Re: HOWTO: Backup nightly via rsync

Thank you, very usefull!!
Mister X is offline   Reply With Quote
Old June 28th, 2008   #9
Toky
5 Cups of Ubuntu
 
Join Date: Sep 2005
Location: Virginia
Beans: 23
Ubuntu 9.04 Jaunty Jackalope
Send a message via ICQ to Toky Send a message via AIM to Toky Send a message via MSN to Toky Send a message via Yahoo to Toky Send a message via Skype™ to Toky
Re: HOWTO: Backup nightly via rsync

This is a pretty good contribution, but WHY would you recommend a passphrase-less key??????

I would just create a proper key with passphrase and have a ssh-agent running. This would make it much much much secure/complete solution.

Passphrase-less Keys ARE THE DEVIL!!!

Of course I would do that for any communication across the cloud, if your backup is inside your house's LAN then its not that bad (although not the best practice).

Cheers!
__________________
Smile like a donut..
Toky is offline   Reply With Quote
Old June 28th, 2008   #10
eldragon
Extra Foam Sugar Free Ubuntu
 
eldragon's Avatar
 
Join Date: Aug 2006
Beans: 836
Re: HOWTO: Backup nightly via rsync

Quote:
Originally Posted by Toky View Post
This is a pretty good contribution, but WHY would you recommend a passphrase-less key??????

I would just create a proper key with passphrase and have a ssh-agent running. This would make it much much much secure/complete solution.

Passphrase-less Keys ARE THE DEVIL!!!

Of course I would do that for any communication across the cloud, if your backup is inside your house's LAN then its not that bad (although not the best practice).

Cheers!
if you need it to be automatic, you cannot have it requiring user input.

getting that off my chest. why on earth reinvent the wheel? there are dozen scripts already doing this. my favorite due to simplicity being rsnapshot.

good luck
eldragon is offline   Reply With Quote

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -4. The time now is 09:12 PM.


vBulletin ©2000 - 2010, Jelsoft Enterprises Ltd. Ubuntu Logo, Ubuntu and Canonical © Canonical Ltd. Tango Icons © Tango Desktop Project. lingonberry