How to backup folders over local network automatically using rsync and two computers

during the boot up of the second computer.

Motivation of this tutorial is to document the system I setup so that I do not need to re-find every piece of information later on. Hopefully someone else can also benefit from this document.

I welcome any input on how to improve the setup and feel free to post also other comments. I need to admit that this document has been in process for more than a month and there fore some errors are possible. Please comment if you notice that something is wrong.

For similar and simpler backup tutorial using ssh see
http://ubuntuforums.org/showthread.php?t=795668
For similar and simpler backup tutorial using separate disk
http://ubuntuforums.org/showthread.php?t=786410

Steps covered in this document

Make IP address of the computer you want to backup fixed using router DHCP reservation
Set up rsync daemon to the computer you want to backup
Setup rsync to the computer doing the backup
Setup the rsync to run when the computer starts up

1. Make IP address of the computer you want to backup fixed using DHCP reservation

First step is to make computer IP adresse(s) static using the router DHCP reservation. Some other method can be used if router does not support MAC address based DHCP reservation. http://ubuntuforums.org/showthread.php?t=1426270 describes one method. IP address needs to be fixed at least for the computer which is backed up.

DHCP reservation is set up as follows

1.1. Find out how you can login to your router. For my router I need to open web browser and on to the address field I write http://192.168.0.1.

Please note that this does not necessarily work with your router. There are other methods and some devices may require tampering with subnet settings etc. Make sure you understand how the login procedure works and if you need to change some settings on your computer make sure you know what you are changing and how to reverse those changes before making them.

1.2. Login to you router and find LAN or Local area network settings or DHCP settings. Name may vary a bit but there should be a section for these settings. You need to find a section where your computer name, IP address and your computer MAC address are listed. Copy the MAC address of your computer and current IP that is assigned by router to your computer, use paper or separate document

Shortly (and probably quite inaccuratelly) IP address is like a hotel room that is assigned to you when you visit a hotel. If you are lucky you may end up with same room from visit to visit but there is no guarantee. Eventually you will end up in a different room when the room you usually use is taken by someone else. MAC address is like credit card number of your network card and it will not change just like your credit card number does not change when you are not checked in to your favorite hotel unless you lose the card and you have to get a new one. With DHCP filtering we tell the reception desk of the hotel (i.e. the router) to reserve indefinitely one room from the hotel and that room will be given only to person with matching credit card (i.e. network card).
1.3. Find DHCP Reservation section. This section should have some form of table where you can write up your computer name, your current IP address and the MAC address of your computer. See attachment for one example of the reservation table.

2. Setup rsync-daemon to to the computer you want to backup

2.1. Open the computer you want to backup.

2.2. Follow instructions given in

https://help.ubuntu.com/community/rsync

in chapters installation and rsync daemon.

In my case I edited the
/etc/rsync.conf

Code:

max connections = 2 log file = /var/log/rsync.log timeout = 300 [share] comment = HomeFolder path = /home/MyHomefolderToBeBackedUp/ read only = no list = yes uid = MyUserid gid = sambashare auth users = MyUserid secrets file = /etc/rsyncd.secrets

Now I have changed the parameters

Comment, Path, uid, gid, auth users

to fit my needs so that comment parameter indicates that the [share] module is for my home folder, path variable points to my home folder and uid (user id) parameter is my user id, gid (group id) parameter is the user group sambashare. I admit that I am not 100% sure what these do and if you need all (probably not) but this works for me. I used the sambashare as group since I have samba installed and that particular group is available on both computers and home folders are shared through Samba.

3. Setup rsync to the computer doing the backup

3.1. Open the computer that will store the backup

3.2. Install rsync

I installed all of the packages mentioned in the rsync community documentation but just rsync should suffice. To install packages I normally use synaptic with GUI but following should work also

Open terminal and write:

Code:

sudo apt-get install rsync

3.3 Create exclusion list for the backup script

In order to avoid unnecessary errors and copying of unnecessary data the rsync needs and exclusion list which can be used to tell rsync what to copy and what not to copy. Otherwise copying generates a lot of errors since some of the hidden files in the home folder are not copyable except as root. Also there is no point to copy program specific settings.

Open terminal and type

Code:

gksudo gedit /etc/rsyncexclude

This will create a file named rsyncexclude to the /etc folder
For me the file contents are

Code:

.* *Crypt* *Trash* 'My GCompris*'

.* tells rsync not to copy hidden files and folders
*Crypt* tells rsync not to copy folders containing text string Crypt
*Trash* tells rsync not to copy folders containing text string Trash
'My GCompris*' tells rsync not to copy folders containing text string 'My GCompris'

3.4. Create a password file

in terminal type

gksudo gedit /etc/rsyncpasswd

to create a file named rsyncpasswd and in to the gedit write the password you defined in to the rsync.secrets file.

3.5. Make password file not world readable

In order for the rsync to accept the file as password file it must not be world readable.

I am not 100% sure what chmod number I used but

Code:

sudo chmod 600 /etc/rsyncpasswd

should do the trick. My rights list on terminal as follows -rw-r----- 1 root root 10 2010-03-02 12:22 rsyncpasswd

type

Code:

ls -la /etc/rsyncpasswd

in to the terminal to get the previous output. The user rights are the first 10 digits and when they are as shown then the rsync accepts the password file.

4. Setup rsync to run on when the computer starts up

This was the problematic part for me. In order to read the password file the rsync had to be run as root since only root can access the password file. On the other hand I did not want the rsync asking any input when it is run. I over came this by making a shell script to init.d

4.1 Create script file

open terminal and write

Code:

gksudo gedit /etc/init.d/myautobackupscript.sh

Script contents for me are:

Code:

#!/bin/sh rm '/home/myhomefolder/rsoperlog.txt' sleep 60 rsync -a --password-file='/etc/rsyncpasswd' --progress --log-file='/home/acer5020/rsoperlog.txt' --exclude-from='/etc/rsyncexclude' acer5020@192.168.0.101::share /home/acer5020/rsyncbackup/ exit(0)

Now what this I have intended with this:
First line indicates that the file is a script
Second line removes the old log file from home directory.
Third line tells the script to wait 60 seconds. I added this for reasons I tell later
Fourth and fifth line are the rsync command which is wrapped here so that it can be read.
Sixth line hopefully tells the script to exit when the script is ready.

Now note that I do not have the commonly used / presented parameter -delete -zvv on the rsync command. Reasons for this are as follows

I do not want the rsync to delete anything from the backup directory (-delete)
I do not want to compress the data before sending since 99.9% of the data is digital photos already compressed.
There is no point for verbose since the script is run in background.

This leaves me with the options

-a = create archive style backup I.e maintain many attributes of the files
- -password-file = path to password file
– progress = Indicate progress (now this is probably totally unnecessary and I just noted it)
--log = write a logfile to indicated folder and file name
--exclude-from = file to read exclude parameters from

4.2. Make script executable

I would add a cool code line here with sudo chmod u+x /etc/init.d/mybackupscript.sh command but since by surfing forums I find different variations of this command (u or a or none?) I shall describe the method I know works, but be warned no fast fingers and mistakes after you have started nautilus with sudo!.

open terminal
type
gksudo nautilus
select filesystem from places menu → select etc folder → select init.d folder → select your script → right click the script → properties

Right click the file and select tab permissions and mark the box by “allow executing this file as program”

Next add the script to startup, see instructions from this thread

http://ubuntuforums.org/showthread.p...ipt+at+startup

Some discussion

And at this point I need to point out that for the update-rc.d command the parameter defaults includes/replaces a lot configuration parameters that can be added to command and for a linux toddler/computer hobbyist like me the parameters are far from self explaining.

I would appreciate any input on how to make the script run as last script during the startup after login is ready and network is attached.

As previously noted on section 4.1 I added the sleep 60 parameter to give user time to login and for the wireless to connect. I do not know if that is necessary but I suspect that it does not do any damage. Better solution would be appreciated.

Final word
This method is not secure since transmitted data is not encrypted. Use only in local network.

Copyright
Above text of this post may be distributed under Creative Commons Attribution Share Alike license. Attribution must be made to Ubuntu forums.