using rsync to create an exact mirror

[pataphysician]

Could someone please give me an example of what rsync command needs to be issued to maintain an exact mirror of the source?

I thought I used to just use rsync -a --delete source/ dest but it doesn't seem to be removing subdirectories (e.g., if I rename a folder, the new one gets copied, but the old one is kept at the back-up source as well). I've RTFM'd a few times and Googled all afternoon on this. I've tried all the --delete variations (--delete-after, etc.) as well as using trailing slashes and asterisks in various combinations. I also tried the graphical frontend but it too left subdirectories.

I'd like the end result to be an exact duplicate of my home directory on an external USB drive, including subdirectories, symlinks, dot files, ownership and permissions.

[anglican]

Could someone please give me an example of what rsync command needs to be issued to maintain an exact mirror of the source?

I thought I used to just use rsync -a --delete source/ dest but it doesn't seem to be removing subdirectories (e.g., if I rename a folder, the new one gets copied, but the old one is kept at the back-up source as well). I've RTFM'd a few times and Googled all afternoon on this. I've tried all the --delete variations (--delete-after, etc.) as well as using trailing slashes and asterisks in various combinations. I also tried the graphical frontend but it too left subdirectories.

I'd like the end result to be an exact duplicate of my home directory on an external USB drive, including subdirectories, symlinks, dot files, ownership and permissions.

rsync behaves exactly as you'd expect if you rsync between ext3 filesystems (--delete behaves as you want) but there may be problems if you are using ntfs (or other non-Linux filesystem). Try adding the option "--modify-window=1" if "dest" is ntfs.

H

[plucky]

I think luckybackup has that option to sync your /home directory.

Good Luck

[baddnady23]

Please post your exeact command that you typed into the terminal to copy

[pataphysician]

Hi, thanks for the replies.

Please post your exeact command that you typed into the terminal to copy

I tried about 30 different things, but it mostly got to the point where I was just throwing slashes and asterisks and random switches.

I started with rsync -av --delete /home/mike /media/backup

Also rsync -av --delete /home/mike/ /media/backup/mike

Then rsync -av --delete /home/mike/* /media/backup/mike

Then I retried all of those with the various --delete options (--delete-after, --delete-during, and so on). Then I tried them all with --recursive because I read a thread that said you need to specify it sometimes even though -a implies it.

When none of those worked I re-read the manual and added the --force option to each of the commands above, so that my final three attempts looked like

rsync -av --delete --recursive --force /home/mike /media/backup/

rsync -av --delete --recursive --force /home/mike/ /media/backup/mike

rsync -av --delete --recursive --force /home/mike/* /media/backup/mike

The last one (with the asterisk) seems to work on sub-directories, but it skips a lot of files for some reason. I can't tell which or why, but if I do a dry run with and without it, there's a huge difference in what gets copied. It certainly ignores dot files from the source.

Like I said, I swear I've done this before with the first, simple command. Both drives were probably ext3 then. The source drive here is jfs and the destination is ext4.

[baddnady23]

Refer to my post at #10 here: http://ubuntuforums.org/showthread.php?t=238672

Specifically, it states:

be aware that rsync pays particular attention to the / at the end of a path. If it is there, it will copy only the contents of that directory into the desired location. If it is not there, it will copy the the actual directory and its contents. For example:

rsync -av /home/andrew/ andrew@192.168.1.70:/media/data/backups/ubuntu_home_backup/home_laptop

copies the contents of my /home to the desired location. Whereas

rsync -av /home/andrew andrew@192.168.1.70:/media/data/backups/ubuntu_home_backup/home_laptop

copies the actual /andrew folder and its contents to the desired location.

Although I am copying to a directory on another computer, the principle is exactly the same.

The actual command that I use everyday to backup my personal /home directory is:

rsync -av --delete --exclude='/.gvfs/' --log-file=/home/andrew/Logs/rsync/ubuntu\ 10.04/$(date +%Y%m%d)_rsync.log /home/andrew/ andrew@192.168.1.70:/media/data/backups/ubuntu_home_backup/home_desktop

Take out all the extra crap and it simply is this:

rsync -av --delete /home/andrew/ andrew@192.168.1.70:/media/data/backups/ubuntu_home_backup/home_desktop

Stick to something similar and you should have no problems. Please post back with any further questions!

[pataphysician]

Strange, that's what I tried but it just doesn't work over here.

cd /media/backup/mike/Desktop/
mkdir delete_me
cd delete_me
touch bye.txt
rsync -avn --delete /home/mike/ /media/backup/mike | grep delete_me
[no output]
rsync -avn --delete /home/mike/ /media/backup/mike | grep bye
[no output]

Nor does it work when I add --force. This is all done as user "mike," and everything has the same/correct permissions.

However, if I cd up to /media/backup/mike:

mkdir go_away
cd go_away
touch dunno.txt
rsync -avn --delete /home/mike/ /media/backup/mike | grep go_

deleting go_away/dunno.txt
deleting go_away/

It's like it just refuses to delete subdirectories.

[QLee]

Well, it's looking to me like it might be something about the JFS --> Ext4 that could be similar to what anglican was referring to in post 2 about the time difference between Ext3 -->NTFS and problems with deletes when resyncing from one to the other.

Like the others, I can verify that I mirror correctly a dir from Ext-->Ext


Edit: -n ??? , you're still doing a dry run?

[pataphysician]

Well, it's looking to me like it might be something about the JFS --> Ext4 that could be similar to what anglican was referring to in post 2 about the time difference between Ext3 -->NTFS and problems with deletes when resyncing from one to the other.

Like the others, I can verify that I mirror correctly a dir from Ext-->Ext


Edit: -n ??? , you're still doing a dry run?

Well, I ran two actual back-ups. The second run is when I noticed it wasn't keeping subdirectories in sync. After that I started using -n for testing various switches.

Since I just got the drive and it's only got back-up data on it, maybe I'll just format to JFS and see what happens.

[anglican]

Well, I ran two actual back-ups. The second run is when I noticed it wasn't keeping subdirectories in sync. After that I started using -n for testing various switches.

Since I just got the drive and it's only got back-up data on it, maybe I'll just format to JFS and see what happens.

Have you tried the"--modify-window=1" switch (you can even try numbers larger than 1 - though that shouldn't be necessary)? Whilst I wouldn't expect this to be a problem with jfs (which is normally a very reliable file system) there may be some issue with times that is causing this. If you get the same problem backing up jfs->jfs then I guess it's time to submit an rsync bugreport

H