Page 1 of 2 12 LastLast
Results 1 to 10 of 17

Thread: using rsync to create an exact mirror

  1. #1
    Join Date
    Mar 2005
    Beans
    98

    using rsync to create an exact mirror

    Could someone please give me an example of what rsync command needs to be issued to maintain an exact mirror of the source?

    I thought I used to just use rsync -a --delete source/ dest but it doesn't seem to be removing subdirectories (e.g., if I rename a folder, the new one gets copied, but the old one is kept at the back-up source as well). I've RTFM'd a few times and Googled all afternoon on this. I've tried all the --delete variations (--delete-after, etc.) as well as using trailing slashes and asterisks in various combinations. I also tried the graphical frontend but it too left subdirectories.

    I'd like the end result to be an exact duplicate of my home directory on an external USB drive, including subdirectories, symlinks, dot files, ownership and permissions.

  2. #2
    Join Date
    Nov 2009
    Location
    East Anglia
    Beans
    415
    Distro
    Xubuntu

    Re: using rsync to create an exact mirror

    Quote Originally Posted by pataphysician View Post
    Could someone please give me an example of what rsync command needs to be issued to maintain an exact mirror of the source?

    I thought I used to just use rsync -a --delete source/ dest but it doesn't seem to be removing subdirectories (e.g., if I rename a folder, the new one gets copied, but the old one is kept at the back-up source as well). I've RTFM'd a few times and Googled all afternoon on this. I've tried all the --delete variations (--delete-after, etc.) as well as using trailing slashes and asterisks in various combinations. I also tried the graphical frontend but it too left subdirectories.

    I'd like the end result to be an exact duplicate of my home directory on an external USB drive, including subdirectories, symlinks, dot files, ownership and permissions.
    rsync behaves exactly as you'd expect if you rsync between ext3 filesystems (--delete behaves as you want) but there may be problems if you are using ntfs (or other non-Linux filesystem). Try adding the option "--modify-window=1" if "dest" is ntfs.

    H

  3. #3
    Join Date
    Jun 2007
    Location
    K-o-T,
    Beans
    Hidden!
    Distro
    Ubuntu Development Release

    Re: using rsync to create an exact mirror

    I think luckybackup has that option to sync your /home directory.

    Good Luck
    Hypertension = Take with a pinch of salt
    | Psychocats | Ubuntu Documentation | Howto [Solved] | BBcode list | Brother Website |

  4. #4
    Join Date
    Jan 2008
    Beans
    307
    Distro
    Ubuntu

    Re: using rsync to create an exact mirror

    Please post your exeact command that you typed into the terminal to copy

  5. #5
    Join Date
    Mar 2005
    Beans
    98

    Re: using rsync to create an exact mirror

    Hi, thanks for the replies.

    Quote Originally Posted by baddnady23 View Post
    Please post your exeact command that you typed into the terminal to copy
    I tried about 30 different things, but it mostly got to the point where I was just throwing slashes and asterisks and random switches.

    I started with rsync -av --delete /home/mike /media/backup

    Also rsync -av --delete /home/mike/ /media/backup/mike

    Then rsync -av --delete /home/mike/* /media/backup/mike

    Then I retried all of those with the various --delete options (--delete-after, --delete-during, and so on). Then I tried them all with --recursive because I read a thread that said you need to specify it sometimes even though -a implies it.

    When none of those worked I re-read the manual and added the --force option to each of the commands above, so that my final three attempts looked like

    rsync -av --delete --recursive --force /home/mike /media/backup/

    rsync -av --delete --recursive --force /home/mike/ /media/backup/mike

    rsync -av --delete --recursive --force /home/mike/* /media/backup/mike

    The last one (with the asterisk) seems to work on sub-directories, but it skips a lot of files for some reason. I can't tell which or why, but if I do a dry run with and without it, there's a huge difference in what gets copied. It certainly ignores dot files from the source.

    Like I said, I swear I've done this before with the first, simple command. Both drives were probably ext3 then. The source drive here is jfs and the destination is ext4.

  6. #6
    Join Date
    Jan 2008
    Beans
    307
    Distro
    Ubuntu

    Re: using rsync to create an exact mirror

    Refer to my post at #10 here: http://ubuntuforums.org/showthread.php?t=238672

    Specifically, it states:
    be aware that rsync pays particular attention to the / at the end of a path. If it is there, it will copy only the contents of that directory into the desired location. If it is not there, it will copy the the actual directory and its contents. For example:
    Code:
    rsync -av /home/andrew/ andrew@192.168.1.70:/media/data/backups/ubuntu_home_backup/home_laptop
    copies the contents of my /home to the desired location. Whereas
    Code:
    rsync -av /home/andrew andrew@192.168.1.70:/media/data/backups/ubuntu_home_backup/home_laptop
    copies the actual /andrew folder and its contents to the desired location.
    Although I am copying to a directory on another computer, the principle is exactly the same.

    The actual command that I use everyday to backup my personal /home directory is:
    Code:
    rsync -av --delete --exclude='/.gvfs/' --log-file=/home/andrew/Logs/rsync/ubuntu\ 10.04/$(date +%Y%m%d)_rsync.log /home/andrew/ andrew@192.168.1.70:/media/data/backups/ubuntu_home_backup/home_desktop
    Take out all the extra crap and it simply is this:
    Code:
    rsync -av --delete /home/andrew/ andrew@192.168.1.70:/media/data/backups/ubuntu_home_backup/home_desktop
    Stick to something similar and you should have no problems. Please post back with any further questions!

  7. #7
    Join Date
    Mar 2005
    Beans
    98

    Re: using rsync to create an exact mirror

    Strange, that's what I tried but it just doesn't work over here.

    Code:
    cd /media/backup/mike/Desktop/
    mkdir delete_me
    cd delete_me
    touch bye.txt
    rsync -avn --delete /home/mike/ /media/backup/mike | grep delete_me
    [no output]
    rsync -avn --delete /home/mike/ /media/backup/mike | grep bye
    [no output]
    Nor does it work when I add --force. This is all done as user "mike," and everything has the same/correct permissions.

    However, if I cd up to /media/backup/mike:

    Code:
    mkdir go_away
    cd go_away
    touch dunno.txt
    rsync -avn --delete /home/mike/ /media/backup/mike | grep go_
    
    deleting go_away/dunno.txt
    deleting go_away/
    It's like it just refuses to delete subdirectories.

  8. #8
    Join Date
    Oct 2009
    Beans
    Hidden!

    Re: using rsync to create an exact mirror

    Well, it's looking to me like it might be something about the JFS --> Ext4 that could be similar to what anglican was referring to in post 2 about the time difference between Ext3 -->NTFS and problems with deletes when resyncing from one to the other.

    Like the others, I can verify that I mirror correctly a dir from Ext-->Ext


    Edit: -n ??? , you're still doing a dry run?
    Last edited by QLee; August 12th, 2010 at 06:38 PM. Reason: thought of a question

  9. #9
    Join Date
    Mar 2005
    Beans
    98

    Re: using rsync to create an exact mirror

    Quote Originally Posted by QLee View Post
    Well, it's looking to me like it might be something about the JFS --> Ext4 that could be similar to what anglican was referring to in post 2 about the time difference between Ext3 -->NTFS and problems with deletes when resyncing from one to the other.

    Like the others, I can verify that I mirror correctly a dir from Ext-->Ext


    Edit: -n ??? , you're still doing a dry run?
    Well, I ran two actual back-ups. The second run is when I noticed it wasn't keeping subdirectories in sync. After that I started using -n for testing various switches.

    Since I just got the drive and it's only got back-up data on it, maybe I'll just format to JFS and see what happens.

  10. #10
    Join Date
    Nov 2009
    Location
    East Anglia
    Beans
    415
    Distro
    Xubuntu

    Re: using rsync to create an exact mirror

    Quote Originally Posted by pataphysician View Post
    Well, I ran two actual back-ups. The second run is when I noticed it wasn't keeping subdirectories in sync. After that I started using -n for testing various switches.

    Since I just got the drive and it's only got back-up data on it, maybe I'll just format to JFS and see what happens.
    Have you tried the"--modify-window=1" switch (you can even try numbers larger than 1 - though that shouldn't be necessary)? Whilst I wouldn't expect this to be a problem with jfs (which is normally a very reliable file system) there may be some issue with times that is causing this. If you get the same problem backing up jfs->jfs then I guess it's time to submit an rsync bugreport

    H

Page 1 of 2 12 LastLast

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •