Results 1 to 6 of 6

Thread: Multiple destinations with rsync

  1. #1
    Join Date
    Dec 2011
    Beans
    7

    Multiple destinations with rsync

    Background
    I've just recently installed Ubuntu 11.10 and I thought the first thing I'd do was figure out how to back up everything. I'll be using an external hard drive for the backup.

    It seems as if using tar is an easy way to make non-incremental backups and using rsync is an equally easy way to make incremental backups. Seeing as a full backup might take some time to make, I was thinking of backing up with rsync once a day and making full backups a little less often.

    After having tinkered a little, I came to realize that it took two and a half minutes to synchronize my SSD with the external hard drive with rsync, while it took half a minute less to make a tar archive of the same files and move the archive to the external hard drive! And this was after having run rsync once already. In other words, making an incremental backup with virtually no changes made took longer than making a full backup!

    Not giving up on rsync quite yet, I tried using it on the same files but with a destination on the SSD instead of on the external hard drive, and this took less than a second after having run rsync the first time.

    Question
    I'm wondering if there's any way to specify multiple destinations for rsync so that it uses the first destination to check which files to update and updates the files on all destinations. If so, I'd be able to keep a "backup" on the SSD that I'd use to speed the incremental backup to the external hard drive up.


    Now that I think of it, it doesn't seem like a very efficient solution to essentially have a copy of each file on the SSD ... suggestions on a better way to make incremental backups are welcome!

  2. #2
    Join Date
    Jun 2007
    Location
    Arizona U.S.A.
    Beans
    2,136

    Re: Multiple destinations with rsync

    Is the external hard drive formatted with a FAT file system? If so, that is why your 'incremental' backup is slow.

    You need to use the option

    Code:
    --modify-window=1
    to get fast incremental backups. Example:

    Code:
    rsync -rt --modify-window=1 <source> <destination>
    I has to do with accuracy of the time stamps. The rsync man page explains why:

    When comparing two timestamps, rsync treats the timestamps as being equal if they differ by no more than the modify-window value. This is normally 0 (for an exact match), but you may find it useful to set this to a larger value in some situations. In particular, when transferring to or from an MS Windows FAT filesystem (which represents times with a 2-second resolution), --modify-window=1 is useful (allowing times to differ by up to 1 second).
    So, files in you source directory can get copied again even if they are really unchanged from the last sync because the time stamps are slightly different at the destination.

  3. #3
    Join Date
    Dec 2011
    Beans
    7

    Re: Multiple destinations with rsync

    The external hard drive is formatted with an NTFS file system.

    The precise command I'm using is

    Code:
    sudo rsync -az / --exclude=/backup --exclude=/cdrom --exclude=/dev/*  --exclude=/lost+found/*
    --exclude=/media/* --exclude=/mnt/* --exclude=/proc/* --exclude=/sys/* --exclude=/tmp/*
    --exclude=/var/cache/apt/* /media/LaCie/ubuntu
    For some reason, it seems to take only about one minute to run the command now; earlier it took two minutes every time I ran it ...!

    I tried using the --modify-window option but it doesn't seem to make any difference, regardless of window size.
    I also tried using only the r and t flags and that takes roughly half a minute, if that's of any importance.

    When I time the commands, I typically get results like those below (with -rt and -az respectively)

    Code:
    real    0m32.481s
    user    0m2.232s
    sys    0m10.261s
    
    real    1m0.692s
    user    0m2.240s
    sys    0m17.789s

  4. #4
    Join Date
    Jun 2007
    Location
    Arizona U.S.A.
    Beans
    2,136

    Re: Multiple destinations with rsync

    Well, the man page only mentions FAT, so no information whether NTFS behaves the same. I don't know.

    FYI:

    I only back up folders under home to an FAT external drive, so only ordinary files are involved.

    Using:

    Code:
    rsync -rti --modify-window=1 <source dirs> <destination>
    -i option gives a list of changes.

    You can also use the --stats option to get info on what and how much is transferred.

    Good Luck.

  5. #5
    Join Date
    Jun 2009
    Location
    Chicago Suburbs
    Beans
    Hidden!
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: Multiple destinations with rsync

    You do know when you back up Linux partitions to a Windows formatted partition, you lose permissions & ownership. You then cannot easily recover system files. Data files can be easily reset to your owner & typical permissions.

    The tar backup can be copied and recovered since it is compressed and will maintain permissions & ownership.

    I like rsync so I can easily see a file and do incremental backups & I find from Linux to Linux partition to be very quick.

    discussion of alternatives/strategy:
    https://help.ubuntu.com/community/BackupYourSystem
    For info on UEFI boot install & repair - Updated Mar 2015:
    http://ubuntuforums.org/showthread.php?t=2147295
    Please use Thread Tools above first post to close thread when/if answered completely.







  6. #6
    Join Date
    Dec 2011
    Beans
    7

    Re: Multiple destinations with rsync

    All right, I've got it working properly now!

    When I ran rsync with the i flag, it'd say that all files were being updated, so I decided to format the drive with the ext4 system. After having run the rsync command once, each subsequent run was lightning fast (less than a second)!

    And seeing as ext4 is not really a Windows format, I guess I won't have any problem with lost permissions and ownerships when I try to restore things either.


    Thanks a lot for the help and input, Dennis and Fred!

    I'll mark this as solved even though the original "question" wasn't really answered, as the actual issue I was having has been solved.

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •