Results 1 to 6 of 6

Thread: Improving rsync on moved files

  1. #1
    Join Date
    Jun 2006
    Location
    Antarctica
    Beans
    500
    Distro
    Kubuntu 12.04 Precise Pangolin

    Improving rsync on moved files

    Hello all,
    ok, this is not a question relevant to ubuntu, if only to backups.
    I've long been using rsync -av --delete /home /backup to make my backups and it works great. But I was wondering if there are options that allow rsync to work better on moved files.

    Let me explain: if I move or rename large files in /home, then the command will delete their previous backups and copy them again to their new location. That's time consuming. There are other backup tools that can guess that they are the same files based on various info (date+size or, more securely but slower, hash number). Are there options in rsync that can do that ?

    Note: I'm not really looking for other backup tool suggestions. I use rsync on Linux and robocopy on Windows so that I can script things.

  2. #2
    Join Date
    Apr 2008
    Location
    Columbia, SC
    Beans
    282

    Re: Improving rsync on moved files

    Nope.

    You can always do a preliminary move command yourself, if you've done something REALLY giant and obnoxious on the other end. For example, if you renamed /client/machine/reallybigdir to /client/machine/20GBofCrap one day, you might want to go the backups and mv /backups/reallybigdir /backups/20GBofCrap before your rsync process runs.

    I've considered using inotify to set up a process to "replay" rename operations from a client onto the backup before doing scheduled rsync operations, for exactly this reason... but that's only an option if inotify is available on the client - ie, if you're backing up a Linux machine, not a Windows machine.

  3. #3
    Join Date
    Mar 2010
    Location
    Los Angeles, CA
    Beans
    231
    Distro
    Ubuntu 10.04 Lucid Lynx

    Re: Improving rsync on moved files

    Doing a replay is time consuming enough that I'd probably just replicate a filesystem in near-realtime.
    Postfix problems? Come find me in #postfix on the freenode IRC network.

  4. #4
    Join Date
    Apr 2008
    Location
    Columbia, SC
    Beans
    282

    Re: Improving rsync on moved files

    Quote Originally Posted by KB1JWQ View Post
    Doing a replay is time consuming enough that I'd probably just replicate a filesystem in near-realtime.
    Uh? Replaying move operations isn't time-consuming at all. That's the whole point.

    Whereas if you DON'T replay a move operation on the client on the backup, you can end up with a single directory rename (which a replay would accomplish instantaneously and with only a few bytes of traffic moved over any network inbetween client and backup) requiring tens or hundreds of gigabytes of disk writes and traffic.

  5. #5
    Join Date
    Mar 2010
    Location
    Los Angeles, CA
    Beans
    231
    Distro
    Ubuntu 10.04 Lucid Lynx

    Re: Improving rsync on moved files

    I misspoke, my apologies. Trying to do too many things at once today.

    Doing a replay is COMPLEX enough that I'd rather find an alternate solution. Coming from a Mac world, my solution was this: http://blog.interlinked.org/tutorial...e_machine.html

    Doesn't solve the "I renamed 20 gigs of stuff to another folder" style problems, but my environment rarely sees such things happen.

    I'm from the school of "backups should be fire and forget."
    Postfix problems? Come find me in #postfix on the freenode IRC network.

  6. #6
    Join Date
    Jun 2006
    Location
    Antarctica
    Beans
    500
    Distro
    Kubuntu 12.04 Precise Pangolin

    Re: Improving rsync on moved files

    I'm from the school of "backups should be fire and forget.
    When your computer goes on fire, you figure out that you forgot to do your backup ?

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •