Results 1 to 2 of 2

Thread: rsync help

  1. #1
    Join Date
    Jun 2009
    Beans
    25

    rsync help

    Does anyone know of a way to avoid copying files with rsync if a parent folder is added:

    Say we have:

    /Archives/file.reallybigfile
    /Backup/


    Then after
    Code:
    rsync -a /Archives/ /Backup/
    we would have:
    /Archives/file.reallybigfile
    /Backup/file.reallybigfile


    Now say you
    Code:
    mkdir /Archives/newdir && mv /Archives/file* /Archives/newdir/.
    Then you:
    Code:
    rsync -a /Archives/ /Backup/
    You'd end up with two copies of file.reallybigfile in /Backup

    Code:
    find /Backup
    :
    /Backup
    /Backup/file.reallybigfile
    /Backup/newdir/file.reallybigfile




    Any suggestions on how to get rsync to mv the file instead of copying it?

    Say file.reallybigfile was a 1TB file, it's not so fun to transfer across a network, but if it were just moved on the dest to it's new spot, then we could avoid heavy transfers.

  2. #2
    Join Date
    Feb 2009
    Location
    Dallas, TX
    Beans
    7,735
    Distro
    Ubuntu 16.04 Xenial Xerus

    Re: rsync help

    Quote Originally Posted by jbowen7 View Post
    Say file.reallybigfile was a 1TB file, it's not so fun to transfer across a network, ...
    I would recommend using the option --partial to move big files over the network. rsync will keep all that is being transfer. In case the the transfer is interrupted, rsync won't start from the begining but it will use the temp files to recover much faster.

    If you want to track the progress of what is being transfer, the option --progress is more useful for big files than the regular -v.

    (You can save space by using -P that equals --partial and --progress).

    Now regarding your main concern.

    There are a few options you can add to avoid the transfer, however the success varies (the internal rsync comparison method is limited and not very documented).

    The option --fuzzy will try to do just what you are asking for: look for similar files (similar size, date and filename) so that the transfer can be avoided, or at least speed-ed up.

    This is not a perfect solution since rsync looks for similar files just in the same directory, so if you moved the file to other directory, you are out of luck (maybe it would work if moved to a subdirectory?).

    Please note that, although the --delete option suggested by zackwasa, will effectively remove the duplicates, it does not work well with the --fuzzy option. In order to the fuzzy option to work, use the option --delete-after option.

    I hope this helps,
    Regards.
    Last edited by papibe; September 26th, 2011 at 09:13 AM. Reason: spelling

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •