Results 1 to 10 of 10

Thread: rsync exclude rules nightmare

  1. #1
    Join Date
    Jul 2007
    Beans
    341

    Arrow rsync exclude rules nightmare

    I'm trying to get rsync to only copy certain subfolders by name.

    This works:

    + A/*Good*/
    A/*

    But if I want to get sub-sub folders:

    This doesn't work:
    + A/**Good*/
    A/*

    What do I do?

  2. #2
    Join Date
    Feb 2009
    Location
    Dallas, TX
    Beans
    7,162
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: rsync exclude rules nightmare

    Hi u-slayer.

    One usually end the include/exclude rsync's rules with a '- *', or a '+ *' depending on what are trying to do.

    I would be helpful if you also post the actual rsync command you are using.

    For now I would recommend taking a look at post#3 of this thread, and the linked example here.

    Hope that helps, and tell us if you need more tips with rsync.
    Regards.

  3. #3
    Join Date
    Jul 2007
    Beans
    341

    Re: rsync exclude rules nightmare

    yeah adding that header/footer doesn't do anything

    and the command is rsync -van /input/ /output --delete-before --delete-excluded --exclude-from=file

    - *
    + mkv/**Good*/
    mkv/*
    + *

  4. #4
    Join Date
    Feb 2009
    Location
    Dallas, TX
    Beans
    7,162
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: rsync exclude rules nightmare

    I'm going to guess a little your file structure and the subdirectories you are trying to copy, but this will:
    (1) create the 'mkv' directory.
    (2) create all directories below 'mkv' matching *Good*.
    (3) copy all files and directories (the whole hierarchy) under those dirs created on step (2).

    Code:
    rsync -avn --delete-before --delete-excluded --exclude-from=file  /input/ /output
    where 'file' contains:
    Code:
    + mkv/
    + mkv/*Good*/***
    - *
    Hope it helps, and tell us how it goes.
    Regards.
    Last edited by papibe; March 9th, 2012 at 11:31 PM. Reason: spelling

  5. #5
    Join Date
    Jul 2007
    Beans
    341

    Re: rsync exclude rules nightmare

    That only matches sub folders 1 level in. It doesn't match sub-sub or sub-sub-sub folders.

  6. #6
    Join Date
    Feb 2009
    Location
    Dallas, TX
    Beans
    7,162
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: rsync exclude rules nightmare

    It works fine here, even several levels down. May be it is a version problem?

    I tested it on 11.04 using rsync version 3.0.7 protocol (version 30).

    Just to be clear. Im my example there has to be at least one directory that matches '*Good*' directly under mkv. Then, under that everything is copied. Is that what you are looking for?

    Regards.

  7. #7
    Join Date
    Jul 2007
    Beans
    341

    Re: rsync exclude rules nightmare

    Quote Originally Posted by papibe View Post
    Just to be clear. Im my example there has to be at least one directory that matches '*Good*' directly under mkv. Then, under that everything is copied. Is that what you are looking for?

    No. I want it to match a directory like this:

    A/B/C/D/Good

  8. #8
    Join Date
    Feb 2009
    Location
    Dallas, TX
    Beans
    7,162
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: rsync exclude rules nightmare

    Will the pattern '*Good*' match a file, or a directory?

    Regards.

  9. #9
    Join Date
    Jul 2007
    Beans
    341

    Re: rsync exclude rules nightmare

    Directory Only.

  10. #10
    Join Date
    Feb 2009
    Location
    Dallas, TX
    Beans
    7,162
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: rsync exclude rules nightmare

    Let me cover a couple of cases.

    If you know the directory structure (like in post 13), you can use this rules:
    Code:
    + A/
    + A/B/
    + A/B/C/
    + A/B/C/D/
    + A/B/C/D/Good/
    + A/B/C/D/Good/***
    - *
    The reason for the inclusion of every single directory in the path is because rsync search the directory structure and eliminates anything that is not on the rules. Thus:
    Code:
    + A/**/Good/***
    - *
    or even
    Code:
    + A/B/C/D/Good/***
    - *
    won't work because rsync eliminate the A/, A/B/, etc. from the search early on.

    In case you don't know the exact directory structure. You can't solve your problem using include/exclude rules. You need other tools to reinforce the rsync.

    For example:
    Code:
    find -type d -name '*Good*'
    will find the complete path of the directory you are looking for. Then to get all files and folders under it, you could do another find:
    Code:
    find -type d -name '*Good*' | xargs -I dir find dir
    Then you'll have all files and folders you want to copy.

    rsync has an option called '--files-from' that let's you get the list of file to transfer from a a file or from the standard input (use -). Then this would work:
    Code:
    cd /input/
    find -type d -name '*Good*' | xargs -I dir find dir | rsync -avn --files-from=- /input/ /output
    Note that I 'cd' to /input/ so the find can display the names relative to the source directory (as rsync needs it).

    Now just to extra safe (in case of spaces or other weird characters on the files names) I would separate the list of files using a null character:
    Code:
    cd /input/
    
    find -type d -name '*Good*' -print0 | 
      xargs -0 -I dir find dir -print0 | 
      rsync -0 -avn --files-from=- /input/ /output
    Hope that helps, and tell us how it goes.
    Regards.

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •