Page 2 of 2 FirstFirst 12
Results 11 to 14 of 14

Thread: rsync won't transfer some files

  1. #11
    Join Date
    Oct 2006
    Beans
    87

    Re: rsync won't transfer some files

    been studying mail servers myself all day. Most use hard links ONLY WHILE UP, and do this instead of mv commands to reloacte mail between folders as it is processed.

    As in get a mail,
    save a mail,
    execute a process on a mail that would end with it being moved,
    hard link the mail in new location,
    unlink the original mail.

    These steps are used to avoid data loss and duplication during processing since with a copy command their are two isolated copies in existence for a brief moment, and if a usewr checks their mail at that moment, it gets moved again. Now there are three copies, and only one will be deleted by the interrupted process.

    So,
    If your not running rsync while the server is in operation, no hard link references exist.

    Part of the mail protocol (don't remember which mail protocol though)

  2. #12
    Join Date
    Nov 2006
    Location
    Belgium
    Beans
    3,023
    Distro
    Ubuntu 10.04 Lucid Lynx

    Re: rsync won't transfer some files

    Quote Originally Posted by jhuhmann View Post
    Is using ./* redundant? Does it mean something different than . does?
    couple of things

    1-
    with *, the shell will expand that to subdirs of and files in /, before rsync sees the options, so it differs from "." or "./".
    A trailing slash in the src is also relevant to rsync, so "." and "./" are not identical.
    Since they're fifferent commands, they might cause a different behaviour, although I don't know if that could account for 18000 files missing

    You probably want ./


    2-
    are you sure your diff is accuratie ?

    3-
    rsync has specific default behaviour and options for symlinks and extra mounted filesystems
    If the "current dir" you work with has mount points for 'other file systems' such as extra disks or partitions, these defaults will come into play.
    Don't remember the details, but it's worth looking into.

    4-
    +1 to Herman's suggestion to run verbose and see what files get skipped (and what rsync complains about).
    You 'll probably need to grep [Ee]rror or something if you're dealing wat that many files/.



    -----
    Edit
    I might be wrong about 3- in that the default behaviour would be the scan files on extra filesystems and you'd need to specify an option to restrict rsync to 'one filesystem".
    Still, it's somthing to be aware of.
    Last edited by koenn; October 10th, 2011 at 09:57 PM.

  3. #13
    Join Date
    Apr 2009
    Beans
    9

    Re: rsync won't transfer some files

    Thank you for all the replies. I ran it with the full path for the source directory doing away with ./* and using /storage/backup/sync/zimbra/ and it correctly transferred everything.

    Consider this solved!

  4. #14
    Join Date
    Apr 2009
    Beans
    9

    Re: rsync won't transfer some files

    Quote Originally Posted by jazzon View Post
    been studying mail servers myself all day. Most use hard links ONLY WHILE UP, and do this instead of mv commands to reloacte mail between folders as it is processed.

    As in get a mail,
    save a mail,
    execute a process on a mail that would end with it being moved,
    hard link the mail in new location,
    unlink the original mail.

    These steps are used to avoid data loss and duplication during processing since with a copy command their are two isolated copies in existence for a brief moment, and if a usewr checks their mail at that moment, it gets moved again. Now there are three copies, and only one will be deleted by the interrupted process.

    So,
    If your not running rsync while the server is in operation, no hard link references exist.

    Part of the mail protocol (don't remember which mail protocol though)
    This makes complete sense. The scripts I was looking at are doing the initial backup while the server is active and then stopping it and rsyncing the changes. I probably just need rsync -a.

    Thanks!

Page 2 of 2 FirstFirst 12

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •