Page 2 of 2 FirstFirst 12
Results 11 to 18 of 18

Thread: rsync and progress in log files

  1. #11
    Join Date
    Feb 2008
    Location
    Chicago, IL
    Beans
    115
    Distro
    Lubuntu 14.04 Trusty Tahr

    Re: rsync and progress in log files

    Quote Originally Posted by SeijiSensei View Post
    It sounds like you're looking for a "progress meter" or the like which would say you've moved 53% of the files or give the number of bytes transferred and remaining. Rsync doesn't support this function as far as I know.

    As others have said, if you use "-v" you'll get a list of files as they are sent that you can track with "tail -f /var/log/rsync" or wherever you direct the output. The "--stats" parameter provides summary statistics but not a progress measure.
    It has the -P option, but it only shows while you are running the command at a terminal, not as a cron job and into the log file.

  2. #12
    Join Date
    Jan 2012
    Beans
    342

    Re: rsync and progress in log files

    Quote Originally Posted by SeijiSensei View Post
    It sounds like you're looking for a "progress meter" or the like which would say you've moved 53% of the files or give the number of bytes transferred and remaining. Rsync doesn't support this function as far as I know.
    Actually it does, '--progress', or the '-P' switch (I see now form the OP's edits that s/he was using '-P'). The problem is that because --progress is updating the screen there are control characters printed, so its not really designed for non-interactive shells (such as cron).

    The output of --progress could be redirected to a file but then the file woud contain control characters and would need to be parsed with sed 's/\r/\n/g' to remove these chars. So, obviously thats not going to work with 'tail -f'.

    So, really, what the OP wants is an interactive feature for a non-interactive shell, and that is somwhat of an ask. The closest they will get, I think, is via '--log-file='.

    best ... khay

  3. #13
    Join Date
    Feb 2008
    Location
    Chicago, IL
    Beans
    115
    Distro
    Lubuntu 14.04 Trusty Tahr

    Re: rsync and progress in log files

    Quote Originally Posted by Khayyam View Post
    Actually it does, '--progress', or the '-P' switch (I see now form the OP's edits that s/he was using '-P'). The problem is that because --progress is updating the screen there are control characters printed, so its not really designed for non-interactive shells (such as cron).

    The output of --progress could be redirected to a file but then the file woud contain control characters and would need to be parsed with sed 's/\r/\n/g' to remove these chars. So, obviously thats not going to work with 'tail -f'.

    So, really, what the OP wants is an interactive feature for a non-interactive shell, and that is somwhat of an ask. The closest they will get, I think, is via '--log-file='.

    best ... khay
    <shrugs> I figured I was asking too much, but I am not as knowledgeable as some folks around here. So I thought I would ask.

    Thanks for everyone's help.

  4. #14
    Join Date
    Feb 2009
    Location
    Dallas, TX
    Beans
    7,274
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: rsync and progress in log files

    Quote Originally Posted by diskmaster23 View Post
    Hmm.....any tips on the deattached virtual terminal? Will that allow me to see the progress of the already executed cron job?
    It would allow you to connect to the terminal where the command it is actually being executed.

    IMHO, this is very helpful when you are rsyncing a few big files, instead of several small to medium files. In this case the option --progress make more sense than -v.

    I personally use a combination of screen and rsync to share family photos and videos with my brother.

    Inside cron you need to create a detached screen:
    Code:
    screen -dmS NameOfScreen your_rsync_script.sh
    At any time you can check if there are screens actives by doing:
    Code:
    screen -ls
    If you see something like this:
    Code:
    There is a screen on:
    	3637.NameOfScreen	(02/12/2012 08:31:02 PM)	(Detached)
    1 Socket in /var/run/screen/S-user.
    it means that the script is still running inside the screen named 'NameOfScreen'. In order to attach (connect) to the virtual screen do this:
    Code:
    screen -r NameOfScreen
    To leave things running as they were, de attach from the screen by pressing Ctrl+a d (control a, and then d).

    Hope it helps.
    Regards.

  5. #15
    Join Date
    Feb 2008
    Location
    Chicago, IL
    Beans
    115
    Distro
    Lubuntu 14.04 Trusty Tahr

    Re: rsync and progress in log files

    Quote Originally Posted by papibe View Post
    It would allow you to connect to the terminal where the command it is actually being executed.

    IMHO, this is very helpful when you are rsyncing a few big files, instead of several small to medium files. In this case the option --progress make more sense than -v.

    I personally use a combination of screen and rsync to share family photos and videos with my brother.

    Inside cron you need to create a detached screen:
    Code:
    screen -dmS NameOfScreen your_rsync_script.sh
    At any time you can check if there are screens actives by doing:
    Code:
    screen -ls
    If you see something like this:
    Code:
    There is a screen on:
    	3637.NameOfScreen	(02/12/2012 08:31:02 PM)	(Detached)
    1 Socket in /var/run/screen/S-user.
    it means that the script is still running inside the screen named 'NameOfScreen'. In order to attach (connect) to the virtual screen do this:
    Code:
    screen -r NameOfScreen
    To leave things running as they were, de attach from the screen by pressing Ctrl+a d (control a, and then d).

    Hope it helps.
    Regards.
    I'll give it a try.

  6. #16
    Join Date
    Feb 2008
    Location
    Chicago, IL
    Beans
    115
    Distro
    Lubuntu 14.04 Trusty Tahr

    Re: rsync and progress in log files

    Quote Originally Posted by papibe View Post
    It would allow you to connect to the terminal where the command it is actually being executed.

    IMHO, this is very helpful when you are rsyncing a few big files, instead of several small to medium files. In this case the option --progress make more sense than -v.

    I personally use a combination of screen and rsync to share family photos and videos with my brother.

    Inside cron you need to create a detached screen:
    Code:
    screen -dmS NameOfScreen your_rsync_script.sh
    At any time you can check if there are screens actives by doing:
    Code:
    screen -ls
    If you see something like this:
    Code:
    There is a screen on:
    	3637.NameOfScreen	(02/12/2012 08:31:02 PM)	(Detached)
    1 Socket in /var/run/screen/S-user.
    it means that the script is still running inside the screen named 'NameOfScreen'. In order to attach (connect) to the virtual screen do this:
    Code:
    screen -r NameOfScreen
    To leave things running as they were, de attach from the screen by pressing Ctrl+a d (control a, and then d).

    Hope it helps.
    Regards.
    The script looks like this
    Code:
    #!/bin/bash
    cmd="rsync -avzhP --verbose --log-file=/data/server-rsync-log --owner=david --group=admin --chmod=a+rw,g+rw,g+rw,o-wx /data/server-remote/ /data/server-local/"
    eval $cmd # execute command
    The problem is, when I run
    Code:
    screen -dmS rsync -avzhP --verbose --log-file=/data/server-rsync-log --owner=david --group=admin --chmod=a+rw,g+rw,g+rw,o-wx /data/server-remote/ /data/server-local/
    I get this
    Code:
    Screen version 4.00.03jw4 (FAU) 2-May-06
    So I thought I had to run it like this
    Code:
    screen -dmS /usr/bin/ssh-server-rsync
    but when I do, I get this
    Code:
    Must run suid root for multiuser support.
    What am I doing wrong?

  7. #17
    Join Date
    Feb 2009
    Location
    Dallas, TX
    Beans
    7,274
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: rsync and progress in log files

    screen expects a name after the -S option, and then a command. Try this:
    Code:
    screen -dmS myscreen rsync -avzhP --verbose --log-file=/data/server-rsync-log --owner=david --group=admin --chmod=a+rw,g+rw,g+rw,o-wx /data/server-remote/ /data/server-local/
    Although, I would rather run it like this:
    Code:
    screen -dmS myscreen /path/to/myrsync,sh
    where myrsync.sh contains the rsync command:
    Code:
    #!/bin/bash
    rsync -avzhP --verbose --log-file=/data/server-rsync-log --owner=david --group=admin --chmod=a+rw,g+rw,g+rw,o-wx /data/server-remote/ /data/server-local/
    Let us know how it goes.
    Regards.

  8. #18
    Join Date
    Feb 2008
    Location
    Chicago, IL
    Beans
    115
    Distro
    Lubuntu 14.04 Trusty Tahr

    Re: rsync and progress in log files

    Quote Originally Posted by papibe View Post
    screen expects a name after the -S option, and then a command. Try this:
    Code:
    screen -dmS myscreen rsync -avzhP --verbose --log-file=/data/server-rsync-log --owner=david --group=admin --chmod=a+rw,g+rw,g+rw,o-wx /data/server-remote/ /data/server-local/
    Although, I would rather run it like this:
    Code:
    screen -dmS myscreen /path/to/myrsync,sh
    where myrsync.sh contains the rsync command:
    Code:
    #!/bin/bash
    rsync -avzhP --verbose --log-file=/data/server-rsync-log --owner=david --group=admin --chmod=a+rw,g+rw,g+rw,o-wx /data/server-remote/ /data/server-local/
    Let us know how it goes.
    Regards.
    I just wanted to let everyone know that it worked. Consider this thread solved.

Page 2 of 2 FirstFirst 12

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •