Results 1 to 10 of 10

Thread: Can't run rsync bash script with cron

  1. #1
    Join Date
    May 2008
    Beans
    18

    Can't run rsync bash script with cron

    I wrote the following backup script:

    Code:
    #!/bin/bash
    rsync -rtpogv -progress --delete --rsh='ssh -p 1338' bertwagner@xx.xx.xx.xxx:/Users/bertwagner/Pictures/ /home/bert/backup/Pictures/
    I saved this file as 'homebackup', placed it in /usr/local/bin, gave it execute permissions. If I run 'homebackup' in terminal it executes the script like I want it to and there are no problems.

    I tried setting up the following cron job using 'sudo crontab -e':
    Code:
    * * * * * /usr/local/bin/homebackup
    (I have it running every minute just for testing purposes now)
    Although the script worked fine when I ran it from terminal, it for some reason does not work with cron. I added a line to the script 'homebackup' both before and after the rsync command in the form of:
    Code:
    echo "TEST $(date)" >> /home/bert/backup/backup.log
    in order to check if cron was actually working.

    I came to the conclusion that cron is working because every minute 'homebackup' is writing to the log file both before and after the rsync command in the script, however the rsync command is not being executed. Can anyone help me figure out what I'm doing wrong? Thanks!

  2. #2
    Join Date
    Aug 2007
    Beans
    874
    Distro
    Ubuntu 11.10 Oneiric Ocelot

    Re: Can't run rsync bash script with cron

    Quote Originally Posted by bertwagner View Post
    I wrote the following backup script:

    Code:
    #!/bin/bash
    rsync -rtpogv -progress --delete --rsh='ssh -p 1338' bertwagner@xx.xx.xx.xxx:/Users/bertwagner/Pictures/ /home/bert/backup/Pictures/
    So, it appears that you are connecting via ssh, have you checked so that it does not get stuck on the ssh authentication? I don't see you specifying your ssh key file with the -i flag.

    Btw, why don't you use the -a flag instead of the -rtpog flag?

  3. #3
    Join Date
    May 2008
    Location
    United Kingdom
    Beans
    4,265
    Distro
    Ubuntu 12.04 Precise Pangolin

    Re: Can't run rsync bash script with cron

    Also, you can get the program to redirect the output -- including any error messages -- to a file. Try adding it to the end of the command, like this:
    Code:
    rsync -rtpogv -progress --delete --rsh='ssh -p 1338' bertwagner@xx.xx.xx.xxx:/Users/bertwagner/Pictures/ /home/bert/backup/Pictures/ >>/home/bert/backup/backup.log 2>&1
    That way, you'll see if there are any errors (such as the SSH that XCan mentioned).
    Problems with WINE?
    Full Circle Magazine :: Confused "allot"? :: Cheap Linux stickers
    In my day, we had outdoors in which to run, play, and socialise. Now we have computers to do those.

  4. #4
    Join Date
    May 2008
    Beans
    18

    Re: Can't run rsync bash script with cron

    Thank you for your help so far XCan and Paddy Landau. Using your suggestions I've made some progress figuring out what's going on but the script still isn't working like it should. My new rsync command is as follows:

    Code:
    rsync -avvv -progress --delete --rsh='ssh -p 1338 -i /home/bert/.ssh/id_dsa' bertwagner@xx.xx.xx.xxx:/Users/bertwagner/Pictures/ /home/bert/backup/Pictures/ >>/home/bert/backup/backup.log 2>&1
    Running this command in terminal works fine; my SSH key works and I am able to backup across the network without any manual input from me. However, when I put this command in my script and run it via cron, the following error occurs:

    Code:
    opening connection using: ssh -p 1338 -i /home/bert/.ssh/id_dsa -l bertwagner xx.xx.xx.xxx rsync --server --sender -vvvogtpre.iLs . /Users/bertwagner/Pictures/
    Host key verification failed.
    rsync: connection unexpectedly closed (0 bytes received so far) [receiver]
    rsync error: unexplained error (code 255) at io.c(600) [receiver=3.0.5]
    _exit_cleanup(code=12, file=io.c, line=600): about to call exit(255)
    I interpret this as meaning there is a problem with authentication, but I'm stumped because the command works fine in terminal on its own and only creates errors when ran from within the script. Any additional help is appreciated.

  5. #5
    Join Date
    May 2008
    Location
    United Kingdom
    Beans
    4,265
    Distro
    Ubuntu 12.04 Precise Pangolin

    Re: Can't run rsync bash script with cron

    When you run rsync from the terminal, does it prompt you for a password? Where is the SSH command getting its password from?

    EDIT: Doh! Sorry, it's getting the password from /home/bert/.ssh/id_dsa

    Does cron have permission to access that file, I wonder?
    Problems with WINE?
    Full Circle Magazine :: Confused "allot"? :: Cheap Linux stickers
    In my day, we had outdoors in which to run, play, and socialise. Now we have computers to do those.

  6. #6
    Join Date
    May 2008
    Beans
    18

    Re: Can't run rsync bash script with cron

    Quote Originally Posted by Paddy Landau View Post
    When you run rsync from the terminal, does it prompt you for a password? Where is the SSH command getting its password from?

    EDIT: Doh! Sorry, it's getting the password from /home/bert/.ssh/id_dsa

    Does cron have permission to access that file, I wonder?

    I switched from using sudo crontab to just using my user's crontab. I set 777 permissions to the id_dsa file and this is the error I get now:
    Code:
    opening connection using: ssh -p 1338 -i /home/bert/.ssh/id_dsa -l bertwagner xx.xx.xx.xxx rsync --server --sender -vvvogtpre.iLs . /Users/bertwagner/Pictures/ 
    @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
    
    @         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
    
    @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
    
    Permissions 0777 for '/home/bert/.ssh/id_dsa' are too open.
    
    It is recommended that your private key files are NOT accessible by others.
    
    This private key will be ignored.
    
    bad permissions: ignore key: /home/bert/.ssh/id_dsa
    
    Permission denied, please try again.
    
    Permission denied, please try again.
    
    Received disconnect from xx.xx.xx.xxx: 2: Too many authentication failures for bertwagner
    
    rsync: connection unexpectedly closed (0 bytes received so far) [receiver]
    rsync error: unexplained error (code 255) at io.c(600) [receiver=3.0.5]
    _exit_cleanup(code=12, file=io.c, line=600): about to call exit(255)
    If I play around with permissions I can't seem to get any combination that will allow the command to run - I either give id_dsa too much access and receive the error above or I restrict access to only the user and receive:
    Code:
    opening connection using: ssh -p 1338 -i /home/bert/.ssh/id_dsa -l bertwagner xx.xx.xx.xxx rsync --server --sender -vvvogtpre.iLs . /Users/bertwagner/Pictures/ 
    Permission denied, please try again.
    
    Permission denied, please try again.
    
    Received disconnect from xx.xx.xx.xxx: 2: Too many authentication failures for bertwagner
    
    rsync: connection unexpectedly closed (0 bytes received so far) [receiver]
    rsync error: unexplained error (code 255) at io.c(600) [receiver=3.0.5]
    _exit_cleanup(code=12, file=io.c, line=600): about to call exit(255)

  7. #7
    Join Date
    May 2008
    Location
    United Kingdom
    Beans
    4,265
    Distro
    Ubuntu 12.04 Precise Pangolin

    Re: Can't run rsync bash script with cron

    I'm no expert in this.

    Permissions for my .ssh are drwx------. The permissions for files within it are -rw-r--r--.

    Try setting those permissions and try again.

    If that still doesn't work (and it probably won't), perhaps some more knowledgeable person can tell you how to get your cron job to run with the right permissions.
    Problems with WINE?
    Full Circle Magazine :: Confused "allot"? :: Cheap Linux stickers
    In my day, we had outdoors in which to run, play, and socialise. Now we have computers to do those.

  8. #8
    Join Date
    May 2008
    Location
    Albany, NY
    Beans
    147
    Distro
    Ubuntu 10.04 Lucid Lynx

    Re: Can't run rsync bash script with cron

    I am struggling with this right now too. I have my rsync set up through rsnapshot and keychain. Upon researching the rsync 255 error, I found this:

    It looks like SSH auth is failing as you thought.

    Bash will not use your '.bash_profile' for non-interactive shells by
    default. There are easy ways around this, but I would suggest that you
    rather embed the commands into your script or use a helper like
    'keychain' and then source it's file from your script.

    Read the 'INVOCATION' section of bash(1) for more details.

    Okay...a little more research and I will know for sure in about 1/2 hour if it really works but: in rsnapshot.conf under cmd_preexec instead of writing the source /root/.keychain/someone@somewhere.com-sh right there. You need to point to a shell script with that command written in it.
    I did a shell script called keychain.sh
    #!/bin/bash
    source /root/.keychain/someone@somewhere.com-sh

    and under cmd_preexec /usr/local/bin/keychain.sh

    That didn't work you need to follow the instructions here:
    # ~/.bashrc: executed by bash(1) for non-login shells.
    # see /usr/share/doc/bash/examples/startup-files (in the package bash-doc)
    # for examples

    Does Anyone Know how to invoke bash(1) for non interactive shell scripting?
    Last edited by neilevan814; September 28th, 2009 at 07:06 AM.
    When life hands you lemons...make a sour face



  9. #9
    Join Date
    May 2008
    Beans
    19

    Re: Can't run rsync bash script with cron

    I'm getting the same problem. using 9.10 32 bit. googling came up with the "Read the 'INVOCATION' section of bash(1) for more details" stuff, and all i could find that helped was to set the BASH_ENV variable in my crontab. which didnt help...


    anyone?

    ------

    me@babydweezil:~$ crontab -l
    BASH_ENV=/home/me/.bashrc
    # m h dom mon dow command
    */2 * * * * /home/me/watchcopy.sh
    me@babydweezil:~$


    watchcopy.sh:
    #!/bin/sh
    rsync -ave ssh --delete --include "*.filetype1" --include "*.filetype2" --exclude "*.*" --log-file=/home/me/watchrsync.log /media/1tb/downloads/chromedownloads/ user@externalbox:/data/me/watch


    watchrsync.log:
    2009/11/15 22:36:04 [6155] rsync: connection unexpectedly closed (0 bytes received so far) [sender]
    2009/11/15 22:36:04 [6155] rsync error: unexplained error (code 255) at io.c(600) [sender=3.0.6]
    2009/11/15 22:38:04 [6166] rsync: connection unexpectedly closed (0 bytes received so far) [sender]
    2009/11/15 22:38:04 [6166] rsync error: unexplained error (code 255) at io.c(600) [sender=3.0.6]
    and so on..... (repeated every 2 mins)

  10. #10
    Join Date
    Oct 2009
    Beans
    7
    Distro
    Ubuntu

    Re: Can't run rsync bash script with cron

    Hi, this is probably because you are using ssh agent (having protected your ssh key with a password). You may want to make sure you set the environment variable SSH_AUTH_LOCK when starting the script via cron. For me this solution works fine (thanks to the author):

    http://www.codealpha.net/163/cron-ss...phrase-ubuntu/

    Set it via cron ...

    Code:
    # m h  dom mon dow   command
    00 04 * * *     SSH_AUTH_SOCK=$(find /tmp/keyring*/ -perm 0755 -type s -user matthias -group matthias -name 'socket.ssh' | head -1) /home/matthias/bin/backup.sh
    ... or within your backup script ...

    Code:
    #!/bin/sh
    
    export SSH_AUTH_SOCK=$(find /tmp/keyring*/ -perm 0755 -type s -user matthias -group matthias -name 'socket.ssh' | head -1)
    
    rsync -av --delete-after --log-file=/home/matthias/.rsync.log \
            ~/dir1 \
            ~/dir2 \
            ~/dir3 \
            matthias@somewhere.com:backup/

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •