Page 1 of 7 123 ... LastLast
Results 1 to 10 of 67

Thread: Rsync - What's YOUR setup like?

  1. #1
    Join Date
    Jul 2006
    Beans
    4,659

    Rsync - What's YOUR setup like?

    Just a general question. I use rsync on my system because I have two pairs of drives I like to keep redundant. I have two 500gb drives (1 is mine, 1 is backup) and two 250gb drives for samba network storage where I keep backups of the windows machines on the network (1 main, 1 backup).

    I mount the drives by fstab when the system boots. The mount points when unmounted are owned by root, whereas they are owned by me when mounted. As a result, I decided to avoid using the -a switch which requires root access, that way when I would auto run these backups via crontab, if the drive happened to be down due to hardware failure, the lack of root access wouldn't force the files to go to the mount point.

    Because the trick is - If /storage drive dies, /media/storage still exists, and root permissions will push all of my data to that folder, which would then reside on my root partition, therefore definitely maxing it out.

    Anyway - What's your setup like? Is my setup in need of some work? Should I use the -a command to keep permissions on the files/folders, etc all the same?

    What do you think and what's your backup situation like?

  2. #2
    Join Date
    Apr 2009
    Beans
    1,173

    Re: Rsync - What's YOUR setup like?

    Quote Originally Posted by Roasted View Post
    What's your setup like?
    I use the remote rsync daemon on the backup box and I find that a) it keeps the process alive even on really bad wireless connections and b) it is a lot faster than mounting the BackupBox drives locally. One line of my backup script is:

    Code:
    rsync -i -avh --delete ~/Documents/ rsync://BackupBox.local:/Stuartn/Documents/ >> backup.log
    BackupBox is a big disk and a tiny processor running Linux. My username Stuartn is also configured as a module name in rsyncd.conf on BackupBox.

  3. #3
    Join Date
    Jul 2006
    Beans
    4,659

    Re: Rsync - What's YOUR setup like?

    See, I love Linux and all but that kind of command you use there confuses the crap out of me.

    Is there a GUI application that runs on rsync that does the same thing you're doing via command line?

  4. #4
    Join Date
    Apr 2009
    Beans
    1,173

    Re: Rsync - What's YOUR setup like?

    Quote Originally Posted by Roasted View Post
    Is there a GUI application that runs on rsync that does the same thing you're doing via command line?
    Why would you want a GUI for something that runs so well? Just save it in a command file with comments on your command-line options. If you must, then Grsync is a front-end for rsync.

    My rsync uses -i (itemize all changes in detail) -a (archive settings, which includes permissions etc) -v (verbose, tell you what its doing) -h (human-readable numbers, like 1 GB) and --delete to delete anything on the destination that has been deleted on the source. I also send the output to a text file (> backup.log) where I can see when and what I last backed up.

  5. #5
    Join Date
    Sep 2006
    Beans
    7,366
    Distro
    Lubuntu Development Release

    scripting

    Quote Originally Posted by Roasted View Post
    See, I love Linux and all but that kind of command you use there confuses the crap out of me.

    Is there a GUI application that runs on rsync that does the same thing you're doing via command line?
    Think of it as scripting, that's what you are really doing.

    Anyway, in most cases the GUI is just a crippled front-end for the regular program.

    And to answer the original question, here's an older rsync script:

    Code:
    rsync --exclude '*.AppleDouble*' --exclude '*~' --exclude 'gz.?' -tavv \
       -e "ssh -i /Users/me/.ssh/mykey" \
       /Users/me/Files/Projects/2B1202/ me@$HOST:/home/me/Desktop/Projects/2B1202/
    If bourne shell is to foreign, then try perl or python.

  6. #6
    Join Date
    Sep 2006
    Beans
    7,366
    Distro
    Lubuntu Development Release

    ps. Steve's Bourne / Bash shell scripting tutorial


  7. #7
    Join Date
    Jul 2006
    Beans
    4,659

    Re: Rsync - What's YOUR setup like?

    Going back to a side question I had listed above, do I kind of have a goofy setup for my rsyncing?

    Take this for example. /media/localbackup is a 2nd 500gb drive in my system which backs up data from my primary drives home directory. Okay, fine. But when I rsync, I don't run it as root and I don't run it with the -a command. So it'll sync everything, but not with permissions.

    My thinking was this. /media/localbackup is a mount point for the 500gb backup drive. That mount point is owned by me. If the drive is not mounted, it is owned by root. Therefore, if there's no root permissions, and the drive isn't mounted, the drive doesn't get synced. That's good.

    The problem comes into play when I DO decide to use root, because if I run my crontab entry as root, and my backup drive isn't mounted due to hardware failure or something, then /media/localbackup becomes a folder on my root partition of my primary drive.

    Have you guys ever tried to push 380gb of data into a 20gb spot? I can only imagine my system would be so slow it'd be nearly unusable, and if I'd reboot - unable to log in due to the space being maxed.

    Any advice in this scenario?

  8. #8
    Join Date
    Sep 2007
    Location
    Nashville, TN
    Beans
    182
    Distro
    Ubuntu 12.04 Precise Pangolin

    Re: Rsync - What's YOUR setup like?

    Quote Originally Posted by Roasted View Post
    See, I love Linux and all but that kind of command you use there confuses the crap out of me.

    Is there a GUI application that runs on rsync that does the same thing you're doing via command line?
    Yes it is called Grsync I believe that you can install from Synaptic.
    ASRock ION3D 152D Intel Atom D525 (1.8GHz, dual-core) NVIDIA GT218-ION 1 x HDMI 2GB/320GB, Ubuntu 12.04

  9. #9
    Join Date
    Mar 2009
    Beans
    927
    Distro
    Ubuntu 12.04 Precise Pangolin

    Post Complicated

    Mine's complicated, I found that rsync was taking ages to back up while the partition being backed up was in use. So I have a separate OS to take care of backups and general maintenance. I have a backup program made up of three files, here it is:

    [BASH] - /usr/local/bin/backup
    PHP Code:
    #!/bin/bash

    ##### CONSTANTS #####
    NAME="${0##*/}"
    DESCRIPTION=
    VERSION=0.2
    LOCATION
    ="$(readlink -f $0)"
    LOCATION="${LOCATION%$NAME}"
    EXCLUDE="${LOCATION}exclude-list"
    WRITE='/media/backup'  # WARNING: Directory specified will be overwritten
    READ='/media/system'  # WARNING: Directory specified may be overwritten
    BACKUP_NAME='Backup'

    ##### MAIN CODE #####
    if [ "$UID!= ]; then     # If you are not root
        
    exec sudo $0     # Ask for authentication
    fi
    if [ "$1" == '-a' ]; then     # If used with an -a flag: Mount partitions
        
    mount /dev/sda6 "$READ|| exit 1
        mount 
    /dev/sda5 "$READ/boot" || exit 1
        mount 
    /dev/sda7 "$READ/home" || exit 1
    fi
    if [ ! -"$WRITE]; then     # If backup folder does not exist
        
    echo >&"Please mount backup device at $WRITE and press [ENTER] to continue.
    Press [ESC] to do backup later."
    fi
    while [ ! -"$WRITE]; do
        if [ 
    "$(wait-keypress 
    )" 
    == '' ]; then  # Escape/Enter
            
    exit 0
        fi
    done
    echo 'The backup may take some time, please be patient...'
    rsync -acv --del "$READ/" "$WRITE/$BACKUP_NAME/" --exclude-from="$EXCLUDE"

    if [ "$?" == ]; then
        
    echo "Backup completed successfully!"
        
    exit 0
    else 
        echo 
    "Backup did not complete successfully."
        
    exit 1
    fi 

    /usr/local/bin/exclude-list
    Code:
    + /home/*/.themes
    + /home/*/.icons
    + /home/*/.backgrounds
    + /home/*/.wallpapers
    - /root/.gvfs
    - /home/*/.gvfs
    - /root/.local/share/Trash/
    - /home/*/.local/share/Trash/
    - lost+found/
    - /tmp/
    - /cdrom/
    - /media/
    - /mnt/
    - /proc/
    - /sys/

    [BASH] - /usr/local/bin/wait-keypress
    PHP Code:
    #!/bin/bash
    while (true); do
        
    stty cbreak -echo; key=$(dd bs=1 count=1 2>/dev/null); stty -cbreak echo
        if [ 
    "$@" == '' ]; then     # If enter key is pressed
            
    echo -"$key"
            
    exit 0
        elif 
    [[ "$@" =~ "$key]]; then     # If a trigger key is pressed
            
    echo -"$key"
            
    exit 0
        fi
    done 

  10. #10
    Join Date
    Mar 2009
    Beans
    927
    Distro
    Ubuntu 12.04 Precise Pangolin

    Re: Rsync - What's YOUR setup like?

    Quote Originally Posted by Roasted View Post
    Going back to a side question I had listed above, do I kind of have a goofy setup for my rsyncing?

    Take this for example. /media/localbackup is a 2nd 500gb drive in my system which backs up data from my primary drives home directory. Okay, fine. But when I rsync, I don't run it as root and I don't run it with the -a command. So it'll sync everything, but not with permissions.

    My thinking was this. /media/localbackup is a mount point for the 500gb backup drive. That mount point is owned by me. If the drive is not mounted, it is owned by root. Therefore, if there's no root permissions, and the drive isn't mounted, the drive doesn't get synced. That's good.

    The problem comes into play when I DO decide to use root, because if I run my crontab entry as root, and my backup drive isn't mounted due to hardware failure or something, then /media/localbackup becomes a folder on my root partition of my primary drive.

    Have you guys ever tried to push 380gb of data into a 20gb spot? I can only imagine my system would be so slow it'd be nearly unusable, and if I'd reboot - unable to log in due to the space being maxed.

    Any advice in this scenario?
    Yes, run it as root, and make sure the permissions get preserved, but do a check to make sure the drive is mounted. As for how you do that, I'm not totally sure: google it.

Page 1 of 7 123 ... LastLast

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •