Page 1 of 2 12 LastLast
Results 1 to 10 of 14

Thread: Syncing two external hard drives

  1. #1
    Join Date
    Dec 2010
    Beans
    82

    Syncing two external hard drives

    Dear all,

    I have an external hard drive that contains some 600 GB of files and folders. I use this external drive frequently and so the files and folders in it change on a daily basis. I want to back up this drive on another external drive. What is the best way to sync these two external hard drives on a daily basis?

    I have been trying to sync them through the Grsync software. But I think either I am not choosing the right options or else Grsync is not the best/right software for my purpose because the second hard drive does not ever become completely identical to the first one. What am I doing wrong? Should I go with another software? If so, are there suggestions for a good one? Or am I doing something not right?

    When I run Grsync, I choose the first external hd as my source and the second one as my destination. Then below that I check "Preserve time," "Preserve permissions," "Preserve owner," and "Preserve group." Below that, I also check "Verbose" and "Show transfer progress." Other options are all unchecked. Should I reconfigure these options?

    FYI, I frequently rename, edit, modify, or else completely delete files and folders in the primary hard drive. Hence, my need to back it up everyday so that the change would be replicated in the second hard drive.

    Appreciatively,

    R

  2. #2
    Join Date
    Jun 2006
    Location
    Antarctica
    Beans
    500
    Distro
    Kubuntu 12.04 Precise Pangolin

    Re: Syncing two external hard drives

    Use rsync on the command line (makes it easy to script).
    Always start with -n (fake run) before doing a real run.
    For instance I back up my data like you with:
    Code:
    rsync -xavHl --delete /InternalDisk /media/ExternalDisk/
    Warning, the trailing / has some importance.

  3. #3
    Join Date
    Mar 2010
    Location
    ρθΦ=000
    Beans
    Hidden!
    Distro
    Ubuntu 10.04 Lucid Lynx

    Re: Syncing two external hard drives

    Aye, rsync is the way to go

    Setup your command and test it ensure it works the way you want it too (maybe something like)

    Code:
    rsync -ruv --delete /source /destination
    then add a crontab entry to run your command (or bash script) everyday (or whenever you want to run it). See my crontab link in my signature for more info, or just post back if you have questions.

  4. #4
    Join Date
    Dec 2010
    Beans
    82

    Re: Syncing two external hard drives

    You know the problem is that I don't know even basic things about rsync and I am generally not all that great when it comes to working off the Terminal. Where can I learn some basic command lines for rsync? How can I download rsync by the way?

    Thanks so much!

  5. #5
    Join Date
    Mar 2010
    Location
    ρθΦ=000
    Beans
    Hidden!
    Distro
    Ubuntu 10.04 Lucid Lynx

    Re: Syncing two external hard drives

    You already have rsync and all the commands on your system. Click the link "linuxcommands" in my sig for a nice walkthrough of some of them.

    You can also start reading up here

    Finally (and most importantly), man will/should be your best friend

    Code:
    man commandname
    gives you the manual for commandname, so the rsync manual (man page) is accessed by typing

    Code:
    man rsync

  6. #6
    Join Date
    Dec 2010
    Beans
    82

    Re: Syncing two external hard drives

    Quote Originally Posted by dargaud View Post
    Use rsync on the command line (makes it easy to script).
    Always start with -n (fake run) before doing a real run.
    For instance I back up my data like you with:
    Code:
    rsync -xavHl --delete /InternalDisk /media/ExternalDisk/
    Warning, the trailing / has some importance.

    Thanks much for the tip. What does

    -xavHl
    mean in your command line?

    Also does this mean this command line mean that the files and folders in the destination will be deleted (since you have "delete" in the command line) and all the material from the source will be copied to the destination drive once again? If so, this is not what I want.

    I want an incremental backup meaning that I want to backup all the files that have *changed* since the last backup. I don't want to delete the entire destination drive and recopy everything from the source. That would take just too much time everyday.

  7. #7
    Join Date
    Dec 2010
    Beans
    82

    Re: Syncing two external hard drives

    Quote Originally Posted by seawolf167 View Post
    Aye, rsync is the way to go

    Setup your command and test it ensure it works the way you want it too (maybe something like)

    Code:
    rsync -ruv --delete /source /destination
    then add a crontab entry to run your command (or bash script) everyday (or whenever you want to run it). See my crontab link in my signature for more info, or just post back if you have questions.
    Many thanks for the links and the advice. I am beginning to read the material you suggested. I wonder why your command line differs from the person who suggested I should type in:

    rsync -xavHl --delete /InternalDisk /media/ExternalDisk/
    What does -ruv mean in your command line?

  8. #8
    Join Date
    Mar 2010
    Location
    ρθΦ=000
    Beans
    Hidden!
    Distro
    Ubuntu 10.04 Lucid Lynx

    Re: Syncing two external hard drives

    Its all in the man pages

    But:

    -r = recursive (copy all directories, contents and their subdirectories)
    -u = update (if a file/folder exists, do not overwrite unless newer)
    -v = be verbose (so when testing in a terminal you can see what is being moved/etc)
    --delete = delete files/folders from /destination that are not on /source (so /destination is an exact mirror of /source)

    As said previously, you can add the -n switch to preform a "dry-run", and if you have the -v switch on, you can redirect the output to a file so you can see exactly what would be copied/deleted/etc.

    Code:
    rsync -ruvn --delete /source /destination > rsync-dry-run-file-list
    gedit rsync-dry-run-file-list

  9. #9
    Join Date
    Feb 2011
    Location
    Maryland
    Beans
    2,259
    Distro
    Ubuntu

    Re: Syncing two external hard drives

    Open a terminal

    rsync --help

  10. #10
    Join Date
    Jun 2006
    Location
    Antarctica
    Beans
    500
    Distro
    Kubuntu 12.04 Precise Pangolin

    Re: Syncing two external hard drives

    When you use rsync, there are so many options that you need to consider the corner cases:
    - what should it do if files have changed both on the source and the destination (does not apply in the case of a backup)
    - what should it do if source files are deleted or moved
    - what should it do if the source files belong to various users
    - what should it do if there are hard/soft links
    - what should it do if the source spread over several mounted filesystems
    - what should it do if some permissions block access to source files
    - what should it do if there are special files (devices, pipes...)
    - should it transfer 'useless files' (backups, CVS, SVN...)
    - what should it do with strange characters in filenames

    Other than that, it's pretty simple to use...

Page 1 of 2 12 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •