Results 1 to 8 of 8

Thread: How to speed up rsync?

  1. #1
    Join Date
    Dec 2008
    Location
    Taxes
    Beans
    458
    Distro
    Ubuntu Studio 12.04 Precise Pangolin

    How to speed up rsync?

    I am copying about 12TB of daya from one server to another using rsync via ssh. At best I can get 9.25MiB over the link according to bmon on both sides of the link.
    I issue the rsync command using nice -15 and that's what got me to 9.25 up from about 8.5 before.

    I'm pushing this over a bonded pair of 1GB NICs on both servers and I can get other applicaitons to push data at much higher rates - 40 to 60 MiB depending on where on the source the file sits - I suspect most of the time I'm hitting an upper limit on the ability of the source server to read from a software RAID5 array on 5900RPM SATA disks on an eSATA link. The system load on both sides is neglible.

    So, at this point, I have been copying data for about 3 days and I'm just over a 25% through.

    Would I gain throughput by breaking the round-robin bond and going to a direct connect crossover between two NICs and use jumbo frames? Or is rsync via ssh just that big of drag on the transfer that it won't help?
    I'm almost at the point of just trying to do NFS over a crossover with jumbo frames -- I have done that before on other hardware (completely differnet everything though) and been able to reach close to saturation point on a 1GB NIC.

    What I'm doing is working, but the wait time is killing me... Any suggestions to make this go faster?

  2. #2
    Join Date
    Dec 2010
    Beans
    573
    Distro
    Ubuntu 12.04 Precise Pangolin

    Re: How to speed up rsync?

    For the initial copy of large amounts of data it is best to do it via NFS, the overhead is much less and you can increase the number of NFS threads on both systems to max out the systems disk and network IO.

  3. #3
    Join Date
    Oct 2009
    Beans
    Hidden!
    Distro
    Ubuntu 22.04 Jammy Jellyfish

    Re: How to speed up rsync?

    The slowness comes from the encryption. If you have that much data you can either reduce the level of encryption used or use something else like NFS for the initial sync.

    See here:
    http://serverfault.com/questions/377...-rsync-so-slow
    Come to #ubuntuforums! We have cookies! | Basic Ubuntu Security Guide

    Tomorrow's an illusion and yesterday's a dream, today is a solution...

  4. #4
    Join Date
    Aug 2009
    Location
    Makati City, Philippines
    Beans
    2,269
    Distro
    Xubuntu 16.04 Xenial Xerus

    Re: How to speed up rsync?

    1GB NIC with only 9.5MB/sec. Quite slow as I can reach 50MB and up from network transfer.
    Anyway, some pointer from me:
    All all the swicthes that the data passes from server to server all have gigabit interfaces?
    Also, the transfer rate of "a lot of files with small file sizes" is significantly slower than "a few files with large file sizes".
    The encryption really is a bottleneck.
    Also see this for other alternatives in transferring files over the network. http://intermediatesql.com/linux/scr...g-pigz-and-nc/

  5. #5
    Join Date
    Jun 2006
    Location
    Brisbane Australia
    Beans
    713

    Re: How to speed up rsync?

    Quote Originally Posted by MakOwner View Post
    I am copying about 12TB of daya from one server to another using rsync via ssh.
    Perhaps try rsyncing directly via rsync protocol (i.e. rsync to/from rsync://) rather than over ssh?

  6. #6
    Join Date
    Jul 2013
    Beans
    45

    Re: How to speed up rsync?

    Have you tried using ftp to transfer data?

  7. #7
    Join Date
    Dec 2008
    Location
    Taxes
    Beans
    458
    Distro
    Ubuntu Studio 12.04 Precise Pangolin

    Re: How to speed up rsync?

    I just went ahead and moved to NFS and was able to push 40 ~ 60 MB/s across the link and synced up that way for the restore.

    So now that I have my data back where I want it, I want to set up a more automated backup process than I had before.

    I am looking at rsnapshot and just striaight rsyncd.

    I tried rsyncd first, and it looks pretty straight forward - I have the rsyncd daemon responding to a remote query at the very least.
    I can' find any CLI examples for backup TO the rsyncd daemon - only commands to read data from the rsyncd daemon.

    I'm using this: http://nwlinux.com/how-to-configure-...-rsync-server/ for rsyncd setup and the example given is this:

    Code:
    A working rsync example within Terminal would be
    
    rsync -avh rsync://bbc.nwlinux.com/bbc /home/mark/Desktop/
    
    The above statement would copy the BBC share of bbc.nwlinux.com to your Desktop.

    I want to push data from the client TO the rsyncd server. Have I misunderstood the setup? The rsyncd should be running on the system to be backed up perhaps?

  8. #8
    Join Date
    Dec 2008
    Location
    Taxes
    Beans
    458
    Distro
    Ubuntu Studio 12.04 Precise Pangolin

    Re: How to speed up rsync?

    I just went ahead and moved to NFS and was able to push 40 ~ 60 MB/s across the link and synced up that way for the restore.

    So now that I have my data back where I want it, I want to set up a more automated backup process than I had before.

    I am looking at rsnapshot and just striaight rsyncd.

    I tried rsyncd first, and it looks pretty straight forward - I have the rsyncd daemon responding to a remote query at the very least.
    I can' find any CLI examples for backup TO the rsyncd daemon - only commands to read data from the rsyncd daemon.

    I'm using this: http://nwlinux.com/how-to-configure-...-rsync-server/ for rsyncd setup and the example given is this:

    Code:
    A working rsync example within Terminal would be
    
    rsync -avh rsync://bbc.nwlinux.com/bbc /home/mark/Desktop/
    
    The above statement would copy the BBC share of bbc.nwlinux.com to your Desktop.

    I want to push data from the client TO the rsyncd server. Have I misunderstood the setup? The rsyncd should be running on the system to be backed up perhaps?



    OK, got that figured out -- just in case anyone else is looking -- here is a good config and command examples
    [edit] oops -- I meant this:[/edit]
    http://www.yolinux.com/TUTORIALS/Rsync.html

    Not that this https://rsync.samba.org/examples.html wasn't good, I just used the other examples...





    This is a minimal rsyncd.conf file that will let you push data to the rsync daemon with minimal security:
    Code:
    max connections = 5
    log file = /var/log/rsync.log
    hosts allow = 192.168.20.7
    hosts deny = *
    list = true
    uid = root
    gid = root
    timeout = 60
    read only = false
    
    [media]
    comment = Media Server Backup Path
    path = media/md1/rsyncd_test

    The command(s) I used:
    Code:
     rsync -avh /home/test/ media2::media
    and

    Code:
    rsync -avh /home/test/ rsync://media2/media

    I didn't notice any perceptible performance differences in the syntax, anyone know if there are functional differences that may show up over multiple GBs of data?
    Last edited by MakOwner; August 21st, 2013 at 08:43 PM.

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •