Page 1 of 2 12 LastLast
Results 1 to 10 of 11

Thread: Large transfers over the Internet stalling.

  1. #1
    Join Date
    Mar 2007
    Beans
    971

    Large transfers over the Internet stalling.

    I am attempting to create an offsite repository of the data on a number of servers in two office locations. The total amount of data is 1.5TB on 5 different computers 3-500GB on each machine. This is not a backup but more of an archive in case of emergency. Years ago our main computers were in an office in New Orleans after Katrina and we had to mount a "rescue mission" to retrieve them. That eventually worked out fine, however the computers might have been damaged so I've been working to create an automatic archive. I have one both offices data in each office and that has been the case for a while this is an additional layer of insurance. This is also in addition to the versioned backups we have on each site.

    I have set up a computer at a third site and have been working to replicate all of the data there. I chose to use rsync so I could run periodic crontabs to update the information and resume transfers if they failed. I believe that should be sufficient for our purposes, one of the offices has already been transferred without problems and the crontabs are keeping it up to date. However I've run into a problem with initial copies on the other office. The transfered archive keep stalling after what looks like a random amount of time. I can log into both ends at the same time and see what is going on. Neither computer is heavily loaded and the transfers run at a good rate but after a time simply stop sending data. There is no error message it just stops sending or receiving data. I can stop the interrupt the transfer and restart it but it eventually dies again. The office that has been transfered uses the same ISP as the archive site (Comcast) the office that is stalling uses ATT Fiber.

    Any ideas how I can determine what is causing the transfers to stop?

  2. #2
    Join Date
    Nov 2008
    Location
    Metro Boston
    Beans
    15,581
    Distro
    Kubuntu 20.04 Focal Fossa

    Re: Large transfers over the Internet stalling.

    Do you use the "-v" option in rsync to get a list of the transferred files? Does the failing transfer always hang at the same file, or does it happen randomly?
    If you ask for help, do not abandon your request. Please have the courtesy to check for responses and thank the people who helped you.

    Blog · Linode System Administration Guides · Android Apps for Ubuntu Users

  3. #3
    Join Date
    Oct 2005
    Location
    Lab, Slovakia
    Beans
    10,332

    Re: Large transfers over the Internet stalling.

    Rsync has a bandwidth parameter. Slowing it down a bit can prevent complete stalls in my experience.

    Also check your equipment ethernet ports for packet errors indicative of hardware failures or bad cables.

  4. #4
    Join Date
    Mar 2007
    Beans
    971

    Re: Large transfers over the Internet stalling.

    Quote Originally Posted by SeijiSensei View Post
    Do you use the "-v" option in rsync to get a list of the transferred files? Does the failing transfer always hang at the same file, or does it happen randomly?
    I use the -v parameter when I'm doing it from the command line but not in the crontabs. Both stall. When it stalls if I simply run the same command again it picks up with untransfered files.

  5. #5
    Join Date
    Mar 2007
    Beans
    971

    Re: Large transfers over the Internet stalling.

    Quote Originally Posted by HermanAB View Post
    Rsync has a bandwidth parameter. Slowing it down a bit can prevent complete stalls in my experience.

    Also check your equipment ethernet ports for packet errors indicative of hardware failures or bad cables.

    I'll try reducing the bandwidth, that may also prevent rsync from taking over the connections and slowing other traffic.

    I don't think it is the local network since I've been able to make large transfers from other locations. I have no indication that the location I'm having issues with is behaving badly. It actually has more bandwith than the local connection.
    Last edited by rsteinmetz70112; April 4th, 2021 at 04:35 PM.

  6. #6
    Join Date
    Oct 2005
    Location
    Lab, Slovakia
    Beans
    10,332

    Re: Large transfers over the Internet stalling.

    The programs ifconfig or ip can show you whether there are packet errors (bad HW).

  7. #7
    Join Date
    Mar 2007
    Beans
    971

    Re: Large transfers over the Internet stalling.

    I tried reducing the bandwidth and that didn't help, it just slowed down the transfers.

    I don't see a problem at this end ifconfig is showing zero errors , of course the problem could be upstream from my computer. It's possible the remote site has some issues. I can login in there and try to see.

  8. #8
    Join Date
    Mar 2007
    Beans
    971

    Re: Large transfers over the Internet stalling.

    I've now gotten a new error:

    Code:
    rsync -avz user@remote.domain.com:/files /backups/remote/
    ...
    packet_write_wait: Connection to XX.XX.XX.XX port 22: Broken pipe
    rsync: connection unexpectedly closed (12610026455 bytes received so far) [receiver]
    rsync error: error in rsync protocol data stream (code 12) at io.c(235) [receiver=3.1.2]
    rsync: connection unexpectedly closed (1381386 bytes received so far) [generator]
    rsync error: unexplained error (code 255) at io.c(235) [generator=3.1.2]

  9. #9
    Join Date
    Oct 2005
    Location
    Lab, Slovakia
    Beans
    10,332

    Re: Large transfers over the Internet stalling.

    It looks like packets are getting dropped somewhere along the line.

    You need to devise a script that will watch rsync and if necessary, kill it and restart it.
    Last edited by HermanAB; April 6th, 2021 at 08:26 PM.

  10. #10
    Join Date
    Mar 2007
    Beans
    971

    Re: Large transfers over the Internet stalling.

    Now I've got two problems.

    First the transfer is stalling, that is it stops transferring without any errors
    Second it errors out like the last post.

Page 1 of 2 12 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •