Page 3 of 4 FirstFirst 1234 LastLast
Results 21 to 30 of 31

Thread: HOWTO: Download Ubuntu ISOs from multiple mirrors at once

  1. #21
    Join Date
    Aug 2006
    Location
    Pompano Beach, FL, USA
    Beans
    57

    Re: HOWTO: Download Ubuntu ISOs from multiple mirrors at once

    Quote Originally Posted by smartbei View Post
    EDIT: I made my computer a temporary DeMilitarized Zone for testing and I see that rsync works now. Apparently I was still blocking a neccessary port.
    EXCELLENT, glad you got it working. rsync probably won't be as fast as the metalink download, but it's a good way to finish the download and have it verified, if the torrent was really slow.

    Once aria2 supports some of the advanced error correction features of metalink, this shouldn't be a problem. or aria2 could support rsync so this step could be automatic, but that might be harder to add.

  2. #22
    Join Date
    Aug 2006
    Location
    Pompano Beach, FL, USA
    Beans
    57

    Re: HOWTO: Download Ubuntu ISOs from multiple mirrors at once


  3. #23
    Join Date
    Aug 2006
    Location
    Pompano Beach, FL, USA
    Beans
    57

    Smile Re: HOWTO: Download Ubuntu ISOs from multiple mirrors at once

    aria2 now supports repairing corrupted downloads with metalinks that contain chunk checksums - which http://download.packages.ro/metalink/ubuntu/ and elsewhere include.

    metalink support (ftp/http, file selection, whole file verification) has been added to DownThemAll! 1.0 Beta, a Firefox Add-on at http://www.downthemall.net/main/inst...emall-10-beta/

    Here's what metalinks listing multiple files look like in DTA...

    & KGet for KDE 4.0 has also added metalink support to SVN at
    http://websvn.kde.org/trunk/KDE/kdenetwork/kget/

  4. #24
    Join Date
    Aug 2006
    Location
    Pompano Beach, FL, USA
    Beans
    57

    Smile Re: HOWTO: Download Ubuntu ISOs from multiple mirrors at once

    metalinks for all the recent Ubuntu releases can be found at http://www.metalinker.org/samples/ubuntu/

  5. #25
    Join Date
    Aug 2006
    Location
    Pompano Beach, FL, USA
    Beans
    57

    Re: HOWTO: Download Ubuntu ISOs from multiple mirrors at once

    KGet 2.0, part of KDE 4.0, now supports metalinks.

  6. #26
    Join Date
    Sep 2007
    Location
    Owen Sound, ON
    Beans
    75
    Distro
    Ubuntu 11.10 Oneiric Ocelot

    Re: HOWTO: Download Ubuntu ISOs from multiple mirrors at once

    the raphink site appears to be gone, and the current apt-get install aria2 fetches and installs version 0.11.0, which is quite old, and it doesn't work in Gutsy Ubuntu 7.10

    is there some trick to making this happen? I also tried downloading the source code, installing the dev libs for ares and it STILL just sits there until it times out even with a simple http:// url! A search on the forums here didn't find any reports, so is it possible there's just something wrong with my overall ubuntu config? It's a pretty straight-out-of-the-box install of 7.06 that was upgraded to 7.10, nothing fancy. is Aria2 trying to do something funky with ports and maybe hits my ISP's firewall? I don't see any need for that for a simple HTTP transfer, but I tried

    $ aria2c http://ubuntu.media.mit.edu/ubuntu-r...sktop-i386.iso http://ftp.belnet.be/mirror/ubuntu.c...sktop-i386.iso http://mirror.csclub.uwaterloo.ca/ub...sktop-i386.iso

    and all I get are timeouts, whether it's one, two or three urls in that list

  7. #27
    Join Date
    Sep 2007
    Location
    Owen Sound, ON
    Beans
    75
    Distro
    Ubuntu 11.10 Oneiric Ocelot

    Re: HOWTO: Download Ubuntu ISOs from multiple mirrors at once

    btw, it is illogical to assume that downloading from three servers would deny other people from downloading from those servers: With 3 URLs, you are tying up each server for only 1/3 of the time, and in practical results, most often transfers run into snags and the bitrate will vary, whereas the multiple-server download will interleave the gaps so you can disconnect from the servers even faster. Since HTTP is stateless, unlike FTP, you do not consume a connection to any one server for the full length of time, only for the period of the current fetch, and if the software is smart it will do range-requests anyway; if you use multi-URLs with aria2 (if it works) each of your connections is relatively short lived and thus isn't it friendlier to the server and to other users to fetch even multiple connections to the same server?

  8. #28
    Join Date
    Aug 2006
    Location
    Pompano Beach, FL, USA
    Beans
    57

    Re: HOWTO: Download Ubuntu ISOs from multiple mirrors at once

    Quote Originally Posted by teledyn View Post
    the raphink site appears to be gone, and the current apt-get install aria2 fetches and installs version 0.11.0, which is quite old, and it doesn't work in Gutsy Ubuntu 7.10

    is there some trick to making this happen? I also tried downloading the source code, installing the dev libs for ares and it STILL just sits there until it times out even with a simple http:// url! A search on the forums here didn't find any reports, so is it possible there's just something wrong with my overall ubuntu config? It's a pretty straight-out-of-the-box install of 7.06 that was upgraded to 7.10, nothing fancy. is Aria2 trying to do something funky with ports and maybe hits my ISP's firewall? I don't see any need for that for a simple HTTP transfer, but I tried

    $ aria2c http://ubuntu.media.mit.edu/ubuntu-r...sktop-i386.iso http://ftp.belnet.be/mirror/ubuntu.c...sktop-i386.iso http://mirror.csclub.uwaterloo.ca/ub...sktop-i386.iso

    and all I get are timeouts, whether it's one, two or three urls in that list
    I suppose you could test the same URL w/ curl or wget. if they have problems, then it's your config. if they don't, its aria2.

    aria2 won't use weird ports on just http downloads.

    you also might want to try the "-l logfile" option to save a log, that might shed some light on your problem. you could post it here or report the bug on SF for aria2. what about your system? is it x86_64?

    let us know what you find.

  9. #29
    Join Date
    Aug 2006
    Location
    Pompano Beach, FL, USA
    Beans
    57

    Re: HOWTO: Download Ubuntu ISOs from multiple mirrors at once

    Quote Originally Posted by teledyn View Post
    btw, it is illogical to assume that downloading from three servers would deny other people from downloading from those servers: With 3 URLs, you are tying up each server for only 1/3 of the time, and in practical results, most often transfers run into snags and the bitrate will vary, whereas the multiple-server download will interleave the gaps so you can disconnect from the servers even faster. Since HTTP is stateless, unlike FTP, you do not consume a connection to any one server for the full length of time, only for the period of the current fetch, and if the software is smart it will do range-requests anyway; if you use multi-URLs with aria2 (if it works) each of your connections is relatively short lived and thus isn't it friendlier to the server and to other users to fetch even multiple connections to the same server?
    I've seen one server admin complain, but most don't seem to mind.

    using metalinks, you can make downloads more efficient by using local mirrors & giving servers priority. a dynamic mirror "observer" like Bouncer or mirror manager could create up to date metalinks depending on server load & do some really cool stuff.

  10. #30
    Join Date
    Aug 2006
    Location
    Pompano Beach, FL, USA
    Beans
    57

    Re: HOWTO: Download Ubuntu ISOs from multiple mirrors at once

    DownThemAll 1.0 final is out, with very nice metalink support.

Page 3 of 4 FirstFirst 1234 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •