pkgsrc-Users archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
FETCH_USING won't drive aria2 in parallel in pkgsrc
I like and use aria2 (aria2c) as a download tool. In my opinion it has
smarter defaults and a superior feature list vis-a-vis wget, curl, or fetch.
The chief feature I like about aria2 is that it can use multiple download
sites as sources for a single file download. It also seems to intelligently
adapt when some of the sites are faster than others. The result is that I
can often speed up downloads by a factor of 3-4 times when using aria2. I
max out it's 'splitting' options (-x 16 -s 16). This usually results in
faster downloads even when I've only got one source for the file (because of
the bandwidth-delay product issue being mitigated by multiple streams).
Right now, I have a script that parses MASTER_SITES out of the Makefile in a
given pkgsrc package directory. It can then feed the information back to
aria2 in a similar fashion to "make fetch". However, this is hard. There are
a lot of weird exceptions to deal with.
I know pkgsrc has a FETCH_USING parameter that allows you to override it's
default file-downloader. However, I don't believe this will take advantage
of aria2's parallel capabilities. It won't allow it to branch out to all
the master_sites at once since it'll only get passed one site at a time as
they fail or time out.
So, my real question is this: is there any reasonable way to use aria2 with
pkgsrc and take advantage of the parallelism?
-Swift
Home |
Main Index |
Thread Index |
Old Index