[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
Re: pkg/47424 (pkgsrc "make fetch" fails to fetch fotoxx-13.01.tar.gz for graphics/fotoxx)
The following reply was made to PR pkg/47424; it has been noted by GNATS.
From: Robert Elz <kre%munnari.OZ.AU@localhost>
Subject: Re: pkg/47424 (pkgsrc "make fetch" fails to fetch fotoxx-13.01.tar.gz
Date: Fri, 11 Jan 2013 19:15:47 +0700
Date: Wed, 9 Jan 2013 18:17:28 +0000 (UTC)
| Distfile on nbftp is enough for submitter; he'll investigate ftp(1)
I have done, and I don't think it is a bug in ftp(1), I think it is a bug
in the HTTP server that's serving the file (though I am certainly no
expert on the intricacies of HTTP file transfers, nor do I particularly
want to become one...)
The difference between using ftp and wget (or so it appears to me) is that
the http client in ftp(1) does
in the header of the GET request, and wget does
instead. The server (at www.kornelix.com) (which appears to be Apache, though
I have no idea which version) responds to the wget request with a header that
whereas with the ftp request, the first of those is present in the reply
header, but there is no "Connection:" field at all.
Either way, in both cases, the server seems to send the data in the file,
and then stop (implementing keep-alive type connections). NetBSD's ftp
client is assuming it will get a FIN to conclude the transfer, so after
all the data has arrived, it just sits and waits for that FIN. The server
on the other hand appears to be waiting for the next request.
Eventually after almost 3 minutes) I assume their server gets tired
of waiting for a new request, and closes the connection. That (finally)
provides the FIN that ftp(1) has been waiting for, it is happy, and the
connection completed properly.
But that 3 minutes is too much for pkgsrc, which tells ftp to give up
after 2 minutes. When that happens, ftp never bothers to flush the last
(partially filled) buffer, and just aborts, leaving those final 1902
bytes missing from the file (if there's any bug in ftp(1) it would be
that, it did receive the data, it could have written it to the file before
quitting, and had it done do, pkgsrc would have verified the file checksum,
I guess, if not on the same attempt as when ftp failed, then next time
it went to look and found the file already existing) and all would have been
OK. But demanding that processes clean up fully when they are failing,
is probably too much to exoect.
With wget, the client TCP (ie: my system running wget) sends the first FIN,
immediately after receiving the end of the file data, which is what you'd
expect with a client doing Keep-Alive (why it bothers when it has only one
file to fetch I have no idea, but it does, I guess just to be consistent with
usages when it is fetching entire trees of files.)
The real problem here appears to be the HTTP server that seems to be
implementing keep-alive connection mode, when just the opposite was
requested by the client.
I have complete tcpdump (binary form) dumps of the two transactions,
if anyone else (someone who speaks more http than I do) would like to
take a look and confirm (or refute) my analysis. They're each about 2.3MB
big (and probably do not compress much, as most of that will be the file
data, which was a .gz file, though I haven't tried to see).
I can e-mail one of both of those, or make them available for ftp (but
not for http...) if someone would like to take a look - or this seems to
be consistently repeatable enough, that you could make your own trace
(other than the 3 minute timeout, actually about 150 secs idle) all of
this happens fairly quickly, it is not a very big file to transfer.
Main Index |
Thread Index |