Subject: Robustness of file fetches
To: None <firstname.lastname@example.org>
From: Joerg Sonnenberger <email@example.com>
Date: 06/09/2005 00:08:28
there are a lot of broken sites out there which return
an error document, but forget to also set the HTTP status
to 404. A good example seems to be provided by
chat/ircd-hybrid. Can we add a heuristic to threat large
files (e.g. > 100k) which fail within the first 25k as
possible non-existent and just go on to the next mirror?
Alternatively, can we allow to provide a size argument
to the normal fetch command as well and let that deal with
too short files?