Subject: pkg/26448: resuming pkgsrc distfile fetches implementation is poor
To: None <gnats-bugs@gnats.netbsd.org>
From: None <kre@munnari.OZ.AU>
List: pkgsrc-bugs
Date: 07/27/2004 20:05:24
>Number:         26448
>Category:       pkg
>Synopsis:       resuming pkgsrc distfile fetches implementation is poor
>Confidential:   yes
>Severity:       critical
>Priority:       high
>Responsible:    pkg-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Jul 27 13:07:00 UTC 2004
>Closed-Date:
>Last-Modified:
>Originator:     Robert Elz
>Release:        NetBSD 1.6X  (--pkgsrc as of date of this message)
>Organization:
>Environment:
System: NetBSD jade.coe.psu.ac.th 1.6X NetBSD 1.6X (JADE) #17: Wed Sep 24 20:25:35 ICT 2003 kre@jade.coe.psu.ac.th:/usr/src/real-sys/arch/i386/compile/JADE i386
Architecture: i386
Machine: i386
>Description:
	The new feature that causes pkgsrc to automatically attempt to
	resume fetching of an interrupted distfile fetch is well motivated,
	but not necessarily a good idea (certainly not always a good idea)
	and needs to be at least made optional (so it can be disabled),
	but worse than that, the implementation currently assumes that if
	the distfile fetched is not the exact size that was expected,
	then the fetch must have been interrupted - nothing is less likely,
	in most environments, the more likely cause is that the distfile
	fetched has changed from the one expected, and that's why the
	size is now different.   As implemented now, the file will just
	keep on attempting to resume being fetched, always getting
	nothing any different from what was fetched last time (one hopes).
>How-To-Repeat:
	By code inspection.
>Fix:
	For now, just delete the feature (revert to what existed before)
	until a properly worked out solution can be found.

	At least, it needs an on/off switch.

	The test probably needs to be "is substantially smaller than"
	rather than "is not equal to" when comparing the sizes (that is,
	not just 50 butes smaller fetched than intended...)

	And parsing ls -l output with awk may be the most portable way to
	find the size of a file, but it is is horribly unreliable with some
	versions of ls (that simply allow fields to run together when the
	numbers get big) - maybe it is the best possible, but it does need
	some thought.
>Release-Note:
>Audit-Trail:
>Unformatted: