Re: fetch: download to temporary location

[David Brownlee <abs%absd.org@localhost>]

On Sun, 8 Feb 2026 at 13:23, Thomas Klausner <wiz%netbsd.org@localhost> wrote:
>
> Especially in bulk builds it can happen that multiple versions of the
> same package are being built at the same time (e.g. a Python module),
> without the distfile being present. Then two downloads are started,
> both to the same location, and one of them finishes and tries to
> verify the incomplete file written by the other, leading to checksum
> errors, breaking the build.
>
> For this reason I've changed the fetch script to download to a
> temporary file instead, and move the file to the final location when
> the download is finished - so the file is there or not, but not a
> partially downloaded copy.
>
> The script already supported resuming downloads - the support is still
> there but basically will never happen again, because it only looks at
> a partial file of the correct name (which can still happen if the
> download finishes but provides an incomplete file, and the file cannot
> be verified by during download e.g. because the distinfo file is not
> available - a theoretically supported use case).
>
> Instead we might end up with more foo.tar.gz.12345 files, when
> downloads are interrupted.
>
> I think these are acceptable trade-offs.
>
> Please try it out, I plan to commit this in a week.

Great to see this. I'm a *huge* fan of writing things to temporary
files first :-p

Could I ask if the temporary filename could include an easily
identifiable suffix to make cleanup easier?
Maybe ".$$.tmp"? (or ".$$.part" or ".$$.downloading" or
"$$.prettymuchanything" :)

Thanks

David