Re: download dependencies in background while building

[Marc Espie <espie%nerim.net@localhost>]

On Wed, Mar 12, 2008 at 01:29:34PM -0500, Jeremy C. Reed wrote:
> I was told today: It's a shame it doesn't download dependancies in the 
> background while building.
> 
> There is a way to do that by building multiple packages at same time. But 
> don't thnk this locking works anymore. (It didn't work last time I tried.)

We have a variety of mechanisms in OpenBSD to take care of that, don't
know what you can use...

- we compute and manipulate dependency chains. We compute the full graph of
dependencies, and pass it to tsort to do various things. Our tsort has
extensions to try to favor some order of the result while respecting the
constraints.
- make fetch-list works, and is used all the time with make -j (actually,
we have a `mirror-maker' target that builds a makefile that records all
fetch stuff.
- Nikolai Sturm wrote dpb, a ports builder that can distribute load over
machines based on that information, this can include the same host several
times.
- we have locking that works, with a granularity at the port level, and
separate locks for each distfiles.
- dependencies and downloads are `true' dependencies, so if you start a
build with make -j, you'll notice it starts building several dependencies at
once. Our infrastructure is mostly clean for parallel builds, so correct
fetching just happens...