Re: bulk builds incompatible with changes

[nia <nia%NetBSD.org@localhost>]

On Fri, Apr 11, 2025 at 05:33:06PM +0000, John Klos wrote:
> Hi,
> 
> Does anyone have any suggestions about what to do about updates to the
> pkgsrc tree when running bulk builds? Stopping a bulk build and rescanning
> from scratch is really not ideal, particularly on slower systems.
> 
> Ideally, I'd like if there was a straightforward way to do a new scan on a
> fast system, then merge the updated things in to the build for the slower
> system. I've done this manually, but it's definitely not straightforward.
> 
> John

Interrupting the build is really the last way I'd want to do this,
there's too much room for stuff to go wrong, e.g. shared library
inconsistencies.

Bulk builds on well... anything that isn't a supercomputer are
clearly a problem, though. There are a few ways around that.

- Large sections of the tree can be ignored. Start by building
bulk-small, move up to bulk-medium if there's still time, etc.
There's not much value to doing a _full_ bulk build on say, vax
or m68k. Better to prioritize the most popular packages than
the entire of texlive and hundreds of fonts, for example.

- The size of the tree can be reduced, removing e.g.  all haskell,
ocaml, erlang, go packages on platforms where they have no hope of
working (me and a few other developers have the opinion that pkgsrc
is "too big for its boots", but this can at least be easily achieved
locally).

- The scan process can be parallelized across multiple chroots or
machines (contrary to popular beliefs, using multiple chroots doesn't
require any modifications).