Re: make modular X11 the default on NetBSD

[David Holland <dholland-pkgtech%netbsd.org@localhost>]

On Wed, May 16, 2012 at 11:26:22PM +0200, John Marino wrote:
 > >We(TM) also really need to do something to make the scan phase faster,
 > >or at least to make it possible to cache the results.
 > 
 > Absolutely.
 > If the pkg version / revision of a specific package don't change
 > between runs, can a cached result be used instead of performing a
 > new scan?  Or are there other dynamics involved?

No. It potentially depends on the whole buildlink tree, and currently
there's no useful way to shortcut evaluating it all.

One possible way forward is to teach make to do the equivalent of gcc
-MM, and reuse the prior scan results if none of the included files
have changed. This will be messy inside make, but nothing like as much
work as the major revisions I've been looking at.

For the stable branch builds, though, a decent approximation should be
to rerun the scan only for package directories where a file changed,
and any/all package directories that depend on those. And do a full
rescan if anything changes in mk/. There are some things that would
defeat this rule, but mostly they aren't things likely to be changed
on a stable branch, and we could probably get away with handling the
exceptions manually and/or automatically doing a full rescan once in a
while.

The other thing is, bulk building has two purposes. One is generating
binary packages for people to use; the other is testing. Right now the
tools favor the testing role. pbulk, for example, will rebuild a
package if any of its files get updated or if anything it depends on
gets rebuilt; this is necessary for testing (now and then it turns up
unexpected problems) but for the most part not necessary for producing
binary packages, especially on the stable branch. We can and probably
should adjust this sort of thing for stable builds; we do far too
much belt-and-suspenders-and-maglev-and-antigrav stuff in pkgsrc and
it's unnecessarily expensive.

-- 
David A. Holland
dholland%netbsd.org@localhost