Re: Build failure tooling improvements

[Thomas Klausner <wiz%NetBSD.org@localhost>]

On Fri, Mar 21, 2025 at 09:29:19AM +0000, Jonathan Perkin wrote:
> The way I'd do this is collate all of the touched packages in a certain
> time-frame, and insert them into limited_list bulk builds running on
> multiple platforms.  The reports would then be sent to pkgsrc-bulk (perhaps
> excluding 100% successful builds to avoid too much spam), and any failures
> would be emailed to the committers responsible.

This sounds very useful to me. I'd like to have that please :-)

> Other suggested features would include running pkglint on the files and
> again emailing any failures or suggestions,

While I would like to get pkgsrc pkglint clean, I don't think it's
ready for this yet.

> and configuring multiple bulk
> builds for the same platform (probably NetBSD for the widest coverage) but
> with different PKG_OPTIONS to test for e.g. 'doc' fallout, non-default
> PREFER_*, PKG_SYSCONFDIR, etc.

I don't know how easy that would be but having an 'all-options on' and
an 'all-options off' build would probably catch most of the option
problems.

Do you have experience how much stuff breaks when changing PREFER_*?

> The simplest way to do this would be to create some workflow triggers in a
> .github directory at the root.  I know some people have a strong aversion to
> GitHub, but this would be practical for a number of reasons, with one of the
> more compelling ones that the conversion sync always waits for a quiet
> period, so this would avoid builds being triggered in the middle of a large
> update and resulting in unnecessary failures.

I don't have a problem with running it on github for now, but
long-term I think a non-github version would be preferable.

(Relying on the git-conversion-quiet-period will not work longterm,
when we switch SVM away from CVS, but you're probably aware of that.)

> If people are willing to get behind this then I'm more than happy to work on
> it, and I can provide hardware to run builds for multiple operating systems.

This sounds very useful, yes, please.
 Thomas