Re: Combining NOT_FOR_* and BROKEN_ON_*

[Jonathan Perkin <jperkin%pkgsrc.org@localhost>]

* On 2025-04-03 at 10:35 BST, Thomas Klausner wrote:

> On Thu, Apr 03, 2025 at 10:29:52AM +0100, Jonathan Perkin wrote:
> > My opinion is that BROKEN* is a solution looking for a problem, and I've yet
> > to hear a reason why having it makes pkgsrc better.
>
> When I look at bulk build reports, I don't want to see the same error
> again and again - it can be analyzed once, marked with BROKEN*
> explaining the problem, and will not show up as a build failure in the
> next bulk build, saving me from having to remember it, and other
> people from analyzing the same problem again.

Ok, so here the problem you are trying to solve is that you only want to 
see new failures.  That's a reasonable requirement!  I believe this 
problem would be better solved by improving the bulk build tooling, and 
having better reporting of new failures, as well as consolidation of 
existing failures so that we can identify when a package changes from 
being broken by one thing to being broken by something else.

The issues with using BROKEN to solve this problem is that:

 * It's a manual process to add the BROKEN in the first place.

 * It is incomplete (many thousands of broken packages are not marked
   BROKEN).

 * Any changes to the package state (different failure, or perhaps it
   now even builds fine) are not tracked.

 * Removing the BROKEN requires that someone be attentive to the
   situation and actively review the build manually as they no longer
   have bulk build tracking for it.

Having these handled by the bulk build tooling would avoid all of these 
issues.

> It also saves time in the bulk build (installing the dependencies, 
> configuring, building until it fails).

Ok, so here the problem is you want bulk builds to be faster.  That's 
also reasonable and something I'm very passionate about and have done a 
significant amount of work on.  I'm more than happy to work through any 
suggestions to improve this.

Perhaps it's worth considering having some options that differentiate 
between types of bulk builds.  At least for me I run two types of bulk 
builds:

 * Daily builds where I do not care about the generated packages, I am
   only interested in the failure reports, where I want a 100% accurate
   up-to-date and full picture of the exact state of pkgsrc for that OS.

 * Package builds where I do not care about repeat failures, I am only
   interested in generating successful packages as quickly as possible
   so that I can deliver security and reliability fixes to users.

For the latter I have for many years had a bit of mk.conf logic that 
will add PKG_FAIL_REASON for packages that I know will fail:

  https://github.com/TritonDataCenter/pkgbuild/blob/master/include/pkgfail.mk

I think it would be good to have something similar to this built-in to 
the bulk builds that is easy for others to use.

I don't think manually adding BROKEN to thousands of packages is a good 
solution to this problem.

> If someone is interested in a package, they try building it, see it's
> BROKEN and can either decide that they don't spend the time to fix it,
> or remove the BROKEN line and work on it.

If someone wants to build a package then I'm not sure why BROKEN makes 
things better for them.  At best it will be accurate, but they will 
still want to build for themselves to see if it's still true and start 
working on the problem, in which case it did nothing other than force 
them to remove it before being allowed to start building the package.  
Or maybe the build fails differently since the BROKEN was added, or 
maybe it even works now, in which case it wasn't helpful at all.  I 
don't see what problem BROKEN is solving here.

Cheers,

--
Jonathan Perkin                    pkgsrc.smartos.org
Open Source Complete Cloud   www.tritondatacenter.com