Re: ~4000 packages failing on my own bulk build

[Jonathan Perkin <jperkin%joyent.com@localhost>]

* On 2014-12-08 at 08:40 GMT, Emile `iMil' Heitor wrote:

> Since pkgsrc-2014Q3, the bulk build is starting at ~4000, so I guess something
> went wrong on the scanning process, and many critical packages (cairo for
> instance) are not built anymore, leading to many `indirect-prefailed'.
> I guess something is wrong in my setup now, but I can't figure out how to debug
> it, any idea anyone?

We worked through this on IRC, but I'll reply anyway so that if anyone
else runs into similar problems they can debug them too.

The 'Packages causing the most breakage' in the bulk build report only
considers packages where a build was attempted - it doesn't include
packages that were prefailed or indirect-prefailed, and often (as in
your case) these can be the biggest failures.

The simplest way to find them, as long as the bulk build has
completed, is to look at the 'pbuild' file generated in the
bulklog/meta directory.  Sorting on the number of 'prefailed' failures
can be useful, i.e. for the current upstream SunOS builds:

  $ awk -F'|' '$2 == "prefailed" {print}' meta/pbuild \
    | sort -t'|' -k4rn | head -4
  fuse-2.9.3|prefailed||31
  libusb1-1.0.9nb1|prefailed||28
  mbrola-301hnb2|prefailed||9
  compat40-4.0|prefailed||8

Once you've identified which packages are responsible for the most
failures, you can look for their PKG_FAIL_REASON either in the
'presolve' file or in the main report.html in the second column.  This
should explain the reason for them failing.

If you want to identify relationships, the 'report.dot' file can be
used to see which 'prefailed' package is responsible for causing its
dependants 'indirect-prefailed'.

> The resulting bulk report is available here:
> http://packages.netbsdfr.org/latest/6.0/amd64/reports/20141205.1639/meta/report.html

In iMil's case the cause is the failure of x11-links, and you can see
from the report above it was responsible for 3,857 failures.

One feature which would be useful is an option to show the top
'prefailed' packages in the report.  It may not be desired as a
default option as for the most part it will be NOT_FOR_PLATFORM noise,
but maybe having it for the occasions where it is useful is more
important.  I'm ambivalent on this one.

-- 
Jonathan Perkin  -  Joyent, Inc.  -  www.joyent.com