Bulk builds as a helper for pkgsrc developers

[Aleksey Cheusov <cheusov%tut.by@localhost>]

One of the goal of running bulk builds regularly is to be informed about
problems as soon as they appear. At the moment the only thing we really
have is an information about packages failure, so package maintainer or
anyone interested can easily see problems with package they maintain or
use and fix it ASAP.

I believe bulk build are able to provide much more.  Right now, I'm
working on distbb where I'd like to implement the following additional
sanity checks and promptings for developers.  If you think you may
enrich the list below or have a comment, let me know.

By implementing these features in distbb I'd like to reduce a number of
PRs related to pkgsrc we have in gnats and to see (and fix) problems as
soon as possible.

1) missing CONFLICTS

   It's well-known problem that many pkgsrc packages have common files
   but have no corresponding CONFLICTS entries.
   If we keep such packages as is, both binary- and source-based
   upgrade may fail in middle of the process leaving user with broken
   system for a while (In pkgtools/nih I don't start upgrade when I
   encounter this kind of problem but other package managers do).

2) unnecessary CONFLICTS

   Some package may have CONFLICTS entry but don't conflict.
   This problem is trivial.

3) PROVIDES/REQUIRES consistency

   In order to garantee "working" system after upgrade it makes sense to
   check packages to be installed for PROVIDES/REQUIRES consistency.  I
   do this in pkgtools/nih, for example. pkgin probably does the same.
   The problem here is that pkg_summary(5) provided by our bulk builds
   contains a few unsolved problems of this type. As a result packages
   having such problems cannot be installed using nih or pkgin.
   For example, openoffice2 had such problem several months ago.

4) Possibility of installing each individual package in repo.

   Recently I encountered the following problem. I had package A-0.1 and
   package B-0.1 installed on my system. Package B depends on package A.
   Both these packages were updated to A-0.2 and B-0.2 in pkgsrc tree.
   I ran bulk build. B-0.2 succeeded but A-0.1 failed. As a result B-0.2
   and A-0.1 become a part of generated pkg_summary(5) i.e. repository
   for nih/pkgin. The problem here is that B-0.2 conflicts with A-0.1,
   thus B-0.2 being in a repository could not be installed. It would be
   nice to see such problem at the time of bulk builds without waiting
   for users' PRs.

5) config.log and stage_build.tmp may help to recognize
     bad DEPENDS and BUILD_DEPENDS

   Yet another well-known problem of pkgsrc on non-NetBSD system is that
   "foreign" build hosts may significantly differ from NetBSD base
   system. As a result tools required for build (yacc, lex etc.) may be
   absent and cause failures. Not all such tools are a part of
   USE_TOOLS. Some packages require "ping", "dig" and others.

6) Versions going backward

   I raised this problem several years ago but it is still not fixed
   properly.

7) Bad DEPENDS/BUILD_DEPENDS and result from previous bulk builds.

   Suppose we have libA-0.1 and B-0.1, where B depends on
   libA-[0-9]*. Also suppose that both libA and B were updated to 0.2 in
   pkgsrc tree and we have binary package for libA-0.1. In this case
   bulk build tool may try to build B against libA-0.1 or libA-0.2.  I
   don't know about pbulk but in distbb I build B-0.2 against libA-0.1
   in order to minimize problems of potential libA build failure and to
   provide more packages in the repository.

   However, this approach raises another problem. From time to time I
   see that DEPENDS and BUILD_DEPENDS are not set correctly. Suppose
   that in our example B-0.2 cannot be built against A-0.1 and DEPENDS
   should be updated to libA>=0.2. Recently I found such problem with
   rtorrent and librtorrent, for example. In general it is not trivial
   to recongnize such problems, because they are aften unseen not only
   by packagers but by upstream too. Bulk builds may help here too. In
   distbb I track ALL dependencies used to build the package and results
   from at least one previous bulk build. If, for example, package B-0.1
   depending on libA-[0-9]* fails with libA-0.1 but succeded with A-0.2
   this may sign maintainer/observer that DEPENDS should be adjusted
   accordingly.

-- 
Best regards, Aleksey Cheusov.