Re: Bob woes

[Jonathan Perkin <jperkin%pkgsrc.org@localhost>]

* On 2026-09-01 at 20:22 BST, Rhialto wrote:

> I see issues where not all of the dependencies of a package are
> installed before it is built. After long searching I found relevant log
> files in /root/.local/share/bob/logs/ (can't I set those somewhere else?

Yes, either set options.dbdir to change the entirety of 
~/.local/share/bob to a different location (you might want to copy 
history.db first), or set options.logdir if you only want to move the 
logs directory (defaults to dbdir/logs).

> => Tool dependency checkperms>=1.1: NOT found
> => Verifying /nonexistent for ../../sysutils/checkperms
> make[1]: don't know how to make /nonexistent. Stop

This is the classic marker to indicate that there is a difference 
somewhere in your setup, either between when the scan is performed and 
when the build is performed, or when you built your existing binary 
packages compared to when you are now building.

Whether using pbulk or bob, the scan phase runs 'make pbulk-index' for 
each package, and records the list of dependencies for each package.  
So for example:

  $ cd devel/m4
  $ bmake pbulk-index | grep ^ALL_DEPENDS
  ALL_DEPENDS=   nbpatch-[0-9]*:../../devel/nbpatch cwrappers>=20150314:../../pkgtools/cwrappers checkperms>=1.1:../../sysutils/checkperms    pkg_install-info-[0-9]*:../../pkgtools/pkg_install-info

Now when pbulk or bob go to build m4, they first install those packages 
from binary package tarballs into the clean build area, and then the 
rest of the build is performed with DEPENDS_TARGET=/nonexistent, so that 
if the build somehow needs a different set of packages it will fail.  
Yes the error message is terrible, but that's basically what it means.

My guess is that you have a difference when running the scan phase which 
means that PKG_DEVELOPER is set to no, or similar, such that checkperms 
is no longer listed in ALL_DEPENDS.  checkperms is only added as a 
dependency to builds when CHECK_PERMS is enabled, which is usually only 
when PKG_DEVELOPER=yes.  So:

  $ bmake PKG_DEVELOPER=no pbulk-index | grep ^ALL_DEPENDS
  ALL_DEPENDS=   nbpatch-[0-9]*:../../devel/nbpatch cwrappers>=20150314:../../pkgtools/cwrappers    pkg_install-info-[0-9]*:../../pkgtools/pkg_install-info

and checkperms ends up missing.

It's really important that nothing changes at all in the environment 
once you have started a build.  You will get similar errors for example 
if you change PKG_OPTIONS after you have already built binary packages, 
as the dependencies that are pulled in automatically by 'pkg_add dep' 
will conflict with ALL_DEPENDS from the scan.

>    environment = {
>        build = {
>            clear = true,
>            inherit = { "TERM", "HOME" },
>            vars = {
>                PATH = "/sbin:/bin:/usr/sbin:/usr/bin",
> 		PACKAGES = rhialto.packages_inside,  -- to override mk.conf's PACKAGES ?=
>                BINPKG_SITES = rhialto.packages_inside,
> 		DISTDIR = rhialto.distdir_outside,  -- to override mk.conf's DISTDIR ?=

This will also be contributing to the issue.  I made a mistake in very 
early versions of bob by showing examples where pkgsrc variables are set 
in the bob config, while I was exploring the best way to handle this.  I 
was wrong, it was the wrong direction, and you shouldn't do this.  

Everything pkgsrc-related should be encoded directly in mk.conf.

One reason is so that builds will continue to work outside of bob.  
Another reason is to keep things simple and ensure a single source of 
truth.  The main reason from bob's point of view is that the first run 
that caches pkgsrc variables and sets the bob_* variables runs *without* 
these variables being set, so PACKAGES from bob's point of view will be 
the mk.conf ?= setting, not rhialto.packages_inside.

It's obviously on me to document this correctly and I'm trying to find 
some time to do that, but if you are looking for a way to make things 
different when running inside bob (I wouldn't recommend it, but I 
understand that e.g. some people might want different bob configs for 
different branches or similar), then the best way to do that would be 
something like:

 - A custom "copy" action per bob config that copies /etc/mk.conf.local
   or similar into the sandbox that is ".sinclude"'d from /etc/mk.conf.

 - Some construct in /etc/mk.conf like:

     .if exists(/etc/mk.conf.2026Q4)
     PACKAGES=...
     .elif ...

   and a "cmd" action that touches /etc/mk.conf.2026Q4 for that config.

or similar.  The way I do it is that I have a separate mk.conf for every 
single one of my builds:

  https://github.com/TritonDataCenter/pkgbuild/tree/master/conf

and the bob config (I have one per build) will include the correct 
mk.conf for that build.

--
Jonathan Perkin                    pkgsrc.smartos.org
Open Source Complete Cloud   www.tritondatacenter.com