Re: trip report from a first-time bob user: zero to a built package in 2h

[Greg Troxel <gdt%lexort.com@localhost>]

Jonathan Perkin <jperkin%pkgsrc.org@localhost> writes:

> Thanks Greg, I'll respond to the issues you've raised on GitHub in due
> course,

Sure - they are a pile of nits, except for lack of clarity on dynamic
stuff (https://github.com/jperkin/bob/issues/18) which I see as the
most serious of the filed issues.

> and will just provide a little commentary to some of the
> general points here so that they're useful for others.

Thanks - will trim things for which a reply comment is not useful.

> * On 2026-06-23 at 21:02 BST, Greg Troxel wrote:
>
>>Some of the config file contents are active, and some are commented out.
>>I can't figure out why.  I can't figure out whether running bob with an
>>empty config file assumes the contents that are in the config file, or
>>runs with an empty config and thus mostly errors/does-nothing.
>
> There are two extremes with default config files:
>
>  - Minimal that requires reading lots of docs and adding necessary
>    sections before you can run the command for the first time.  This can
>    often put a lot of people off before they even start due to the
>    cognitive overhead.
>
>  - Overly verbose with many sections that may need to be trimmed or
>    edited before you can run the command for the first time.

Agreed, and my preference is to omit any recitation of defaults,
commented out, and very strongly omit non-commented-out defaults.  So I
think you're pretty much there, except for the commented-out sections
that most people will want, and I see the point there.

> My goal with bob is a happy medium where should be able to run it out
> of the box with zero editing.  This is the case on NetBSD (on others
> you'd need to provide a bootstrap kit, I can't help that) - assuming a
> default NetBSD install, a 'bob init' followed by 'bob build' will
> immediately start building packages inside sandboxes with no other
> setup required.

Well, it didn't work for me, because I have non-default DISTDIR and
package dir, and a /links dir that's needed.  But that's ok; I
knew those were my deviations and that I'd have to adjust for them.  The
sandbox config was easy to follow; almost like a shell script turned
into lua (with the inverses being automatic).

> The commented out sections are features that aren't required, but that
> I believe the vast majority of users will want, and so they are there
> to provide a quick starting point should people want them.  Not every
> feature is in the default configuration file.

Maybe a comment line or two to say that; as a first-time config file
reader it was not clear, even though it's becoming clear already (my one
chance at being new is spent, as I said :-).

While can make lots of tmpfs mounts, that doesn't affect the one thing
that matters, which is a limited amount of RAM, and thus a limited
amount of space in use across the groups of tmpsfs mounts.  In this way
it is just like a jobs limit.  As it is, with build_jobs 4, I'd have to
take the amount I'm willing to allocate to tmpfs for builds, divide by
4, and assign that to each build, whereas with giving the amount in
total, the dynamic scheduler could decide to run the piggy job by
itself, with more build threads, which might well be a win vs pushing it
to disk.  Or to run it with a bunch of builds that use a small amount of
space.   I am guessing many packages use < 100 MB.

Maybe this is in the user guide you are working on, but
guidance/discussion around choosing parallel limits would be good.  A
stab at it:
  https://github.com/jperkin/bob/issues/25

> As a real-world example, on my macOS daily bulk build host where I
> want as much on tmpfs as possible, I've had to tweak it so that on a
> 3GB ram disk, threshold is 1.3GB, because there are still some
> packages that will blow past 3GB during their build but only record a
> little over 1.3GB by the time they have finished.

I had the impression part of the dynamic scheduler was noticing that a
build failed with a tmpfs allocation and adjusting the "needed wrkdir"
score upwards to the failure point, and that this was on balance better
than having to kick many packages to disk because of a few that behave
oddly.   So I am surprised at this comment.

But then I see

  TIMESTAMP            PKGNAME    OUTCOME  MAKE_JOBS  WRKOBJDIR  DISK_USAGE  DURATION
  2026-06-23 20:45:29  m4-1.4.21  success         12  -               33.9M     1m11s

so I can see DISK_USAGE being reset to the success amount later.

(In my experience, thunderbird is the most problematic package.)

> This ratio will change depending on what size tmpfs is set to and thus
> what types of packages are able to fit inside the target tmpfs.

I get it that this is messy.

> Set everything pkgsrc-related in mk.conf.  That way all builds are
> correct whether performed by bob or built manually.

I ended up figuring out that this worked.   Might be nice to say up
front that this is the plan.

>>I have /etc/mk.conf, a symlink into a file managed in CVS in my homedir.
>>That does an include of /etc/mk.conf.local.    I don't feel confident
>>that's going to be ok, and didn't find discussion about how system
>>mk.conf* is carried into sandboxes but reading the setup seems like
>>  - /etc is mapped, so mk.conf.local is ok
>>  - symlink to home is a problem
>
> Yes this likely won't work by default, the "copy" action will preserve
> symlinks not follow them, so you would need something like this added
> after the "/etc" copy:
>
>   { action = "copy",
>     src = "/home/gdt/cvs/mk.conf.local",
>     dest = "/etc/mk.conf.local" },

It did actually work, I'm pretty sure.   I was perhaps fuzzy:

  /etc/mk.conf -> /home/gdt/repo-name/dir/mk.conf
  /etc/mk.conf.local (an actual file)

but I'm not sure how it worked.

> I think this being missing might explain your ccache issue.

I now think it's because I enabled ccache if /usr/pkg/bin/ccache exists;
will do it differently and see.  When I set this up long ago, I built a
few packages on some machines without ccache, but now I mostly build on
a few build machines and low-resource machines use pkgin from that, so
"just require ccache" is now reasonable.

>>One might want to write mk.conf code that's different under bob than not
>>(e.g. my WRKOBJDIR stuff), so there should be something defined you can
>>switch on.
>
> You could use one of the existing environment variables for this,
> e.g. bob_make, though I think adding this feels like a code smell.  As
> long as you set e.g. WRKOBJDIR?= ... then it'll work fine outside bob,
> and bob's passing of WRKOBJDIR=/some/where/dynamic will take
> precedence, the same way it would if you did a manual (b)make
> WRKOBJDIR=/whatever.

I can see how I can avoid many things, but I still think it should be
possible to write something that looks like

  .if PKGSRC_BUIDLING_IN_SANDBOX
  VAR1=VALUE2
  .else
  VAR3=VALUE4
  .endif

and I can't see a general proof that anyone who ever wants to do that is
wrong.

>>Tried to ^Z bob.  Nothing happened.  Feels like a bug to me; I want to
>>be able to stop high-load processes when I want.  Maybe I have something
>>odd.
>
> No, this isn't supported yet, and would be a reasonable project I
> think given all of the threads and chroots and sandbox processes that
> bob has spawned.  It would be nice to have, but will require some
> thought and careful planning.

I think that should be clearly called out as a known
bug/not-implemented-yet in the man page.  I get it that it's hard, but
it's a big violation of the Principle Of Least Astonishment.  I was
actually surprised, as I see the builds as just a bunch of subprocesses
after sandbox setup, and don't see why just not blocking ^Z would do
something wrong.

> At the moment if you press ^C then bob will continue to run, but will
> not issue any new builds, so you will see each worker come to a stop
> once they are finished and bob then exits when all done.  A subsequent
> ^C will forcibly kill everything and shutdown much more quickly.

I did sort of come to understand that mostly, that there was some sort
of clean shutdown, but it was not obvious that it was "waiting for
already-launched builds to finish" vs "waiting for killing build jobs
and then cleaning up".  Sounds like the second ^C is that, and there's
no "really stop right now and don't try to clean up", or maybe the third
is that.