Re: pbulk-index performance improvement

[David Brownlee <abs%absd.org@localhost>]

On Tue, 24 Mar 2026 at 15:52, Jonathan Perkin <jperkin%pkgsrc.org@localhost> wrote:
>
> * On 2025-12-23 at 15:07 GMT, Jonathan Perkin wrote:
>
> >Here's a diff that in theory avoids this problem for packages that do
> >not support multi-version in their PKGNAME:
> >
> >  https://gist.github.com/jperkin/ce0b765a3a868389b09e235b58105a32
> [...]
>
> Now that bob has a decent amount of tooling for testing things like
> this, I've discovered that it's unworkable due to a couple of packages:
>
>    lang/ruby
>    www/unit-python
>
> ruby expands RUBY_VERSION and unit-python expands PYVERSSUFFIX, and
> while we could add additional checks for these variables, I feel this is
> just a landmine waiting to explode when some other new package starts
> using a different variable that isn't matched yet.
>
> I'm out of ideas for how to do this in a reliable way (e.g. trying to
> uniq'ify PKGNAME just runs into the same problem as you have to pay the
> parse tax up-front), so until a novel solution presents itself then I
> retract this proposal.

Sorry to hear that - it was such a nice speed up :(

Is there a reliable way to tell during normal (full Makefile parse)
build on a single package whether it has a multi-version issue?

If so, then maybe...

I had (a long time back), been thinking about a way to tag a package
as 'fast parse' eligible, for simple pkgsrc scanning tools

- Define a way to tag a package to indicate that it cannot be parsed
via the fast route (will come back to how it is tagged)
- For a standard build if the package is not so tagged, the build
compares a fast parse PKGNAME with the actual PKGNAME, and throws an
error if they do not match, requiring the developer to tag the package
to get it to build

The catch was how to tag the package in such a way that it doesn't
require a full parse to find the tag. The two (somewhat
unsatisfactory) options I had come up with where either:
a) the presence of a "magic" file in the package directory
b) mandating a NO_FAST_PKGNAME_PARSE=yes in the Makefile itself,
allowing a quick 'fake parse', enforced as much as possible by pkglint

The situation is annoying enough and the potential benefit significant
enough that a less than elegant solution might be worth trying, but
its all down to "how much" less elegant...

David