Re: pkgsrc mktool utility

[Thomas Klausner <wiz%gatalith.at@localhost>]

On Mon, Aug 05, 2024 at 03:08:07PM +0100, Jonathan Perkin wrote:
> Inspired by the recent struggles with wip/grafana and the number of
> distfiles it has grown to, as well as frustrations I've had for a while with
> some parts of the infrastructure, I've started work on a separate utility as
> an optional alternative to some of the slower parts of pkgsrc/mk.
> 
> Let's start with a simple benchmark and 100x performance improvement:
> 
>   $ cd wip/grafana
> 
>   $ time bmake mdi
>   real    3m30.218s
>   user    1m13.843s
>   sys     1m49.071s
> 
>   $ time bmake TOOLS_PLATFORM.mktool=mktool mdi
>   real    0m2.094s
>   user    0m8.217s
>   sys     0m6.979s
> 
> "mktool distinfo" is the first command implemented, and as far as I'm aware
> it is bug-for-bug compatible with distinfo.awk, with the exception of
> "ignorefile" support, which I don't believe is used anywhere.
> 
> If you want to try it out, "cargo install mktool", set
> TOOLS_PLATFORM.mktool, and apply the following trivial patch:
> 
>   https://github.com/TritonDataCenter/pkgsrc/commit/18c7342525f6a6ab23531ec48be98936984d312b
> 
> My general query is how upset would people be if I committed this patch,
> and others like it in the future?  I wouldn't ever make use of it mandatory,
> but I would like it to be an easy optional drop-in replacement for any part
> of the infrastructure where a dedicated tool will far outperform any
> shell/awk implementation.
> 
> The obvious next candidate is a replacement for checksum.awk.  Let me know
> if you're interested in any other replacements.

That's a very neat tool, thank you!

I could update the distinfo file for wip/grafana with it (not that I
can build it, because extraction fails due to the shell argument
limit).

(I have no idea how we could add it as a base package though if it's
in rust. If we want to replace the shell/awk/make/... infrastructure
it needs to be in something that we can bootstrap everywhere, and I
fear rust's not it, currently. So we'd need a compatible C version, or
something.)
 Thomas