grafana hits pkgsrc limits again

[Thomas Klausner <wiz%NetBSD.org@localhost>]

Hi!

For the last grafana update, I had to update the distinfo code to
handle a huge list of go module files that grafana needs.

I just tried updating grafana to 11.1.0 (see wip/grafana), and now I
run into a similar limit in the checksum step, from
mk/checksum/checksum.mk:

${_CHECKSUM_CMD} ${DISTINFO_FILE:tA} ${_CKSUMFILES}

The ${_CKSUMFILES} variable gets too big for the sh(1) command line
length limit.

So I implemented a similar change like the one I did for 'make
distinfo' - putting the list of files to check into a file and then
pass that to checksum.awk (see attached).

This seems to work better, but... it doesn't find a checksum for
'buf.build_gen_go_bufbuild_protovalidate_protocolbuffers_go_@v_v1.31.0-20230802163732-1c33ebd9ecfa.1.mod'.

I think the problem is that we ... yes, pass the whole file list again
on the command line, this time from checksum.awk to digest(1), see
mk/checksum/checksum.awk:

        # We now have a list of distfiles to be checked for each algorithm,
        # pass them all to a single digest(1) command and parse the checksums                                                                                                                     # to be compared against distinfo.
        #
        for (algorithm in distsums) {
                cmd = sprintf("%s %s %s", DIGEST, algorithm,                                                                                                                                                   distsums[algorithm])
                while ((cmd | getline) > 0) {

and I guess that hits a shell limit again.

Suggestions for fixing this?

Passing them one by one will be a huge speed decrease for packages
with many modules, so that's out.

Should we try passing it in groups of a hundred or a thousand? Or
probably better teach digest(1) to accept a list of files argument?

Anyone interested in working on this?
 Thomas