C++ compiler excessive memory usage?

[Havard Eidnes <he%NetBSD.org@localhost>]

Hi,

as part of my efforts to maintain rust for our various NetBSD
targets and testing / validating the result, I have come across a
problem where my C++ abilities are coming up short, so I'm
looking for others to take a look and give some guidance.


Rust 1.99.0 which for our targets is in it's testing / validation
round has uncovered an issue with the embedded LLVM, which is now
version 23.1.1, so newer than what we have in pkgsrc-wip.

In short, the issue I am observing is that the C++ compiler is
consuming extraordinary amounts of virtual memory while building

lib/Target/RISCV/CMakeFiles/LLVMRISCVCodeGen.dir/GISel/RISCVInstructionSelector.cpp

In fact, so much that the rust compiler can now no longer be built
with the embedded "full" LLVM on NetBSD/macppc, ref.

  https://github.com/llvm/llvm-project/issues/230790

I'm also trying to build the same combination on a NetBSD/sparc64
10.0 system with 3GB of physical memory, using gcc14, and I've
now had to add 3 more 2GB swap files, and it's still going.
"top" shows:

27767 he        31    4  8311M 1087M RUN        72:26 28.81% 28.81% cc1plus

and of course it's thrashing, so I'm applying copious amounts of
patience.

Now, for rust 1.99.0 for powerpc I'm pulling the "trick" I did
earlier while rust was still possible to "self-host" on our 32-bit
ARM ports, namely to omit cross-build-ability for most of the CPU
types LLVM by default supports, and it appears to have worked as far
as the embedded LLVM is concerned (the build is still going, but has
progressed to later stages of the rust build).

But it's many, many rust versions ago that "self-hosting" on 32-bit
ARM ports was possible, where around 1.8GB of user virtual address
space is available.  It looks like powerpc with an oea CPU has
"almost 4GB" of user virtual address space?  (VM_MAX_ADDRESS from
sys/arch/powerpc/include/oea/vmparam.h).


So... The ask: can anyone with better C++ abilities than me take a
look at what is so peculiar about this particular file that it
causes this excessive resource consumption by the cc1plus compiler
component?

The next step is of course then to tell me/us if there is something
which can be done to change this behaviour?  By the looks of it,
there won't be much assistance from the LLVM side.

This issue may crop up in our own backyard when the time comes to
upgrade LLVM to 23.x and beyond.


Best regards,

- Håvard