Re: C++ compiler excessive memory usage?

[Havard Eidnes <he%NetBSD.org@localhost>]

> On Mon, Oct 12, 2026 at 14:48:50 +0200, Havard Eidnes wrote:
> 
>> In short, the issue I am observing is that the C++ compiler is
>> consuming extraordinary amounts of virtual memory while building
>> 
>> lib/Target/RISCV/CMakeFiles/LLVMRISCVCodeGen.dir/GISel/RISCVInstructionSelector.cpp
>>
> [...]
>> So... The ask: can anyone with better C++ abilities than me take a
>> look at what is so peculiar about this particular file that it
>> causes this excessive resource consumption by the cc1plus compiler
>> component?
>
> Make sure the build doesn't do something like -fkeep-inline-functions
> (or whatever its clang counterpart is).  Building Self (the language
> that javascript got the inspiration from and java got its hotspot vm)
> used to take ages and each object file was measured in the hundreds of
> kilobytes, b/c each object file ended up with copies of every inline
> method in every class #include'd in the source.  Then another eternity
> would be spent by the linker g/c'ing/coalescing all the hundres of
> copies of those methods found in each .o file.
>
> -uwe

The actual compiler invocation was (from "ps axdww" output,
quoted line wraps for readability):

/usr/pkgsrc/wip/rust199/work/.gcc/bin/g++ \
	-fstack-protector-strong -D_FORTIFY_SOURCE=2 \
	-Wl,-zrelro -fPIC -D_GLIBCXX_USE_CXX11_ABI=1 \
	-D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS \
	-D__STDC_LIMIT_MACROS \
	-I/usr/pkgsrc/wip/rust199/work/rustc-1.99.0-src/build/sparc64-unknown-netbsd/llvm/build/lib/Target/RISCV \
	-I/usr/pkgsrc/wip/rust199/work/rustc-1.99.0-src/src/llvm-project/llvm/lib/Target/RISCV \
	-I/usr/pkgsrc/wip/rust199/work/rustc-1.99.0-src/build/sparc64-unknown-netbsd/llvm/build/include \
	-I/usr/pkgsrc/wip/rust199/work/rustc-1.99.0-src/src/llvm-project/llvm/include \
	-isystem /usr/pkgsrc/wip/rust199/work/.buildlink/include \
	-O2 -ffunction-sections -fdata-sections -fPIC -gz -O2 \
	-I/usr/pkgsrc/wip/rust199/work/.buildlink/include \
	-I/usr/pkgsrc/wip/rust199/work/.buildlink/include/libxml2 \
	-fdebug-prefix-map=/usr/pkgsrc/wip/rust199/work/rustc-1.99.0-src=/rustc/llvm \
	-fPIC -fno-semantic-interposition -fvisibility-inlines-hidden \
	-Werror=date-time -ffunction-sections -fdata-sections \
	-O3 -DNDEBUG -std=c++17 -fvisibility=hidden -fno-exceptions \
	-funwind-tables -fno-rtti -MD \
	-MT lib/Target/RISCV/CMakeFiles/LLVMRISCVCodeGen.dir/GISel/RISCVInstructionSelector.cpp.o \
	-MF CMakeFiles/LLVMRISCVCodeGen.dir/GISel/RISCVInstructionSelector.cpp.o.d \
	-o CMakeFiles/LLVMRISCVCodeGen.dir/GISel/RISCVInstructionSelector.cpp.o \
	-c /usr/pkgsrc/wip/rust199/work/rustc-1.99.0-src/src/llvm-project/llvm/lib/Target/RISCV/GISel/RISCVInstructionSelector.cpp \
	-specs=/usr/pkgsrc/wip/rust199/work/.cwrapper/specs.libgcc

No -fkeep-inline-functions to see here, at least.

And, sadly, the build bombed in the end on the same file, not
with "out of memory" error, but:

[ 71%] Building CXX object lib/Target/RISCV/CMakeFiles/LLVMRISCVCodeGen.dir/GISel/RISCVInstructionSelector.cpp.o
during GIMPLE pass: forwprop
In file included from /usr/pkgsrc/wip/rust199/work/rustc-1.99.0-src/src/llvm-project/llvm/lib/Target/RISCV/GISel/RISCVInstructionSelector.cpp:18:
/usr/pkgsrc/wip/rust199/work/rustc-1.99.0-src/src/llvm-project/llvm/include/llvm/CodeGen/GlobalISel/GIMatchTableExecutorImpl.h: In function 'bool llvm::GIMatchTableExecutor::executeMatchTable(TgtExecutor&, MatcherState&, const ExecInfoTy<PredicateBitset, ComplexMatcherMemFn, CustomRendererFn>&, llvm::MachineIRBuilder&, const uint8_t*, const llvm::TargetInstrInfo&, llvm::MachineRegisterInfo&, const llvm::TargetRegisterInfo&, const llvm::RegisterBankInfo&, const PredicateBitset&, llvm::CodeGenCoverage*) const [with TgtExecutor = const {anonymous}::RISCVInstructionSelector; PredicateBitset = llvm::Bitset<125>; ComplexMatcherMemFn = std::optional<llvm::SmallVector<std::function<void(llvm::MachineInstrBuilder&)>, 4> > ({anonymous}::RISCVInstructionSelector::*)(llvm::MachineOperand&) const; CustomRendererFn = void ({anonymous}::RISCVInstructionSelector::*)(llvm::MachineInstrBuilder&, const llvm::MachineInstr&, int) const]':
/usr/pkgsrc/wip/rust199/work/rustc-1.99.0-src/src/llvm-project/llvm/include/llvm/CodeGen/GlobalISel/GIMatchTableExecutorImpl.h:43: internal compiler error: Segmentation fault
   43 | bool GIMatchTableExecutor::executeMatchTable(
      | 

:(

Perhaps re-try with gcc16? (Yes, I'm grasping at straws, I'm
already forcing gcc14 for sparc64.)

I've managed to make a post-processed file, and it's Huge, at
least by my habits:

s1: {11} ls -l ~/RISCVInstructionSelector.cpp.E
-rw-r--r--  1 he  users  62191074 Oct 12 17:38 /home/he/RISCVInstructionSelector.cpp.E
s1: {12} wc -l ~/RISCVInstructionSelector.cpp.E
 1196883 /home/he/RISCVInstructionSelector.cpp.E
s1: {13} 
s1: {13} ls -lh ~/RISCVInstructionSelector.cpp.E
-rw-r--r--  1 he  users   59M Oct 12 17:38 /home/he/RISCVInstructionSelector.cpp.E
s1: {14} 

The file from 19% to 99% consists of this function which has a
single static table inside of it:

const uint8_t *RISCVInstructionSelector::getMatchTable() const {

  constexpr static uint8_t MatchTable0[] = {
....
  };
  return MatchTable0;
}


Best regards,

- Håvard