Subject: Re: "virtual memory exausted" building netbsd-1-6 for sparc on alpha
To: NetBSD/alpha Discussion List <port-alpha@NetBSD.ORG>
From: Greg A. Woods <woods@weird.com>
List: tech-toolchain
Date: 11/16/2004 13:03:42
[ On Saturday, November 8, 2003 at 00:01:47 (+0100), Martin Husemann wrote: ]
> Subject: Re: "virtual memory exausted" building netbsd-1-6 for sparc on alpha
>
> On Fri, Nov 07, 2003 at 05:22:58PM -0500, Greg A. Woods wrote:
> > I'm trying a cross-build of netbsd-1-6 for sparc on my alpha.  It gets this far:
> > 
> > /build/woods/building/NetBSD-1.6.x-alpha-sparc-tools/bin/sparc--netbsdelf-gcc -O2 -g -DALL_STATE -DUSG_COMPAT -pipe -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wno-uninitialized -Wreturn-type -Wpointer-arith -Wswitch -Wshadow  -Werror  -D_LIBC -DLIBC_SCCS -DSYSLIBC_SCCS -D_REENTRANT -I/proven/work/woods/m-NetBSD-1.6/lib/libc/include -DINET6 -DNLS -D__DBINTERFACE_PRIVATE -I/proven/work/woods/m-NetBSD-1.6/lib/libc/../../libexec/ld.elf_so -I/proven/work/woods/m-NetBSD-1.6/lib/libc/dlfcn -DI18NMODULE_MAJOR=4 -DWITH_RUNE -I/proven/work/woods/m-NetBSD-1.6/lib/libc -DRESOLVDEBUG -DRESOLVSORT -I. -DPOSIX_MISTAKE -DPORTMAP -DFLOATING_POINT -nostdinc -isystem /build/woods/building/NetBSD-1.6.x-sparc-destdir/usr/include  -c /proven/work/woods/m-NetBSD-1.6/lib/libc/hash/sha1.c
> > /proven/work/woods/m-NetBSD-1.6/lib/libc/hash/sha1.c: In function `_SHA1Transform':
> > /proven/work/woods/m-NetBSD-1.6/lib/libc/hash/sha1.c:206: virtual memory exhausted
> 
> Gcc 2.95.x had lots of problems compiling sha* code for sparc on 64bit 
> machines, most sha* places in our kernel have #ifdef __sparc64__ versions
> that split otherwise inlined code into smaller procedures for this reason.
> 
> Seems like host=alpha target=sparc triggers the same problem.

OK, so a year later (since 2.0 still isn't out and I'm still stuck with
gcc-2.95.3 in the netbsd-1-6 branch tools directlry :-) I've once again
been experimenting with using my alpha's spare CPU cycles to cross-build
NetBSD-1.6.x for sparc.  I've come up with some very simple little
patches to a couple of makefiles so that I can compile sha1.c with
different optimizer flags.

(The frustrating part is finding out just how many times this one little
source file gets compiled during a build -- at least 8 times, depending
on how many kernels are built!  Even more annoying is the fact that all
of libkern gets built for every bootblock and every kernel.  You'd think
one libkern.a would suffice!  I can understand needing to build it fresh
with a stand-alone kernel build, but surely the build.sh infrastructure
should be able to manage to compile just one!  What good is a library if
it's never used more than once?)

Foolishly I've been trying to get away with using "-O" (i.e. "-O1"),
which seemed to work until I got through to building the INSTALL kernel
where it failed.  Just for fun I've tried the failing one again with the
different -D options (and even with "env -i" to avoid any hidden side
effects and to shave a few more bytes of space), but it never works.  I
see no relevant difference between the failed one and the working ones!

Here are three successful compiles grep'ed from my build.log followed by
the failing one.

/build/woods/building/NetBSD-1.6.x-alpha-sparc-tools/bin/sparc--netbsdelf-gcc -O -mcpu=v8 -mtune=supersparc -DLOCKDEBUG -mno-fpu -ffreestanding -pipe -Werror -Wall -Wno-main -Wno-format-zero-length -Wpointer-arith -Wmissing-prototypes -Wstrict-prototypes -Wno-uninitialized -I/work/woods/m-NetBSD-1.6/sys/lib/libkern/arch/sparc -I../../. -I/work/woods/m-NetBSD-1.6/sys/arch -I/work/woods/m-NetBSD-1.6/sys -nostdinc -DDIAGNOSTIC -DNMBCLUSTERS="0x8000" -DMAXUSERS=32 -D_KERNEL -D_KERNEL_OPT -nostdinc -isystem /build/woods/building/NetBSD-1.6.x-sparc-destdir/usr/include -c /work/woods/m-NetBSD-1.6/sys/lib/libkern/sha1.c

/build/woods/building/NetBSD-1.6.x-alpha-sparc-tools/bin/sparc--netbsdelf-gcc -O -mcpu=v8 -mtune=supersparc -DLOCKDEBUG -mno-fpu -ffreestanding -pipe -Werror -Wall -Wno-main -Wno-format-zero-length -Wpointer-arith -Wmissing-prototypes -Wstrict-prototypes -Wno-uninitialized -I/work/woods/m-NetBSD-1.6/sys/lib/libkern/arch/sparc -I../../. -I/work/woods/m-NetBSD-1.6/sys/arch -I/work/woods/m-NetBSD-1.6/sys -nostdinc -DDIAGNOSTIC -DNMBCLUSTERS="0x8000" -DMAXUSERS=32 -D_KERNEL -D_KERNEL_OPT -nostdinc -isystem /build/woods/building/NetBSD-1.6.x-sparc-destdir/usr/include -c /work/woods/m-NetBSD-1.6/sys/lib/libkern/sha1.c

/build/woods/building/NetBSD-1.6.x-alpha-sparc-tools/bin/sparc--netbsdelf-gcc -O -mcpu=v8 -mtune=supersparc -DLOCKDEBUG -mno-fpu -ffreestanding -pipe -Werror -Wall -Wno-main -Wno-format-zero-length -Wpointer-arith -Wmissing-prototypes -Wstrict-prototypes -Wno-uninitialized -I/work/woods/m-NetBSD-1.6/sys/lib/libkern/arch/sparc -I../../. -I/work/woods/m-NetBSD-1.6/sys/arch -I/work/woods/m-NetBSD-1.6/sys -nostdinc -DDIAGNOSTIC -DNMBCLUSTERS="0x8000" -DMAXUSERS=128 -D_KERNEL -D_KERNEL_OPT -nostdinc -isystem /build/woods/building/NetBSD-1.6.x-sparc-destdir/usr/include -c /work/woods/m-NetBSD-1.6/sys/lib/libkern/sha1.c

/build/woods/building/NetBSD-1.6.x-alpha-sparc-tools/bin/sparc--netbsdelf-gcc -O -mcpu=v8 -mtune=supersparc -DLOCKDEBUG -mno-fpu -ffreestanding -Os -Werror -Wall -Wno-main -Wno-format-zero-length -Wpointer-arith -Wmissing-prototypes -Wstrict-prototypes -Wno-uninitialized -I/work/woods/m-NetBSD-1.6/sys/lib/libkern/arch/sparc -I../../. -I/work/woods/m-NetBSD-1.6/sys/arch -I/work/woods/m-NetBSD-1.6/sys -nostdinc -DMAXUSERS=32 -D_KERNEL -D_KERNEL_OPT -nostdinc -isystem /build/woods/building/NetBSD-1.6.x-sparc-destdir/usr/include -c /work/woods/m-NetBSD-1.6/sys/lib/libkern/sha1.c
/work/woods/m-NetBSD-1.6/sys/lib/libkern/sha1.c: In function `SHA1Transform':
/work/woods/m-NetBSD-1.6/sys/lib/libkern/sha1.c:175: virtual memory exhausted


Anyway, back to the build with "-O0" this time....  :-)

Very soon now I should be able to report success or fail of the whole
cross-built release.....

-- 
						Greg A. Woods

+1 416 218-0098                  VE3TCP            RoboHack <woods@robohack.ca>
Planix, Inc. <woods@planix.com>          Secrets of the Weird <woods@weird.com>