Subject: 2.0RC4: defeating non-exec stack
To: None <tech-kern@netbsd.org>
From: der Mouse <mouse@Rodents.Montreal.QC.CA>
List: tech-kern
Date: 11/08/2004 17:33:42
At $DAYJOB, I'm working with a box with a "hardware" (firmware, really)
RAID array a bit over 1TB.  1.6.2 can't handle that; 2.0_RC4 (the most
recent i386 kernel I found on ftp.netbsd.org) can.

So I was booting the 2.0_RC4 kernel with a fully 1.6.2 userland.  And
one of my tools started coredumping on me.  On investigation, this
proves to happen exactly when it tries to enter the stack trampoline
that implements a nested function.

Now, I think I understand why non-executable stack is seen as a good
thing.  And presumably the 2.0RC4 compiler does whatever tapdancing is
necessary to make nested functions work.  But this breaking of backward
compatability is a problem.  How do I disable this misfeature so I can
run my 1.6.2 executables?  I did a little sysctl -a | grep searching
and found nothing; perhaps I just didn't try the right grep pattern -
or perhaps it's that I was using the 1.6.2 sysctl, though a machine I
can get a shell on running 2.0_BETA didn't have anything
helpful-looking in its sysctl -a output.

Note that I don't have a 2.0 build environment, so anything that
involves recompiling (eg) the kernel is not workable.  The kernel is
the GENERIC_DIAGNOSTIC kernel from
/pub/NetBSD-daily/netbsd-2-0/200410160000/i386 on ftp.netbsd.org.

/~\ The ASCII				der Mouse
\ / Ribbon Campaign
 X  Against HTML	       mouse@rodents.montreal.qc.ca
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B