Subject: Re: kern/30008 & NetBSD 3.0_BETA: "cannot enable executable stack"
To: Christos Zoulas <christos@astron.com>
From: Chuck Silvers <chuq@chuq.com>
List: current-users
Date: 10/25/2005 23:12:02
ok, I see what's going on now.  linux mprotect() has a prot flag called
PROT_GROWSDOWN which will change the protection on an entire mapping
without needing to know how big that mapping is.  glibc tries to use this
flag to make the entire stack executable in one call, but our linux emulation
doesn't support this flag.  so glibc falls back on a loop calling mprotect()
on 8 page ranges until it gets ENOMEM, which I guess means that the range
was not entirely mapped.  however, we get EACCES instead of ENOMEM:

...
  7276 gtk-query-immodu CALL  mprotect(0xbbbf7000,0x8000,7)
  7276 gtk-query-immodu RET   mprotect 0
  7276 gtk-query-immodu CALL  mprotect(0xbbbef000,0x8000,7)
  7276 gtk-query-immodu RET   mprotect 0
  7276 gtk-query-immodu CALL  mprotect(0xbbbe7000,0x8000,7)
  7276 gtk-query-immodu RET   mprotect -1 errno -13 Permission denied


with the topdown address space arrangement, we put other stuff right below
the stack, so the glibc loop walks off the end of the stack and starts
changing those other mappings too:

BB999000   2320K read/exec         /usr/pkg/emul/linux/opt/gnome/lib/libgtk-x11-2.0.so.0.200.4
BBBDD000     32K read/write        /usr/pkg/emul/linux/opt/gnome/lib/libgtk-x11-2.0.so.0.200.4
BBBE5000     12K read/write          [ anon ]
BBBE8000     12K read              /usr/pkg/emul/linux/etc/ld.so.cache
BBBEB000      4K read/exec           [ uvm_aobj ]
BBBEC000     76K read/exec         /usr/pkg/emul/linux/lib/ld-2.3.3.so
BBBFF000      4K read/write          [ anon ]
BBC00000  43292K                     [ stack ]
BE647000     32K read/write/exec     [ stack ]
BE64F000     32K read/write/exec     [ stack ]
BE657000     32K read/write/exec     [ stack ]
BE65F000     32K read/write/exec     [ stack ]
BE667000     32K read/write/exec     [ stack ]
BE66F000     32K read/write/exec     [ stack ]
...

(this is where I caught it during the loop, there are hundreds more
32K chunks after this.)

when the loop hits the read-only mapping of ld.so.cache,
mprotect() fails since the maxprot of that range doesn't allow all access,
and glibc interprets the EACCES as a fatal error.

so to fix this properly, our linux emulation code needs to handle
the PROT_GROWSDOWN flag (and PROT_GROWSUP too, for hppa).
does someone else want to implement this or should I do it?

-Chuck