fixing libpthread noload, another approach

[manu%netbsd.org@localhost (Emmanuel Dreyfus)]

For whoever missed the previous episodes: Starting with NetBSD 6.0,
libpthread has a NOLOAD flag, which will cause dlopen() to refuse
loading it in programs that are not linked with -lpthread.

The rationale is that many libc functions (e.g.: malloc) use mutex and
condvar to be thread-safe. When not linking with -lpthread, they use
no-op libc stubs. When lining with -lpthread, they use libpthread full
implementation. if dlopen() of libpthread is allowed, one can start
initializing and holding a mutex as a no-ops, load libpthread, and then
really attempt to release the unintialized mutex and crash.

The problem of NetBSD 6.0 approach is that some DSO may hold an unneeded
and unwanted libpthread dependency through a third party library, and
therefore get impossible to load by NetBSD 6.0 dlopen(). For instance,
if a PAM module links against libgobject-2.0.so, it gets a libpthread
dependency and is rejected by dlopen() even if it never uses thread
related stuff.

A first proposal to fix the problem has been to implement minimal mutex
and condvar operations in libc stubs: if they initialize enough fields,
then operations can resume safely after libpthread is loaded. The
drawback of this approach is that there is a performance hit for
programs that are not linked with -lpthread.

Here is another proposal: modify dlopen() so that instead of raising an
error when encountering an object tagged with NOLOAD, it just resolves
its symbol to an error stub. Therefore we are able to dlopen() a DSO
linked with -lpthread, but libpthread is not loaded. If we do not use
thread-related stuff, everything is fine, but if we call a function from
libpthread, we abort with a diagnostic.

The advantage is that there is no performance hit for programs not
linked with -lpthread: we leave the libc thread stub as no-ops. The
overhead only happens in dlopen() when we have to load and unload an
object that has the NOLOAD flag.

Here is the patch that implement the thing:
http://ftp.espci.fr/shadow/manu/dlopen.patch

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu%netbsd.org@localhost