tech-userlevel archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Patch to make <stdio.h> reentrant by default

<stdio.h> traditionally (at least in the BSD tradition) contained
optimization for !_REENTRANT and !threaded code in <stdio.h>.

The header was shadowing real functions such as fileno(3) or getc(3)
with inlined non-reentrant ones.

This caused issues with C++ code especially C++11 (notably std::thread)
and was explicitly disabled by Joerg back years ago.

Non-reentrant inlined functions also caused damage to sanitizers (and
other related software) that link transparently with -lpthread without
defining _REENTRANT in preprocessor.

This caused mixing threaded and !reentrant code in sanitizers and caused
breakage in programs using these functions with double variations. The
most sensitive sanitizer to this is MSan.

As a workaround we have added in the clang's NetBSD driver explicit
-D_REENTRANT for all sanitizers to work around the problem on NetBSD and
pick threading safe functions. The same workaround hasn't been
propagated to GCC so far.

We keep detecting that more software is happy to just pick -lpthread
(like LLVM OpenMP) and prebuilt software works by an accident.

Catching every -lpthread user worldwide and teaching about -pthread is a
losing battle. My current observation is that people (in toolchains!)
are surprised that they need to link with -lpthread for C++ threads.

I propose a patch that:

1. Removes alternative versions of <stdio.h> depending on preprocessor
namespace definitions. All code since now is assumed to be
reentrant/threading safe.

2. Adds explicit non-reentrant versions of fileno(), ferror(3), feof(3)
and clearerr(3) called fileno_unlocked(), ferror_unlocked(),
feof_unlocked(), clearerr_unlocked().

These routines have users in GNU tools and are implemented in at least
macOS, Linux and FreeBSD. They are also documented in the GNU world as
BSD extensions.

New unlocked() functions are deliberately implemented as libc symbols in
order to pick them easily from libc and don't break existing software
calling code from pregenerated configure scripts, such as "#define
fileno_unlocked fileno" in base's GCC. (With next upgrade of GCC I
expect GCC to switch to native _unlocked() variations).

Implementing these routines as light functions inside libc is also
sanitizer friendly as inlined code that interacts directly with stdio.h
buffers is more difficult to sanitize. Even if we would want to sanitize
such scenarios we still want _REENTRANT functions only as the LLVM
runtimes of sanitizers are threaded.

3. Not introduces any extra _unlocked() functions, even if they are
available in other systems, as they weren't available in the original
NetBSD stdio.h magic macros.

4. Adapts surrounding code, documentation, etc.

5. Not changes the alternative semantics of _REENTRANT in other headers,
as there this symbol is used as a feature detection for _r() functions
(not sure if there are any users of this semantics, but I'm not changing

6. Defines h_errno in <netdb.h> in a reentrant way always.

7. Silently drops dead feature check _PTHREAD. It is still generated by
GCC for -pthread, but it was never worth to get into Clang/LLVM and has
0 users in worlwide open-source software (according to Debian Code search).

There wre recurring internal discussions about this with at least Joerg,
Christos and Michal (Gorny).

If we want to optimize code for 1 thread, we can now pick _unlocked()
functions explicitly in the basesystem software.

This is something I would like to see in -9.

All ATF tests pass with the proposed patch.

Attachment: signature.asc
Description: OpenPGP digital signature

Home | Main Index | Thread Index | Old Index