[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
Re: using the interfaces in ctype.h
On 21-Apr-08, at 12:09 PM, Alan Barrett wrote:
On Mon, 21 Apr 2008, Greg A. Woods; Planix, Inc. wrote:
If the implementation masked the value before using it, then it
unable to distinguish EOF from UCHAR_MAX (typically '\377').
Indeed, however the current implementation doesn't even try to
"distinguish" EOF, and indeed passing EOF without casting it properly
and/or masking will result in an out-of-bounds array access in the
What are you smoking? The use of constructs like
(_ctype_ + 1)[c]
in NetBSD's implementation (both in the macros defined in ctype.h, and
in the C code defined in libc/gen/isctype.c) will access _ctype_
c == -1, and -1 happens to be the value that NetBSD used for EOF.
Ah, right, OK, sorry, my mistake. However that's really just a
pedantic point irrelevant to my main argument. So, -1, which in our
case happens to be EOF, is OK.
However that does nothing to help for any other negative values.
Assuming that the caller will only use -1 or a value between 0 and
_CTYPE_NUM_CHARS is not safe when the implementation is accessing an
array of only _CTYPE_NUM_CHARS+1 and the prototype for the API
specifies a parameter of type "int". If the implementation were an
inline function that could protect the array from out-of-bounds access
then that would be fine, but it's not on NetBSD.
Since masking inside the
implementation would violate the requirement to distinguish EOF from
UCHAR_MAX, it's good that NetBSD doesn't do that.
Huh? That makes no sense whatsoever.
For example (assuming 8-bit chars), if the implementation did the
c = c & 0xff;
before it used the value of c, then inputs of -1 (EOF) and 0xff (a
perfectly valid unsigned char, not the same as EOF) would both be
changed to 0xff, making it impossible for the rest of the code to
distinguish between these two inputs.
Huh? In NetBSD both (_ctype_+1)[-1]==0 and (_ctype_+1)[0xff]==0 so
what's to be distinguished?!?!?!?
Furthermore how's that any different than suggesting that the caller
cast the parameter with "(unsigned char)" or "(int)(unsigned char)"?
The cast still causes the passed value to be effectively masked with
0xFF and so even if the implementation did want to distinguish a
character of 0xFF from the value of EOF it could not.
You gain far more by building the cast into the implementation rather
than effectively forcing the application to employ it. At least with
it built in then the application won't thwart any future or alternate
implementation from detecting EOF before doing anything else with the
FreeBSD, OpenBSD, and Darwin all seem to have much better
implementations, though they are all using proper (inline)
which makes it easier in some ways to do it right.)
I am mildly curious. In what way are they "better"?
Well they can't as easily be responsible for causing a program to
You haven't shown an example, and I don't know what these other
If you'd like I can point you at HTTP accessible copies of the other
implementations if necessary....
Anyway, I don't subscribe to the theory that it's
"better" for the implementation to go out of its way to prevent an
erroneous program from crashing; I thhik that erroneous programs
However, making it crash with a useful error message and an
abort() is more friendly than just pressing on with bad data.
I would agree entirely though I suspect there are many folks who would
disagree (witness the outrage when assert() was sprinkled elsewhere
about in libc). However the NetBSD implementation doesn't even try --
it just behaves naively and may then access memory outside the defined
object's allocated storage. At least with a built-in mask on the
array access value nothing untoward can happen.
The more expensive inline function style of implementation would
afford both better ways of forcing an application to abort, as well as
better ways of safely ignoring values out of range, thus offering the
ideal solution to both our desire to force broken applications to
crash as well as the desire of others to treat them benignly and allow
them to run safely.
For my own use the built-in mask affords the latter solution
transparently to applications, and without having to hack too much of
the NetBSD code, so that's the way I'll go for the near term.
I recommend the following slightly more portable technique for
#define _CTYPE_MASK ~(UINT_MAX << CHAR_BIT)
I believe that that's identical to UCHAR_MAX, given the way unsigned
arithmetic works, and that UCHAR_MAX+1 is guaranteed to be equal to
Yes it may be true that UCHAR_MAX has the same value as my mask, at
least in NetBSD, but that's not how _ctype_ is defined in NetBSD.
_ctype_ is defined in terms of CHAR_BIT, so the definition I chose is
more readable and more logical (in my opinion, of course) than using
any other unrelated constant or macro referring to an unrelated
constant, and thus both the mask used to access _ctype_ and the
definition of _ctype_ itself are simultaneously dependent on the same
macro and independent of UCHAR_MAX. However perhaps my definition
#define _CTYPE_MASK ~(~0UL << CHAR_BIT)
just to be pedantic and portable and to avoid any reference to any
#define isdigit(c) ((int)(_ctype_ + 1)[((c) & _CTYPE_MASK)] & _N))
That's just wrong, as I explained before. Given two distinct inputs
c == EOF (0xffffffff, if int is 32 bits) and c == UCHAR_MAX (0xff, if
char is 8 bits), the results from ((c) & _CTYPE_MASK) will be 0xff in
both cases, so the macro will be unable to distinguish between the two
inputs. OK, '\xff' doesn't happen to be a digit in any character set
that I know about, so it doesn't matter in this particular case, but
cases in which it does matter are easy to imagine.
In fact with the implementation of the NetBSD "ctype" is*() and to*()
APIs, nothing outside of the proper range of ASCII is meaningful and
so 0xFF is always outside the range of valid inputs.
Hang on, it's even worse than that. The C standard allows signed
integers to have a representation other than two's complement. The
result of (-1 & 0xff) on a one's complement machine will be 0xfe, not
0xff. NetBSD might not run on any one's complement machines, but I
to consider them when writing code that's intended to be portable.
Your tangent about running on systems not supporting two's complement
is interesting, however I think it is well outside and beyond the
context of NetBSD, which I would humbly suggest will not run on any
such hardware any time soon and without vast effort on both the OS
side of things as well as within many applications which use the
"ctype" APIs. That's a boat that sailed off and sank quite some time
Greg A. Woods; Planix, Inc.
Main Index |
Thread Index |