Subject: Re: Regressing testing of system calls
To: Chris G. Demetriou <firstname.lastname@example.org>
From: Darren Reed <email@example.com>
Date: 06/15/1999 11:19:06
In some email I received from Chris G. Demetriou, sie wrote:
> Darren Reed <firstname.lastname@example.org> writes:
> > To do this testing,
> > I'm using the list of errors on the man page for each system call and
> > attempting to call them (with an appropriate environment) in such a way
> > that they do fail as described. Given some of the comments people have
> > made about what manual pages are meant to document, I'm pondering whether
> > or not this is the `correct' approach.
> I'm not convinced it is.
I'm open to suggestion for other methods to achieve the same goal.
> first of all, there's the issue of conditions which may or may not be
> detected as part of the code run in the kernel (e.g. the EFAULT
> vs. SEGV), which you've already noticed. i.e. you're checking error
> input vs. guaranteed type of error return, which is backwards (at
> least for some types of bogus input).
In some sense, "input" is more than just parameters passed to system
> there are also other issues where the manual pages are simply not
> complete, and therefore your testing will not be complete if you just
> follow the manual pages.
Then this is an issue for documentation to be corrected. I don't see
how one can reasonably test something that isn't documented.
> For instance, consider read():
> [EFAULT] buf points outside the allocated address space.
> Depending on how you're reading these things, either:
> (1) you wouldn't check for reading into a buffer in the
> address space but mapped read-only (e.g. read(0, &main, 1); 8-)
Does this in fact cause the above error ?
> (2) the kernel would be considered 'broken' to return EFAULT
> for that case.
I'd agree with this, but given this, what should it return ?
> If you want to do this kind of testing, personally, i'd think you're
> best off either starting with a POSIX (or other) conformance test
> suite, or at least a standards document, and going from there, and, if
> necessary, fixing up the manual pages as you go to match.
Is there a POSIX conformance test suite or can someone pass along where I
can get the relevant standards document ? Hmmm, if we did go down this
path, does it lead to us being able to say we are POSIX compliant ?