In some email I received from Chris G. Demetriou, sie wrote:
> 
> Darren Reed <darrenr@reed.wattle.id.au> writes:
> > To do this testing,
> > I'm using the list of errors on the man page for each system call and
> > attempting to call them (with an appropriate environment) in such a way
> > that they do fail as described.  Given some of the comments people have
> > made about what manual pages are meant to document, I'm pondering whether
> > or not this is the `correct' approach.
> 
> I'm not convinced it is.

I'm open to suggestion for other methods to achieve the same goal.

> first of all, there's the issue of conditions which may or may not be
> detected as part of the code run in the kernel (e.g. the EFAULT
> vs. SEGV), which you've already noticed.  i.e. you're checking error
> input vs. guaranteed type of error return, which is backwards (at
> least for some types of bogus input).

In some sense, "input" is more than just parameters passed to system
calls.

> there are also other issues where the manual pages are simply not
> complete, and therefore your testing will not be complete if you just
> follow the manual pages.

Then this is an issue for documentation to be corrected.  I don't see
how one can reasonably test something that isn't documented.

> For instance, consider read():
> 
>      [EFAULT]      buf points outside the allocated address space.
> 
> Depending on how you're reading these things, either:
> 
> 	(1) you wouldn't check for reading into a buffer in the
> 	    address space but mapped read-only (e.g. read(0, &main, 1); 8-)

Does this in fact cause the above error ?

> 	(2) the kernel would be considered 'broken' to return EFAULT
> 	    for that case.

I'd agree with this, but given this, what should it return ?

> If you want to do this kind of testing, personally, i'd think you're
> best off either starting with a POSIX (or other) conformance test
> suite, or at least a standards document, and going from there, and, if
> necessary, fixing up the manual pages as you go to match.

Is there a POSIX conformance test suite or can someone pass along where I
can get the relevant standards document ?  Hmmm, if we did go down this
path, does it lead to us being able to say we are POSIX compliant ?

Darren