Darren Reed <darrenr@reed.wattle.id.au> writes:
> To do this testing,
> I'm using the list of errors on the man page for each system call and
> attempting to call them (with an appropriate environment) in such a way
> that they do fail as described.  Given some of the comments people have
> made about what manual pages are meant to document, I'm pondering whether
> or not this is the `correct' approach.

I'm not convinced it is.

first of all, there's the issue of conditions which may or may not be
detected as part of the code run in the kernel (e.g. the EFAULT
vs. SEGV), which you've already noticed.  i.e. you're checking error
input vs. guaranteed type of error return, which is backwards (at
least for some types of bogus input).

there are also other issues where the manual pages are simply not
complete, and therefore your testing will not be complete if you just
follow the manual pages.  For instance, consider read():

     [EFAULT]      buf points outside the allocated address space.

Depending on how you're reading these things, either:

	(1) you wouldn't check for reading into a buffer in the
	    address space but mapped read-only (e.g. read(0, &main, 1); 8-)

	(2) the kernel would be considered 'broken' to return EFAULT
	    for that case.

If you want to do this kind of testing, personally, i'd think you're
best off either starting with a POSIX (or other) conformance test
suite, or at least a standards document, and going from there, and, if
necessary, fixing up the manual pages as you go to match.


cgd
-- 
Chris Demetriou - cgd@netbsd.org - http://www.netbsd.org/People/Pages/cgd.html
Disclaimer: Not speaking for NetBSD, just expressing my own opinion.