makesyscalls (moving forward)

[David Holland <dholland-tech%netbsd.org@localhost>]

As I mentioned a few days ago the reason I was prodding
makesyscalls.sh is that I've been looking at generating more of the
system call argument handling code.

It seems to me that all of the following is mechanical and should be
automatically generated, beyond what makesyscalls already does:
   - all the code that calls copyin/copyout
   - compat32 translation for all syscalls and all ioctls
   - compat_otheros translation as well

The first of these has been routine in research OSes for 25+ years and
offers no particular difficulties other than it'll take a fair amount
of coding. The second and third follow directly from a good solution
to the first, modulo some semantic concerns about schema translation
that I believe aren't relevant for the things we need to do. (As in,
anything semantically complicated is going to end up being written by
hand anyway and the translation itself is about different
representations of the same data.)

Past experience (this will not be the first or I think even the second
syscall marshalling generator I've had the pleasure of writing) says
is that doing this well requires real compiler tooling and
infrastructure. The copyin/copyout code can be, but should not be,
generated with just an awk script. Going beyond that requires a real
tool with sources that get compiled.

Meanwhile looking at the large amount of cutpaste in the current
makesyscalls.sh, the number of similar output files already generated,
and talking with Kamil about what's needed by sanitizers, makes me
think that at least part of the generation should be driven by output
templates. That is, you run some tool and you feed it an output
template that describes one set of things and you get the in-kernel
copyin/copyout code. Then, you feed it some other template and out
comes the rump version, a third template and you get sanitizer
wrappers, etc. etc. I will need to think some about how to do this
effectively (it is not as straightforward as just spewing code out)
but I don't think it is going to be particularly difficult.

However, it means that the tool needs to be installed in the system
and so does the master description file for at least NetBSD's own
system calls. (Currently, it seems that the llvm sanitizers build
reads syscalls.master itself from a source tree, so it needs a system
source tree to build, which is bad.)

This raises two points that need to be bikeshedded:

(1) What's the new tool called, and where does it live in the tree?
"usr.bin/makesyscalls" is fine with me but ymmv.

(2) What is the installed syscall description file called and where
does it go? It is likely to be derived from (i.e., not the same as)
syscalls.master. It isn't a C header file so it doesn't belong in
/usr/include. It isn't necessarily particularly human-readable. My
suggestion would be to add a directory in /usr/share for API
descriptions and put it there, something like maybe
/usr/share/api/syscalls.def.


Note that the eventual tool that gets committed will be written in C,
but the development version will probably be written in something with
stronger types. For this reason I expect to not be committing
development versions until things are pretty much done.

-- 
David A. Holland
dholland%netbsd.org@localhost