Re: aligning control message ancillary data

[Taylor R Campbell <campbell%mumble.net@localhost>]

(By the way, I am not subscribed to the list, so please cc me in
replies.)

Right now, there are a number of programs in the NetBSD tree that are
confused about what the correct idiom is.  It turns out that Stevens
documents exactly the correct idiom, and that most of the programs get
it subtly wrong because of bugs in NetBSD and OpenBSD that made the
correct idiom fail on platforms that are strict about alignment.  (Of
course, most people don't notice this because the i386 is lenient
about alignment.)

The kernel wants the cmsghdr struct *and* the datum inside the struct
to be aligned, presumably so that it can dereference their contents
without alignment traps.  Thus the control buffer must be allocated as
a union, if it is to be stack-allocated, with two elements -- a struct
cmsghdr and a char array with a size calculated using CMSG_SPACE.  The
program should then tell the kernel exactly how long is the buffer it
allocated in the msg_controllen field; this is the size of the union.
Finally, it should tell the kernel exactly how much of the buffer it
is using for file descriptors in the cmsg_len field, to be calculated
with CMSG_LEN, so that the kernel does not try to interpret bogus data
after the actual file descriptors.

Each control datum must be aligned to the width of a struct cmsghdr,
in order that a sequence of them will have each header aligned.  I
imagine that it will work to omit padding after the very last one,
however, which is why working programs use CMSG_LEN to calculate
msg_controllen.  But this is probably fragile, and there is no sense
in allocating a buffer whose length is given by CMSG_SPACE if all that
the program will pass to the kernel is CMSG_LEN bytes.  Also note that
if programs use CMSG_LEN on the *receiving* end for msg_controllen,
recvmsg will set MSG_CTRUNC, telling the programs erroneously that the
control data were truncated.

Some existing code is broken and will never work correctly on
architectures with wide alignment anyway.  Some existing code is
correct, but does not presently run on NetBSD because of the bogus
conditional in uipc_usrreq.c.  Some existing code is confused, but
will run on NetBSD.  Changing the conditional to be > instead of !=
will include the existing correct code and not exclude any existing
code that works but is confused.  I'm no NetBSD developer, of course,
so it's not my place to say this, but I believe that changing that one
conditional is the right thing (along with changing the confused
code, but that matters less).