Re: aligning control message ancillary data

[Taylor R Campbell <campbell%mumble.net@localhost>]

After some further thought and discussion on other forums, I think
that I have worked out what the right usage is here.  OpenBSD has a
man page for CMSG_DATA & friends, which on OpenBSD-current (but not
4.3) shows what the OpenBSD folks believe to be, and what I agree is,
the correct idiom.  It would be very helpful, I think, if NetBSD had a
similar man page.  Here is what I ought to have written in my first
message as the `correct' idiom (although it won't work on NetBSD
currently; see below for the two reasons):

   struct msghdr msg;
   struct cmsghdr *cmsg_ptr;
   union {
     struct cmsghdr align_me_please;
     char data [CMSG_SPACE (sizeof (int))];
   } control;

   msg.msg_control = (void *) &control;
   msg.msg_controllen = sizeof (control);
   /* ... */
   cmsg_ptr = CMSG_FIRSTHDR (&msg);
   cmsg_ptr->cmsg_len = CMSG_LEN (sizeof (int));

NetBSD's CMSG_SPACE does not expand to a constant expression (nor does
CMSG_LEN).  While one can allocate objects on the stack with
non-constant sizes, using C99 or alloca(3), this is very sketchy.
Hence monitor_fdpass.c does not do this.  Instead...it picks a random
number that is hoped large enough for most people, and checks that it
is actually large enough at run-time.  I have to wonder whether this
is really better than just dynamically allocating the control buffer.
Some (e.g., Theo) suggest that CMSG_SPACE and CMSG_LEN are wrong to
expand to non-constant expressions.

The other issue with the above code is that currently the NetBSD
kernel requires that the msg_controllen field of a msghdr be equal to
the cmsg_len field of the sole cmsghdr in the control buffer for
SCM_RIGHTS-type control messages.  Thus if CMSG_SPACE (sizeof (int))
is not equal to CMSG_LEN (sizeof (int)) because of wider alignment
than what one finds on the i386, and you run the above code on NetBSD,
sendmsg will yield EINVAL.  This, I believe, is wrong, and caused by
the following fragment from unp_internalize in kern/uipc_usrreq.c
starting on line 1292:

   /* Sanity check the control message header. */
   if (cm->msg_type != SCM_RIGHTS || cm->msg_level != SOL_SOCKET ||
       cm->msg_len != control->m_len)  /* <- This is the problem. */
     return (EINVAL);

The OpenBSD developers who thoroughly scrutinized this several months
ago believe the kernel should check whether CMSG_ALIGN (cm->msg_len)
is inequal to control->m_len.  But doing this might break a bunch of
existing code that used the incorrect yet working idiom.  So in both
FreeBSD as of many years ago, and in OpenBSD-current as of a few
months ago, the analogous code checks whether cm->msg_len is simply
greater than control->m_len, which is clearly an error.