Core statement on sysctl 32-bit/64-bit changes

[Alan Barrett <apb%netbsd.org@localhost>]

The NetBSD core group has considered the sysctl changes
made by David Laight on 23 and 27 Feb 2014 (see
<http://mail-index.netbsd.org/source-changes/2014/02/23/msg051946.html>,
<http://mail-index.netbsd.org/source-changes/2014/02/27/msg052131.html>, and
<http://mail-index.netbsd.org/source-changes/2014/03/01/msg052200.html>),
and objections raised by Andreas Gustafsson (see
<http://mail-index.netbsd.org/source-changes-d/2014/03/05/msg006587.html>,
<http://mail-index.netbsd.org/tech-kern/2014/03/05/msg016706.html>,
and subsequent discussion in the source-changes-d and tech-kern
lists).

We note that the following sysctl nodes (for the i386 and amd64 ports)
have been changed from CTLTYPE_INT (a 32-bit signed integer) to
CTLTYPE_QUAD (a 64-bit unsigned integer):

   machdep.fpu_present
   machdep.osfxsr
   machdep.sse
   machdep.sse2
   machdep.biosbasemem
   machdep.biosextmem

Both binary and source code compatibility are important for NetBSD.
If the types of sysctl variables are changed, then we would want it to
be done in such a way that old code continues to work.  We note that
there has been an attempt to provide compatibility, by allowing 32-bit
sysctl variables to be read into 64-bit buffers, and vice versa, but
we are concerned that this mechanism was introduced without prior
discussion, and there may be cases where compatibility is lost.

Several of the affected sysctl variables appear to be essentially
boolean in nature, and there appears to be no good reason for the
variables to be exposed to userland as 64-bit values.  It's not clear
why variables that are logically boolean (such as machdep.fpu_present)
were ever defined as CTLTYPE_INT instead of CTLTYPE_BOOL, but it does
seem clear that changing them to CTLTYPE_QUAD is not an improvement.

Two of the variables, machdep.biosbasemem and machdep.biosextmem,
represent memory sizes, and it is possible that 32 bits might not
be large enough for them.  For these variables, changing the size
to 64 bits, or adding new 64-bit variables in parallel with the
old variables, may be appropriate.  In the past, new 64-bit sysctl
variables hw.physmem64 and hw.usermem64 were introduced in parallel to
the older 32-bit variables hw.basemem and hw.usermem.  We have heard
suggestions that the same should be done for machdep.biosbasemem and
machdep.biosextmem, if 32 bits is not sufficient for those variables.
We have also heard suggestions that increasing the size of the
variables would be preferable to adding new variables with different
names, provided that it was done in a compatible way.

The core group now recommends as follows:

1. In the short term, the affected sysctl variables should be changed
back to their original type, and the compatibility code should be
removed.

2. sysctl variables should not be wider than necessary.  If there is
no need for the existing variables to be made wider than 32 bits, then
they should not be made wider than 32 bits.

3. If some existing variables need to be made wider, then
consideration should be given to either:

a) adding new 64-bit sysctl variables in parallel to the existing
   32-bit variables (such as adding a new machdep.biosextmem64 in
   parallel to the existing machdep.biosextmem); or

b) adding new infrastructure for 32-bit/64-bit compatibility, and
   using that infrastructure.

4. If new infrastructure is considered, to allow reading 64-bit sysctl
variables into 32-bit buffers, then the design and implementation
should be discussed in public.  Some considerations that we would like
to see addressed are:

a) What types are needed?  Currently, CTLTYPE_INT is a signed
   32-bit type, and CTLTYPE_QUAD is an unsigned 64-bit type.
   Perhaps all four possible combinations of signed/unsigned and
   32 bits/64 bits should be supported.

b) Should the ability to read values with a different size apply
   to all sysctl variables, or only to those defined in a special
   way?  For example, there could be a new CTLFLAG_COMPAT32 flag
   that allows reading a 64-bit value into a 32-bit buffer.

c) What is the appropriate error return when a 64-bit value is too
   large to fit in a user-provided 32-bit buffer?

d) Will old code still work without change?

e) Will new userland code be able to detect the presence of wider
   sysctl variables with 32-bit compatibility?

f) Is coordination with other projects using the sysctl(3) or
   sysctl(9) interface needed?

g) Are the new interfaces adequately documented?

--
Alan Barrett,
on behalf of the NetBSD core group