`root' on cgd with sysctl init.root

[Taylor R Campbell <campbell+netbsd%mumble.net@localhost>]

(I am not subscribed to this list, so please cc me in replies.)

I recently set up NetBSD on a laptop with `root' on cgd using a
minimal unencrypted root.  It would be nice if this were something
sysinst could do -- it took a lot of fiddling to set up.  (I couldn't
use sysinst anyway because this laptop is a MacBook, which requires a
GPT, which sysinst doesn't understand, but that's a separate issue.  I
also tried to set it up to boot from an external USB drive, but that
turns out to be non-trivial on a MacBook...)

Now that I have set it up, it seems pretty kludgey, so I wonder
whether anyone else really does this, or whether I'm doing it wrong?
I made it up as I went along; it wouldn't surprise me if some of what
I have done is silly.  Here are some things that don't seem quite
right -- any suggestions?

1. When I've booted into multi-user mode in the encrypted `root', I
   can't get at the unencrypted root, e.g. to switch kernels, or to
   remind myself what I put in there.  I was thwarted by the obvious
   failure of `mount -t null / /mnt/metaroot' in the unencrypted root,
   where /mnt is the mount point for the encrypted `root'.  Perhaps I
   could statically link a rump_null and use that or something...

2. On multi-user boot, init tries to update utmp -- in the encrypted
   `root', which doesn't work very well if there is no utmp database
   there, and doesn't do much good even if there is one.

3. Transitioning from single-user mode to multi-user mode, if I ever
   return to single-user mode from multi-user mode, seems to require a
   delicate dance to make sure that cgd1 is configured and the
   encrypted `root' file system mounted from it -- which is not
   straightforward because there's no way to ask the kernel `Is cgd1
   configured?'.

Here's my setup.  dk2 is a GPT partition of type `ffs' for the
unencrypted root.  dk3 is a GPT partition of type `cgd' for a cgd that
is subpartitioned with a disklabel into cgd1a (/), cgd1e (/var), &c.,
on which is a pretty much vanilla NetBSD 5.1_STABLE installation.
(There are a few other partitions for purposes unrelated to NetBSD,
hence the dk numbering that starts at 2.)

On the unencrypted root I have the following file system:

   /bin -> rescue
   /boot
   /boot.cfg
   /dev/ (populated by MAKEDEV)
   /etc/cgd/dk3
   /etc/rc
   /mnt/ (mount point for encrypted `root')
   /netbsd
   /rescue/ (populated from a NetBSD 5.1 install)
   /sbin -> rescue

In /etc/rc, I have the following dance to try to configure cgd1 and
mount cgd1a on /mnt if necessary and possible.  If cgd1a is not
mounted, but cgd1 is already configured, then this prompts twice for a
pass phrase, because it's not straightforward to ask what cgds are
already configured.  (Not hard to implement -- the kernel just needs a
new CGDIOCGET ioctl, and cgdconfig(8) a new `-l' option.)

   #!/bin/sh

   set -eu

   # The weird `/mnt/?' pattern is because I have seen both /mnt and
   # /mnt/ in the output of mount(8).  I don't know why the trailing
   # slash sometimes gets there.
   if ! /rescue/mount | /rescue/grep -E '^/dev/cgd1a on /mnt/? type ffs ' \
       >/dev/null 2>&1;
     if ! /rescue/cgdconfig cgd1 /dev/dk3 2>/dev/null; then
       if /rescue/cgdconfig -u cgd1 2>/dev/null; then
         /rescue/cgdconfig cgd1 /dev/dk3
       fi
     fi
     /rescue/mount_ffs -o ro,log,noatime,nodevmtime /dev/cgd1a /mnt
   fi

   /rescue/sysctl -w init.root=/mnt