[Replying to multiple messages, here.]

>> Can you maybe mount the USB key elsewhere and see what the ownership
>> and mode bits on the root and mount point (and everything leading to
>> the mount point, if it's not directly under /) are?
> It actually boots a memory disk with a file system image from the USB
> memory stick,

So this is a kernel-with-embedded-image a la the INSTALL kernels, only
with your stuff rather than the install stuff in the image?  (Just
checking that I have the right picture in mind.)

> so the real root is RAM at all times, once the kernel has been
> loaded.

It's actually in RAM, perhaps, but it's not in RAM in the sense of
(say) inodes being in the inode cache; stuff in a ramdisk is as
inaccessible to everything but the ramdisk driver as stuff on a real
disk is.

> If anyone is willing to try booting from a boot image which
> essentially does what I'm doing, please let me know.  If the problem
> is the least bit reproducible anywhere else, I'll be a much happier
> person.

Me too.  I was actually thinking about this myself.  What port is this
on?  Your PR says amd64; is that what you're using still?  If so, I
can't help directly, as I have no amd64 machines - but if you could
send me your ramdisk image (you said it's FFS, I think?) I'd be
interested to look at it, and possibly even try to reproduce it in
other environments.

>>> Does login in via ssh and exiting cause the same changes? 
>> No.

Strong clue.  I now think Thor's speculation about device node aliasing
and the ramdisk's root's /dev is likely to be on target.  I've been
thinking "...revoke()..." ever since I saw you say that logging *out*
was what "fixed" things, but until Thor said that I was baffled as to
what revoke() could be touching that would be relevant.

>> This kind of problem is why I've never been comfortable having init
>> do the chroot for this sort of system configuration, FWIW.
> Let's for a moment assume that I didn't know it was a bad idea to use
> init.root, and let's also assume that I'm kind of stuck with it now.
> Is there any hope of fixing it?

I would hope so; if not, we might as well get rid of init.root - if
this sort of problem arises whenever it's used, I'd call it unusable.

Perhaps it'd be a good idea for me to push a bit harder on my
root-on-cgd project, though, in case this turns out to be Really Hard
to fix.  (It might, for example, require a complete redesign of device
node aliasing or some such.)

/~\ The ASCII				der Mouse
\ / Ribbon Campaign
 X  Against HTML	       mouse@rodents.montreal.qc.ca
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B