kern/55833: Union filesystem enters kernel debugger.

[rich.neswold%gmail.com@localhost]

>Number:         55833
>Category:       kern
>Synopsis:       Union filesystem enters kernel debugger.
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Nov 30 21:25:00 +0000 2020
>Originator:     Rich Neswold
>Release:        NetBSD 9.0
>Organization:
>Environment:
NetBSD rpi2 9.0_BETA NetBSD 9.0_BETA (GENERIC) #0: Mon Nov  4 14:49:31 UTC 2019  mkrepro%mkrepro.NetBSD.org@localhost:/usr/src/sys/arch/evbarm/compile/GENERIC evbarm
>Description:
On my RaspberryPi, I mount the filesystem read-only to reduce write cycles 
(/home is NFS mounted.) I had been mounting the subdirectories of /var with
several, tiny tmpfs drives but thought it'd be better to use the union
filesystem.

So I created /mnt/var. In /etc/fstab, I added an entry to mount a 64M, tmpfs on
it and an entry to map /mnt/var on top of /var via the union fs.
 
    tmpfs          /mnt/var        tmpfs   rw,hidden,-s64M
    /mnt/var       /var            union   rw

I added both /mnt/var and /var to 'critical_filesystems_local' in /etc/rc.conf.

With this configuration, every time I boot the Pi, it drops into the kernel 
debugger when building the 'dev' database:

    Building databases: dev
    [  18.xxxxxxx] uvm_fault(0x90fb86e0, 0, 1) -> e
    [  18.xxxxxxx] Fatal kernel mode data abort: 'Translation Fault (S)'
    [  18.xxxxxxx] trapframe: 0x9afe3b40
    [  18.xxxxxxx] FSR=00000005, FAR=0000003f, spsr=60010013
    [  18.xxxxxxx] r0 =0afe3b9c, r1 =ffffffff, r2 =00000000, r3 =ffffffff
    [  18.xxxxxxx] r4 =91278860, r5 =00000004, r6 =910bff78, r7 =910bff78
    [  18.xxxxxxx] r8 =00000000, r9 =808ebbc0, r10=80a73ad4, r11=9afe3bbc
    [  18.xxxxxxx] r12=00000000, ssp=9afe3b90, slr=8030e9e8, pc =8030e9ec

    Stopped in pid 146.1 (dev_mkdb) at       netbsd:tmpfs_loadvnode_0x38:    ldr r3, [r1, #0x040]
    db{0}>

A stack trace shows:

    db{0}> trace
    0x9aa3dbbc: netbsd:tmpfs_loadvnode+0x10
    0x9aa3dc24: netbsd:vcache_get+0x1f0
    0x9aa3dc5c: netbsd:tmpfs_gro_lookup+0x90
    0x9aa3dcd4: netbsd:genfs_sane_rename+0xce0
    0x9aa3dd0c: netbsd:tmpfs_sane_rename+0x48
    0x9aa3dd4c: netbsd:genfs_insane_rename+0x130
    0x9aa3dd8c: netbsd:VOP_RENAME+0x70
    0x9aa3ddbc: netbsd:union_rename+0x8c
    0x9aa3ddfc: netbsd:VOP_RENAME+0x70
    0x9aa3decc: netbsd:do_sys_renameat+0x6d0
    0x9aa3deec: netbsd:sys_rename+0x38
    0x9aa3dfac: netbsd:syscall+0x12c
    db{0}>

Restoring the tiny, tmpfs mount points lets the system boot successfully.
>How-To-Repeat:
Using the configuration described above triggers the problem every time the system is booted.
>Fix:
Since it's not particularly easy to switch between single user and multiuser mode on a RPi, I didn't experiment much. Avoiding the union filesystem restored it to a working system.