kern/48611: swapctl broken for multiple swap devs / panic

[kardel%netbsd.org@localhost]

>Number:         48611
>Category:       kern
>Synopsis:       swapctl reports inconsistencies / panic when removing swap devs
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sat Feb 22 10:40:00 +0000 2014
>Originator:     Frank Kardel
>Release:        NetBSD 6.99.28
>Organization:
        
>Environment:
System: NetBSD pip 6.99.28 NetBSD 6.99.28 (PIPGEN) #1: Sun Jan 12 13:08:59 CET 
2014 
kardel@...:/fs/raid1a/src/NetBSD/cur/src/obj.amd64/sys/arch/amd64/compile/PIPGEN
 amd64
Architecture: x86_64
Machine: amd64
>Description:
        When using multiple swap devices a sequence of
          swapctl -a /dev/wd2b
          swapctl -a /dev/sd1b
          swapctl -d /dev/sd1b
        will lead to swapctl -l to output
swapctl: SWAP_STATS different to SWAP_NSWAP (0 != 1)
Device      1K-blocks     Used    Avail Capacity  Priority

        Thus the swapstats data structures seem to have lost track. The NSWAP 
count is still correct.
        When a paging event occurs in this situation I have seen following 
panic:
(gdb) bt
#0  0xffffffff805acc80 in cpu_reboot (howto=260, bootstr=<optimized out>) at 
/fs/raid1a/src/NetBSD/cur/src/sys/arch/amd64/amd64/machdep.c:702
#1  0xffffffff807a2afd in vpanic (fmt=0xffffffff80b9b148 "kernel %sassertion 
\"%s\" failed: file \"%s\", line %d ", ap=0xffff8002425c38c0) at 
/fs/raid1a/src/NetBSD/cur/src/sys/kern/subr_prf.c:284
#2  0xffffffff80935c88 in kern_assert (fmt=<unavailable>) at 
/fs/raid1a/src/NetBSD/cur/src/sys/lib/libkern/kern_assert.c:51
#3  0xffffffff8089d6f8 in uvm_swap_markbad (startslot=<optimized out>, 
nslots=1) at /fs/raid1a/src/NetBSD/cur/src/sys/uvm/uvm_swap.c:1683
#4  0xffffffff808867e0 in uvmfault_anonget (ufi=0xffff8002425c3ae0, 
amap=0xfffffe8368790578, anon=0xfffffe83809d0600) at 
/fs/raid1a/src/NetBSD/cur/src/sys/uvm/uvm_fault.c:437
#5  0xffffffff8088792d in uvm_fault_upper (anons=0xffff8002425c3a60, 
flt=0xffff8002425c3b20, ufi=0xffff8002425c3ae0) at 
/fs/raid1a/src/NetBSD/cur/src/sys/uvm/uvm_fault.c:1327
#6  uvm_fault_internal (orig_map=<optimized out>, vaddr=<optimized out>, 
access_type=<optimized out>, fault_flag=<optimized out>) at 
/fs/raid1a/src/NetBSD/cur/src/sys/uvm/uvm_fault.c:886
#7  0xffffffff807ef3ed in trap (frame=0xffff8002425c3c80) at 
/fs/raid1a/src/NetBSD/cur/src/sys/arch/amd64/amd64/trap.c:567
#8  0xffffffff8010100e in alltraps ()
#9  0x00007f7f381cc000 in ?? ()
#10 0x0000000000000000 in ?? ()
(gdb) print panicstr
$1 = 0xffffffff80fad540 <scratchstr.10810> "kernel diagnostic assertion \"sdp 
!= NULL\" failed: file \"/fs/raid1a/src/NetBSD/cur/src/sys/uvm/uvm_swap.c\", 
line 1683 "

        I have not analized what swap i/o error could have lead to 
uvm_swap_markbad() being called.

>How-To-Repeat:
        see description.
>Fix:
        use 0-1 swap devices only.

        suspected code change in uvm_swap.c 
        ----------------------------
        revision 1.165
        date: 2013-11-23 15:50:40 +0100;  author: christos;  state: Exp;  
lines: +6 -6;  commitid: J9ITLTqbFXMiBpex;
        fix circleq comments
        ----------------------------
        revision 1.164
        date: 2013-11-23 15:32:13 +0100;  author: christos;  state: Exp;  
lines: +64 -16;  commitid: SYu4lVsQYslTupex;
        convert from CIRCLEQ to TAILQ
        add uvm_swap_shutdown(), unused