kern/49007: if_vioif panic when ifconfig up/down under load

[ozaki-r%netbsd.org@localhost]

>Number:         49007
>Category:       kern
>Synopsis:       if_vioif panic when ifconfig up/down under load
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Jul 17 10:10:00 +0000 2014
>Originator:     Ryota Ozaki
>Release:        current (and 6)
>Organization:
>Environment:
NetBSD kvm 6.99.30 NetBSD 6.99.30 (GENERIC) #0: Sun Jan 26 19:39:39 UTC 2014  
builds%b8.netbsd.org@localhost:/home/builds/ab/HEAD/amd64/201401261800Z-obj/home/builds/ab/HEAD/src/sys/arch/amd64/compile/GENERIC
 amd64
and
NetBSD kvm 6.99.47 NetBSD 6.99.47 (KVM) #27: Thu Jul 17 17:38:39 JST 2014  
ozaki-r@hidden:/hidden/

Linux fedora20 3.14.7-200.fc20.x86_64 #1 SMP Wed Jun 11 22:38:05 UTC 2014 
x86_64 x86_64 x86_64 GNU/Linux
qemu-kvm-2.1.0-0.3.rc1.fc20.x86_64
>Description:
vioif_populate_rx_mbufs is not MPSAFE, however, it may run in parallel
on different CPUs and cause panic.

It is called via vioif_ioctl and vioif_rx_softint. vioif_ioctl runs with the
big kernel lock while vioif_rx_softint runs without it because
SOFTINT_MPSAFE is specified. OTOH, vioif_ioctl can run on an arbitrary
CPU while vioif_rx_softint runs only on CPU#0. Eventually,
vioif_populate_rx_mbufs can run on different CPUs simultaneously
and cause panic. Even worse, it kills qemu sometimes.
>How-To-Repeat:
Attach two virtio NICs to a VM (qemu) and run NetBSD (current or 6)
on it.

[VM]
Create a bridge and a tap and attach vioif0, vioif1 and tap0 to
bridge0. Assign some IP to tap0 and run an iperf server on it.

[outside]
Run an iperf server on one side of a VM NIC. Run two iperf clients;
one sends packets to the iperf server in the VM and the other
sends packets to the iperf server outside via the bridge in the VM.

[VM]
ifconfig vioif0 up/down repeatedly.

A simpler configuration may reproduce the problem, but the above
configuration can reproduce it quickly.

>Fix:
cvs -q diff -u sys/dev/pci/if_vioif.c                                           
                                                                     [src]
Index: sys/dev/pci/if_vioif.c
===================================================================
RCS file: /cvs/cvsroot/src/sys/dev/pci/if_vioif.c,v
retrieving revision 1.4
diff -u -r1.4 if_vioif.c
--- sys/dev/pci/if_vioif.c      9 May 2013 12:23:17 -0000       1.4
+++ sys/dev/pci/if_vioif.c      17 Jul 2014 09:10:13 -0000
@@ -557,7 +557,7 @@
                }
        }
 
-       sc->sc_rx_softint = softint_establish(SOFTINT_NET|SOFTINT_MPSAFE,
+       sc->sc_rx_softint = softint_establish(SOFTINT_NET,
                                              vioif_rx_softint, sc);
        if (sc->sc_rx_softint == NULL) {
                aprint_error_dev(self, "cannot establish softint\n");