NetBSD-Bugs archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

kern/50972: unpluging a in-use USB device hangs



>Number:         50972
>Category:       kern
>Synopsis:       unpluging a in-use USB device hangs
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Mar 17 10:05:00 +0000 2016
>Originator:     Manuel Bouyer
>Release:        NetBSD 7.0_STABLE
>Organization:
-- 
Manuel Bouyer <bouyer%antioche.eu.org@localhost>
     NetBSD: 26 ans d'experience feront toujours la difference
--
>Environment:
System: NetBSD armandeche.soc.lip6.fr 7.0_STABLE NetBSD 7.0_STABLE (GENERIC.201603102140Z) amd64
Architecture: x86_64
Machine: amd64
>Description:
	Unpluging a in-use USB device hangs the system hard: no more network,
	no more character echo from console. I can still enter ddb from a
	serial console through.
	The example below is with a umass, but I've also been hit by this with
	usb to serial converter on occasion. This example is with Xen but I've
	also seen this on base-metal GENERIC kernel.

[plug umass device]
borneo:/root#umass0 at uhub2 port 3 configuration 1 interface 0
umass0: Lexar USB Flash Drive, rev 2.00/11.00, addr 2
scsibus0 at umass0: 2 targets, 1 lun per target
sd0 at scsibus0 target 0 lun 0: <Lexar, USB Flash Drive, 1100> disk removable
sd0: fabricating a geometry
sd0: 59904 MB, 59904 cyl, 64 head, 32 sec, 512 bytes/sect x 122683392 sectors
sd0: fabricating a geometry

borneo:/root#mount -r /dev/sd0e /mnt
borneo:/root#df
Filesystem    1K-blocks       Used      Avail %Cap Mounted on
/dev/wd0a      10321774    2770480    7035206  28% /
/dev/wd0g     103220710    2557542   95502134   2% /home
/dev/wd0h     524178708  205575020  292394756  41% /domains
kernfs                1          1          0 100% /kern
ptyfs                 1          1          0 100% /dev/pts
tmpfs            131072          4     131068   0% /var/shm
/dev/sd0e      61326688   41801664   19525024  68% /mnt
[unplug umass device]
borneo:/root#umass0: BBB reset failed, IOERROR
umass0: BBB bulk-in clear stall failed, IOERROR
umass0: BBB bulk-out clear stall failed, IOERROR
umass0: BBB reset failed, IOERROR
umass0: BBB bulk-in clear stall failed, IOERROR
umass0: BBB bulk-out clear stall failed, IOERROR
umass0: BBB reset failed, IOERROR
umass0: BBB bulk-in clear stall failed, IOERROR
umass0: BBB bulk-out clear stall failed, IOERROR
umass0: BBB reset failed, IOERROR
umass0: BBB bulk-in clear stall failed, IOERROR
umass0: BBB bulk-out clear stall failed, IOERROR
umass0: BBB reset failed, IOERROR
umass0: BBB bulk-in clear stall failed, IOERROR
umass0: BBB bulk-out clear stall failed, IOERROR
[hang]
fatal breakpoint trap in supervisor mode
trap type 1 code 0 rip ffffffff801fbad5 cs e030 rflags 202 cr2 7f7ff7b06000 ilevel 8 rsp ffffa00019427900
curlwp 0xffffa00001532100 pid 0.42 lowest kstack 0xffffa000194242c0
Stopped in pid 0.42 (system) at netbsd:breakpoint+0x5:  leave
breakpoint() at netbsd:breakpoint+0x5
xencons_tty_input() at netbsd:xencons_tty_input+0xb2
xencons_intr() at netbsd:xencons_intr+0x4c
evtchn_do_event() at netbsd:evtchn_do_event+0x298
do_hypervisor_callback() at netbsd:do_hypervisor_callback+0x176
hypervisor_callback() at netbsd:hypervisor_callback+0x9e
vrevoke() at netbsd:vrevoke+0x9e
genfs_revoke() at netbsd:genfs_revoke+0x13
VOP_REVOKE() at netbsd:VOP_REVOKE+0x53
vdevgone() at netbsd:vdevgone+0x4f
sddetach() at netbsd:sddetach+0xb4
config_detach() at netbsd:config_detach+0xf8
scsipi_target_detach() at netbsd:scsipi_target_detach+0xbc
scsibusdetach() at netbsd:scsibusdetach+0x35
config_detach() at netbsd:config_detach+0xf8
umass_detach() at netbsd:umass_detach+0xa0
config_detach() at netbsd:config_detach+0xf8
usb_disconnect_port() at netbsd:usb_disconnect_port+0xae
uhub_explore() at netbsd:uhub_explore+0x180
usb_discover.isra.2() at netbsd:usb_discover.isra.2+0x4e
usb_event_thread() at netbsd:usb_event_thread+0x74
ds          6fb0
es          7e49
fs          0
gs          ec12
rdi         ffffa00001027080
rsi         ffffffff80e9eab0    rbuf.11823
rbp         ffffa00019427900
rbx         ffffffff80e9eab0    rbuf.11823
rdx         2b
rcx         2b
rax         1
r8          9
r9          2
r10         2
r11         ffffa000019f0638
r12         ffffa00001029388
r13         ffffffff80e9eab1    rbuf.11823+0x1
r14         ffffa00001027080
r15         ffffa00000e51a08
rip         ffffffff801fbad5    breakpoint+0x5
cs          e030
rflags      202
rsp         ffffa00019427900
ss          e02b
netbsd:breakpoint+0x5:  leave
db> sh ev
evcnt type 0: bus_dma bounces = 128
evcnt type 0: bus_dma loads = 23587
evcnt type 0: bus_dma nbouncebufs = 641
evcnt type 0: vmcmd kills = 253
evcnt type 0: vmcmd extends = 49
evcnt type 0: vmcmd calls = 2189
evcnt type 0: vmem static_bt_inuse = 200
evcnt type 0: vmem static_bt_count = 200
evcnt type 0: softint net/0 = 899
evcnt type 0: softint bio/0 = 23201
evcnt type 0: softint clk/0 = 7631
evcnt type 0: softint ser/0 = 599
evcnt type 0: callout late/0 = 19
evcnt type 0: crosscall unicast = 4
evcnt type 0: namecache entries collected = 449
evcnt type 0: namecache under scan target = 345
evcnt type 1: vcpu0 xencons = 104
evcnt type 1: vcpu0 ioapic0 pin 18 = 25964
evcnt type 1: vcpu0 ioapic0 pin 17 = 35
evcnt type 1: vcpu0 ioapic0 pin 21 = 425
evcnt type 1: vcpu0 ioapic0 pin 16 = 3
evcnt type 1: vcpu0 ioapic0 pin 22 = 488
evcnt type 1: vcpu0 ioapic0 pin 23 = 3
evcnt type 1: vcpu0 hardclock = 36155
evcnt type 1: vcpu0 clock = 36099
evcnt type 1: vcpu0 xenbus = 57
db> ps
PID    LID S CPU     FLAGS       STRUCT LWP *               NAME WAIT
653      1 2   0         0   ffffa000016321c0               tcsh
719      1 3   0        80   ffffa00001a4f6c0                ksh pause
878      1 3   0        80   ffffa00001686a60               tcsh ttyraw
40       1 3   0        80   ffffa0000166a620               sshd select
916      1 3   0        80   ffffa0000166aa40               sshd select
851      1 3   0        80   ffffa00001ad96e0              getty ttyraw
952      1 3   0        80   ffffa000016571e0              getty ttyraw
731      1 3   0        80   ffffa00001657a20              getty ttyraw
953      1 3   0        80   ffffa00001539560              login wait
882      1 3   0        80   ffffa00001ad9b00               cron nanoslp
661      1 3   0        80   ffffa00001a4fae0              inetd kqueue
826      1 3   0        80   ffffa00001a37280               qmgr kqueue
860      1 3   0        80   ffffa00001ad92c0             pickup kqueue
615      1 3   0        80   ffffa000020e4b20             master kqueue
471      1 3   0        80   ffffa00001a4f2a0               sshd select
515      1 3   0        80   ffffa00001a376a0             powerd kqueue
342      2 3   0        80   ffffa00001938680        xenconsoled netio
342      1 3   0        80   ffffa0000166a200        xenconsoled select
460      1 3   0        80   ffffa00001a37ac0          xenstored select
221      1 2   0         0   ffffa00001938aa0            syslogd
158      4 3   0        80   ffffa00001720240           dhclient parked
158      3 3   0        80   ffffa00001686640           dhclient kqueue
158      2 3   0        80   ffffa00001720660           dhclient parked
158      1 3   0        80   ffffa00001720a80           dhclient parked
1        1 3   0        80   ffffa000015629c0               init wait
0       57 3   0       200   ffffa0000102b8e0           scsibus0 sccomp
0       56 3   0       200   ffffa00001686220        xen_balloon xen_balloon
0       55 3   0       200   ffffa000016045c0            physiod physiod
0       54 3   0       200   ffffa000016325e0           aiodoned aiodoned
0       53 3   0       200   ffffa00001632a00            ioflush syncer
0       52 3   0       200   ffffa000016041a0           pgdaemon pgdaemon
0       49 3   0       200   ffffa000016049e0          atapibus0 sccomp
0       46 3   0       200   ffffa00001539980               usb6 usbevt
0       45 3   0       200   ffffa00001538120               usb5 usbevt
0       44 3   0       200   ffffa00001538540               usb4 usbevt
0       43 3   0       200   ffffa00001538960               usb3 usbevt
0    >  42 7   0       200   ffffa00001532100               usb2
0       41 3   0       200   ffffa00001532520               usb1 usbevt
0       40 3   0       200   ffffa00001562180               usb0 usbevt
0       39 3   0       200   ffffa000015625a0          cryptoret crypto_w
0       38 3   0       200   ffffa0000153a160              unpgc unpgc
0       37 3   0       200   ffffa00001539140        vmem_rehash vmem_rehash
0       36 3   0       200   ffffa0000153a580             xenbus rdst
0       35 3   0       200   ffffa0000153a9a0           xenwatch evtsq
0       26 3   0       200   ffffa00001532940            atabus6 atath
0       25 3   0       200   ffffa000014e40e0            atabus5 atath
0       24 3   0       200   ffffa000014e4500            atabus4 atath
0       23 3   0       200   ffffa000014e4920            atabus3 atath
0       22 3   0       200   ffffa000011d00c0            atabus2 atath
0       21 3   0       200   ffffa000011d04e0         usbtask-dr usbtsk
0       20 3   0       200   ffffa000011d0900         usbtask-hc usbtsk
0       19 3   0       200   ffffa0000102b0a0            atabus1 atath
0       18 3   0       200   ffffa0000102b4c0            atabus0 atath
0       16 3   0       200   ffffa0000100a080             sysmon smtaskq
0       15 3   0       200   ffffa0000100a4a0         pmfsuspend pmfsuspend
0       14 3   0       200   ffffa0000100a8c0           pmfevent pmfevent
0       13 3   0       200   ffffa00000e6d060         sopendfree sopendfr
0       12 3   0       200   ffffa00000e6d480           nfssilly nfssilly
0       11 3   0       200   ffffa00000e6d8a0            cachegc cachegc
0       10 3   0       200   ffffa00000e6b040              vrele vrele
0        9 3   0       200   ffffa00000e6b460             vdrain vdrain
0        8 3   0       200   ffffa00000e6b880          modunload mod_unld
0        7 3   0       200   ffffa00000e62020            xcall/0 xcall
0        6 1   0       200   ffffa00000e62440          softser/0
0        5 1   0       200   ffffa00000e62860          softclk/0
0        4 1   0       200   ffffa00000e60000          softbio/0
0        3 1   0       200   ffffa00000e60420          softnet/0
0        2 1   0       201   ffffa00000e60840             idle/0
0        1 3   0       200   ffffffff80cecd20            swapper uvm
db> tr/a ffffa00001532100
trace: pid 0 lid 42 at 0xffffa00019427900
breakpoint() at netbsd:breakpoint+0x5
xencons_tty_input() at netbsd:xencons_tty_input+0xb2
xencons_intr() at netbsd:xencons_intr+0x4c
evtchn_do_event() at netbsd:evtchn_do_event+0x298
do_hypervisor_callback() at netbsd:do_hypervisor_callback+0x176
hypervisor_callback() at netbsd:hypervisor_callback+0x9e
vrevoke() at netbsd:vrevoke+0x9e
genfs_revoke() at netbsd:genfs_revoke+0x13
VOP_REVOKE() at netbsd:VOP_REVOKE+0x53
vdevgone() at netbsd:vdevgone+0x4f
sddetach() at netbsd:sddetach+0xb4
config_detach() at netbsd:config_detach+0xf8
scsipi_target_detach() at netbsd:scsipi_target_detach+0xbc
scsibusdetach() at netbsd:scsibusdetach+0x35
config_detach() at netbsd:config_detach+0xf8
umass_detach() at netbsd:umass_detach+0xa0
config_detach() at netbsd:config_detach+0xf8
usb_disconnect_port() at netbsd:usb_disconnect_port+0xae
uhub_explore() at netbsd:uhub_explore+0x180
usb_discover.isra.2() at netbsd:usb_discover.isra.2+0x4e
usb_event_thread() at netbsd:usb_event_thread+0x74
db> tr/a ffffa000016321c0
trace: pid 653 lid 1 at 0xffffa00019488be0
sleepq_block() at netbsd:sleepq_block+0x8e
cv_timedwait_sig() at netbsd:cv_timedwait_sig+0xfc
ttysleep() at netbsd:ttysleep+0x74
ttread() at netbsd:ttread+0x295
cnread() at netbsd:cnread+0x46
spec_read() at netbsd:spec_read+0x9b
VOP_READ() at netbsd:VOP_READ+0x55
vn_read() at netbsd:vn_read+0x94
dofileread() at netbsd:dofileread+0x90
sys_read() at netbsd:sys_read+0x5f
syscall() at netbsd:syscall+0x9a
--- syscall (number 3) ---
7f7ff6c3c3ba:
db>  reboot

	(gdb) l *(vrevoke+0x9e)
	0xffffffff807199e6 is at /home/bouyer/src-7/src/sys/kern/vfs_vnode.c:1131.
	1126                    dev = vp->v_rdev;
	1127                    type = vp->v_type;
	1128                    mutex_exit(vp->v_interlock);
	1129            }
	1130    
	1131            while (spec_node_lookup_by_dev(type, dev, &vq) == 0) {
	1132                    vgone(vq);
	1133            }
	1134    }
	1135    

	So I suspect it's stuck in the above while() loop.

>How-To-Repeat:
	see above
>Fix:
	unknown, but we really need one



Home | Main Index | Thread Index | Old Index