kern/46596: ehci interrupt storm

[gson%gson.org@localhost (Andreas Gustafsson)]

>Number:         46596
>Category:       kern
>Synopsis:       ehci interrupt storm
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Jun 12 20:05:00 +0000 2012
>Originator:     Andreas Gustafsson
>Release:        NetBSD 5.1
>Organization:
>Environment:
System: NetBSD guido.araneus.fi 5.1 NetBSD 5.1 (GENERIC) #0: Sat Nov 6 13:19:33 
UTC 2010 
builds%b6.netbsd.org@localhost:/home/builds/ab/netbsd-5-1-RELEASE/amd64/201011061943Z-obj/home/builds/ab/netbsd-5-1-RELEASE/src/sys/arch/amd64/compile/GENERIC
 amd64
Architecture: x86_64
Machine: amd64
>Description:

I have a quad-core Intel Core i5 machine running NetBSD/amd64 5.1.
Running "top", I noticed that one of the cores is spending 30% of its
time in the "interrupt" state even though the machine is otherwise
idle.  Except for this, the machine is operating normally.  The
abnormal interrupt load was not there when the machine was first
booted, but appeared at some later time.

Output of "top":

  load averages:  0.49,  0.35,  0.26;               up 48+20:10:21
  37 processes: 31 sleeping, 3 stopped, 3 on CPU
  CPU0 states:  0.0% user,  0.0% nice,  0.0% system, 30.3% interrupt, 69.7% idle
  CPU1 states:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
  CPU2 states:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
  CPU3 states:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
  Memory: 9510M Act, 1478M Inact, 2720K Wired, 27M Exec, 7574M File, 2861M Free
  Swap: 16G Total, 1935M Used, 14G Free

Output of "vmstat 1":

  procs    memory      page                       disks   faults      cpu
  r b w    avm    fre  flt  re  pi   po   fr   sr w0 c0   in   sy  cs us sy id
  1 0 0 11719232 2929396 -363 1  0   26  147  190 95  0 -111  -91 -315 41 5 54
  0 0 0 11719236 2929392 1   0   0    0    0    0  0  0 232564 2118 111 0 6 94
  0 0 0 11719236 2929392 0   0   0    0    0    0  0  0 233937 2028 114 0 7 93
  0 0 0 11719236 2929392 0   0   0    0    0    0  0  0 233731 2028 114 0 9 91
  1 0 0 11719236 2929392 0   0   0    0    0    0  0  0 234077 2072 114 0 8 92
  0 0 0 11719236 2929392 0   0   0    0    0    0  0  0 233330 2071 116 0 7 93

Output of "vmstat -i":

  Interrupt                                     total     rate
  global TLB IPI                          11088118152     2627
  ioapic0 pin 16                          12004189025     2844
  ioapic0 pin 22                                    6        0
  ioapic0 pin 17                            399449104       94
  ioapic0 pin 18                             20577942        4
  ioapic0 pin 23                                   22        0
  ioapic0 pin 19                                11907        0
  Total                                   23512346158     5571

Rerunning "vmstat -i" repeatedly, the totals for "ioapic pin 16" and
"Total" are rapidly increasing, but the one for "global TLB IPI" is
not.

According to dmesg, "ioapic0 pin 16" belongs to ehci0:

  ehci0: interrupting at ioapic0 pin 16

Output from "usbdevs -v":

  Controller /dev/usb0:
  addr 1: high speed, self powered, config 1, EHCI root hub(0x0000), vendor 
0x8086(0x8086), rev 1.00
   port 1 addr 2: high speed, self powered, config 1, product 0x0024(0x0024), 
vendor 0x8087(0x8087), rev 0.00
    port 1 powered
    port 2 powered
    port 3 powered
    port 4 powered
    port 5 powered
    port 6 addr 3: low speed, power 100 mA, config 1, product 0x0103(0x0103), 
vendor 0x1267(0x1267), rev 1.01
   port 2 powered
  Controller /dev/usb1:
  addr 1: high speed, self powered, config 1, EHCI root hub(0x0000), vendor 
0x8086(0x8086), rev 1.00
   port 1 addr 2: high speed, self powered, config 1, product 0x0024(0x0024), 
vendor 0x8087(0x8087), rev 0.00
    port 1 powered
    port 2 powered
    port 3 powered
    port 4 powered
    port 5 powered
    port 6 powered
    port 7 powered
    port 8 powered
   port 2 powered

>How-To-Repeat:

I have no idea.

>Fix: