Subject: Interrupts with MP kernel
To: None <port-i386@netbsd.org>
From: rudolf <netbsd@eq.cz>
List: port-i386
Date: 10/14/2007 16:14:15
Hi,

I have a problem with interrupts with NetBSD 4.0_RC2 GENERIC.MP kernel. 
There seems to be everything ok with plain GENERIC or GENERIC_DIAGNOSTIC 
kernel. The machine is Sun Fire X2200 M2 with one dual-core Opteron.

Here are some numbers to document this problem:
GENERIC.MP:
# vmstat -i -c 3 -w 5
interrupt                                     total     rate
cpu0 softclock                                 1561        8
cpu0 softnet                                    473        2
cpu0 timer                                    19682      100
cpu0 FPU flush IPI                                1        0
cpu0 FPU synch IPI                                2        0
cpu0 TLB shootdown IPI                          113        0
cpu1 timer                                    16418       84
cpu1 FPU synch IPI                                1        0
cpu1 TLB shootdown IPI                          408        2
pic0 pin 0                                     2192       11
ioapic0 pin 20                                   32        0
ioapic0 pin 21                                   85        0
ioapic0 pin 22                                  718        3
ioapic0 pin 23                                 1013        5
Total                                         42699      218

interrupt                                     total     rate
cpu0 softclock                                 1580        7
cpu0 softnet                                    484        2
cpu0 timer                                    20297      100
cpu0 FPU flush IPI                                1        0
cpu0 FPU synch IPI                                2        0
cpu0 TLB shootdown IPI                          113        0
cpu1 timer                                    16907       84
cpu1 FPU synch IPI                                1        0
cpu1 TLB shootdown IPI                          408        2
pic0 pin 0                                     2263       11
ioapic0 pin 20                                   32        0
ioapic0 pin 21                                   86        0
ioapic0 pin 22                                  718        3
ioapic0 pin 23                                 1035        5
Total                                         43927      218

interrupt                                     total     rate
cpu0 softclock                                 1614        7
cpu0 softnet                                    495        2
cpu0 timer                                    20945      100
cpu0 FPU flush IPI                                1        0
cpu0 FPU synch IPI                                2        0
cpu0 TLB shootdown IPI                          113        0
cpu1 timer                                    17478       84
cpu1 FPU synch IPI                                1        0
cpu1 TLB shootdown IPI                          408        1
pic0 pin 0                                     2343       11
ioapic0 pin 20                                   32        0
ioapic0 pin 21                                   88        0
ioapic0 pin 22                                  722        3
ioapic0 pin 23                                 1057        5
Total                                         45299      217

#

GENERIC_DIAGNOSTIC:
# vmstat -i -c 3 -w 5
interrupt                                     total     rate
cpu0 softclock                                 5552      102
cpu0 softnet                                    154        2
cpu0 timer                                     5606      103
pic0 pin 0                                     1022       18
ioapic0 pin 20                                   50        0
ioapic0 pin 21                                    1        0
ioapic0 pin 22                                  798       14
ioapic0 pin 23                                  235        4
Total                                         13418      248

interrupt                                     total     rate
cpu0 softclock                                 6053      102
cpu0 softnet                                    160        2
cpu0 timer                                     6107      103
pic0 pin 0                                     1113       18
ioapic0 pin 20                                   50        0
ioapic0 pin 21                                    1        0
ioapic0 pin 22                                  798       13
ioapic0 pin 23                                  246        4
Total                                         14528      246

interrupt                                     total     rate
cpu0 softclock                                 6554      102
cpu0 softnet                                    166        2
cpu0 timer                                     6608      103
pic0 pin 0                                     1204       18
ioapic0 pin 20                                   50        0
ioapic0 pin 21                                    1        0
ioapic0 pin 22                                  802       12
ioapic0 pin 23                                  260        4
Total                                         15645      244

#

Joerg's GENERIC.MP 
(http://mail-index.netbsd.org/port-i386/2007/10/05/0003.html):
# vmstat -i -c 3 -w 5
interrupt                                     total     rate
cpu0 softclock                                 1061        9
cpu0 softnet                                    108        0
cpu0 timer                                    11086      101
cpu0 FPU synch IPI                                2        0
cpu0 TLB shootdown IPI                          128        1
cpu1 timer                                     8768       80
cpu1 FPU synch IPI                                1        0
cpu1 TLB shootdown IPI                          354        3
ioapic0 pin 20                                   31        0
ioapic0 pin 21                                  192        1
ioapic0 pin 22                                  729        6
ioapic0 pin 23                                  155        1
Total                                         22615      207

interrupt                                     total     rate
cpu0 softclock                                 1105        9
cpu0 softnet                                    116        1
cpu0 timer                                    11756      101
cpu0 FPU synch IPI                                2        0
cpu0 TLB shootdown IPI                          128        1
cpu1 timer                                     9327       80
cpu1 FPU synch IPI                                1        0
cpu1 TLB shootdown IPI                          354        3
ioapic0 pin 20                                   31        0
ioapic0 pin 21                                  200        1
ioapic0 pin 22                                  729        6
ioapic0 pin 23                                  169        1
Total                                         23918      206

interrupt                                     total     rate
cpu0 softclock                                 1170        9
cpu0 softnet                                    124        1
cpu0 timer                                    12283      101
cpu0 FPU synch IPI                                2        0
cpu0 TLB shootdown IPI                          128        1
cpu1 timer                                     9772       80
cpu1 FPU synch IPI                                1        0
cpu1 TLB shootdown IPI                          354        2
ioapic0 pin 20                                   31        0
ioapic0 pin 21                                  213        1
ioapic0 pin 22                                  729        6
ioapic0 pin 23                                  182        1
Total                                         24989      206

#

My intent is to run a NetBSD Dom0 and several domains on this. AFAIK the 
Dom0 kernel is UP so this should not be a problem, but should I expect 
some problems with DomUs or with this XEN setup generally?

I am able to provide more information (dmesg, ...) and test eventual 
patches.

Regards,

rudolf