Port-xen archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: xen and networking problems



Manuel Bouyer writes:
 > Did you run it with -n ? Otherwise, IP reverse resolution could cause it
 > to stall.

Good point.  I ran it before without -n.  This time I tried it again
with -n.  Same thing; no packets captured.  When I terminate tcpdump
with ^C, it reports 0 packets captured/received/dropped.

 > tcpdump being silent makes it worse than that, as the packets are
 > passed to bpf from the driver itself, in the interrupt routine.

So, I'm guessing this implies a problem early in the packet
processing?  Is there a way to verify that?

 > check that it's not running out of mbufs. vmstat -m will tell you
 > if mbuf allocation fail (mbuf and mbuf cluster). 

See the output of vmstat -m below.  I'm not sure what to look at
exactly.  Does this indicate any problems?

 > 'xm dmesg' should give them to you

Thanks.  Just for reference, I've included that output as well.

I hope there are some clues in the output I'm providing.  I greatly
appreciate all the help and hope we can find a solution.

Cheers,
Brook

===========================================================================

# vmstat -m
Memory statistics by bucket size
    Size   In Use   Free   Requests  HighWater  Couldfree
      16      273    239       1759    1280          0
      32      492     20       2191     640          0
      64      787     45        870     320          0
     128      676     28       1216     160          0
     256      551      9        596      80          0
     512       36     12         45      40          0
    1024       94      2        306      20          0
    2048       27      3         40      10          0
    4096       25      2        545       5          0
    8192        5      0          6       5          0
   16384        2      0          2       5          0
  131072        2      0          2       5          0

Memory usage type by bucket size
    Size  Type(s)
      16  ptyfs mount, kernfs mount, prop string, VM swap, USB, USB device,
          soname, sockaddr, vmem, NETSMBDEV, sysctldata, devbuf, pcb, temp,
          routetbl, UFS mount, in_multi, ifaddr, acpi
      32  VM swap, USB, soname, sockaddr, tcpcongctl, blist, RAIDframe,
          sysctldata, devbuf, pcb, temp, routetbl, ifmedia, ifaddr, ether_multi,
          acpi
      64  USB, vmem, NDP, sysctldata, devbuf, DMA map, temp, ifaddr, acpi
     128  prop dictionary, VM swap, USB, USB device, soname, NDP, sysctlnode,
          sysctldata, devbuf, pcb, temp, routetbl, in_multi, ifaddr, acpi
     256  NTFS mount, MSDOSFS mount, VM page, USB, sysctldata, devbuf, DMA map,
          temp, routetbl, UFS mount, ifaddr, acpi
     512  VM map, vmem, sysctldata, devbuf, temp, ifaddr, crypto, acpi
    1024  USB, sysctlnode, sysctldata, devbuf, temp, acpi
    2048  USB device, vmem, RAIDframe, sysctlnode, sysctldata, temp, UFS mount
    4096  vmem, RAIDframe, sysctlnode, devbuf, temp, UFS mount
    8192  sysctlnode, devbuf
   16384  sysctlnode, devbuf
  131072  blist, devbuf

Memory statistics by type                                Type  Kern
           Type InUse  MemUse HighUse   Limit   Requests Limit Limit Size(s)
    ptyfs mount     1      1K      1K  39322K          1     0     0 16
     NTFS mount     0      0K      1K  39322K          1     0     0 256
  MSDOSFS mount     0      0K      1K  39322K          1     0     0 256
   kernfs mount     1      1K      1K  39322K          1     0     0 16
prop dictionary   123     16K     16K  39322K        131     0     0 128
    prop string   120      2K      2K  39322K        120     0     0 16
        VM swap     3      1K      1K  39322K          3     0     0 16,32,128
        VM page     1      1K      1K  39322K          1     0     0 256
         VM map     3      2K      2K  39322K          3     0     0 512
            USB    46      5K      5K  39322K         52     0     0 
16,32,64,128,256,1024
     USB device    18     19K     19K  39322K         18     0     0 16,128,2048
         soname    30      4K      4K  39322K         55     0     0 16,32,128
       sockaddr    52      2K      2K  39322K         91     0     0 16,32
     tcpcongctl     2      1K      1K  39322K          2     0     0 32
           vmem    10     15K     15K  39322K         15     0     0 
16,64,512,2048,4096
          blist     2     73K     73K  39322K          2     0     0 32,131072
      NETSMBDEV     1      1K      1K  39322K          1     0     0 16
      RAIDframe    10     35K     35K  39322K         10     0     0 
32,2048,4096
            NDP     4      1K      1K  39322K          5     0     0 64,128
     sysctlnode   111    163K    163K  39322K        142     0     0 
128,1024,2048,4096,8192,16384
     sysctldata    59      6K      6K  39322K         74     0     0 
16,32,64,128,256,512,1024,2048
         devbuf   145    157K    158K  39322K        190     0     0 
16,32,64,128,256,512,1024,4096,8192,16384,131072
        DMA map   816    148K    148K  39322K        816     0     0 64,256
            pcb    89     11K     12K  39322K        120     0     0 16,32,128
           temp    15      6K     14K  39322K        758     0     0 
16,32,64,128,256,512,1024,2048,4096
       routetbl    32      3K      4K  39322K         69     0     0 
16,32,128,256
      UFS mount     7     13K     13K  39322K          7     0     0 
16,256,2048,4096
       in_multi    22      2K      2K  39322K         22     0     0 16,128
        ifmedia     5      1K      1K  39322K          5     0     0 32
         ifaddr    17      4K      4K  39322K         17     0     0 
16,32,64,128,256,512
    ether_multi     4      1K      1K  39322K          4     0     0 32
         crypto     1      1K      1K  39322K          1     0     0 512
           acpi  1220     86K     86K  39322K       4840     0     0 
16,32,64,128,256,512,1024

Memory totals:  In Use    Free    Requests
                  762K     35K        7578

Memory resource pool statistics
Name        Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
amappl        48      417    0        0     5     0     5     5     0   inf    0
anonpl        16     5620    0        0    23     0    23    23     0   inf    0
aobj          72        7    0        0     1     0     1     1     0   inf    0
ataspl        56     2318    0     2318     1     0     1     1     0   inf    1
biopl        176       12    0        0     1     0     1     1     0   inf    0
buf16k      16384     139    0       17    34     1    33    34     1     1    0
buf1k       1024        6    0        6     1     0     1     1     1     1    1
buf2k       2048      110    0        5     4     0     4     4     1     1    0
buf4k       4096        7    0        0     1     0     1     1     1     1    0
buf512b      512        1    0        1     1     0     1     1     1     1    1
buf8k       8192       10    0        5     2     0     2     2     1     1    1
bufpl        176      252    0        0    11     0    11    11     0   inf    0
cwdi          24       21    0        0     1     0     1     1     0   inf    0
execargs    262144    201    0      201     4     0     4     4     0    16    4
extent        20        2    0        1     1     0     1     1     0   inf    0
fdfile        28      299    0        0     3     0     3     3     0   inf    0
ffsdino1     128     3262    0        0   106     0   106   106     0   inf    0
ffsino       168     3262    0        0   136     0   136   136     0   inf    0
file          64      164    0        0     3     0     3     3     0   inf    0
filedesc     652       21    0        0     4     0     4     4     0   inf    0
fstrans       20       26    0        0     1     0     1     1     0   inf    0
in6pcbpl     144       15    0       14     1     0     1     1     0   inf    0
inmltpl       32        2    0        0     1     0     1     1     0   inf    0
inpcbpl      112       12    0       10     1     0     1     1     0   inf    0
kcredpl      164       59    0        0     3     0     3     3     0   inf    0
kmem-100     100       31    0        0     7     0     7     7     0   inf    0
kmem-1024   1024      112    0        0   112     0   112   112     0   inf    0
kmem-12       12      490    0        0    12     0    12    12     0   inf    0
kmem-128     128       15    0        0     4     0     4     4     0   inf    0
kmem-1280   1280       26    0        0    26     0    26    26     0   inf    0
kmem-16       16      131    0        0     5     0     5     5     0   inf    0
kmem-1792   1792        2    0        0     2     0     2     2     0   inf    0
kmem-20       20       76    0        0     4     0     4     4     0   inf    0
kmem-2048   2048        4    0        0     4     0     4     4     0   inf    0
kmem-2304   2304        3    0        0     3     0     3     3     0   inf    0
kmem-24       24      148    0        0     8     0     8     8     0   inf    0
kmem-2560   2560        5    0        0     5     0     5     5     0   inf    0
kmem-28       28        7    0        0     1     0     1     1     0   inf    0
kmem-32       32     2195    0        0   138     0   138   138     0   inf    0
kmem-3328   3328        1    0        0     1     0     1     1     0   inf    0
kmem-3584   3584        5    0        0     5     0     5     5     0   inf    0
kmem-36       36     2182    0        0   156     0   156   156     0   inf    0
kmem-40       40      127    0        0    11     0    11    11     0   inf    0
kmem-44       44        6    0        0     1     0     1     1     0   inf    0
kmem-48       48       20    0        0     2     0     2     2     0   inf    0
kmem-512     512       27    0        0    27     0    27    27     0   inf    0
kmem-56       56       13    0        0     2     0     2     2     0   inf    0
kmem-64       64        7    0        0     1     0     1     1     0   inf    0
kmem-72       72      306    0        0    44     0    44    44     0   inf    0
kmem-768     768       70    0        0    70     0    70    70     0   inf    0
kmem-8         8        1    0        0     1     0     1     1     0   inf    0
kmem-84       84      112    0        0    19     0    19    19     0   inf    0
ksiginfo      48        1    0        0     1     0     1     1     0   inf    0
kvakernel   4096      610    0        0    20     0    20    20     0   inf    0
kvakmem     4096       72    0        0     3     0     3     3     0   inf    0
lockf         64        8    0        0     1     0     1     1     0   inf    0
lwppl        672       61    0        0    11     0    11    11     0   inf    0
lwpucpl      776       10    0       10     1     0     1     1     0   inf    1
mbpl         256      136    0        0    10     0    10    10     1   inf    1
mclpl       2048      128    0        0    68     0    68    68     4   512    4
mutex         12      415    0        0     2     0     2     2     0   inf    0
ncache        96     3277    0        0    79     0    79    79     0   inf    0
pcache       576      103    0       18    13     0    13    13     0   inf    0
pcglarge     756       91    0       23    14     0    14    14     0   inf    0
pcgnormal    180      174    0        7     8     0     8     8     0   inf    0
pdict16       44        9    0        0     1     0     1     1     0   inf    0
pdict32       60        3    0        0     1     0     1     1     0   inf    0
pdppl       16384      26    0        5    21     0    21    21     0   inf    0
phpool-0      32      149    0        1     2     0     2     2     0   inf    0
phpool-64     36      759    0        0     7     0     7     7     0   inf    0
piperd       188        6    0        0     1     0     1     1     0   inf    0
pipewr       188        6    0        0     1     0     1     1     0   inf    0
plimitpl     196       11    0        0     1     0     1     1     0   inf    0
pmappl       124       21    0        0     1     0     1     1     0   inf    0
pnbufpl     1024       17    0        0     5     0     5     5     0   inf    0
procpl       476       21    0        0     3     0     3     3     0   inf    0
propdict      32      132    0        8     1     0     1     1     0   inf    0
propnmbr      32       14    0        1     1     0     1     1     0   inf    0
propstng      20      186    0        4     1     0     1     1     0   inf    0
pstatspl     268       21    0        0     2     0     2     2     0   inf    0
ptimerpl     164        4    0        0     1     0     1     1     0   inf    0
ptimerspl    140        4    0        0     1     0     1     1     0   inf    0
pvpl          20     7269    0        0    36     0    36    36     0   inf    0
ractx         24      657    0        0     4     0     4     4     0   inf    0
rndsample    528       45    0       41     1     0     1     1     0   inf    0
rtentpl      144       26    0        0     1     0     1     1     0   inf    0
sigacts     2056       21    0        0    21     0    21    21     0   inf    0
socket       320       92    0        0     8     0     8     8     0   inf    0
tcpcbpl      500       13    0       10     1     0     1     1     0   inf    0
tstilepl      52       61    0        0     1     0     1     1     0   inf    0
uaoeltpl      84        2    0        0     1     0     1     1     0   inf    0
uarea       12288      61    0        0    61     0    61    61     0   inf    0
ufsdir       264        2    0        0     1     0     1     1     0   inf    0
vmembt        28     1060    0        0     8     0     8     8     0   inf    0
vmmpepl       84      847    0        0    18     0    18    18     0   inf    0
vmsppl       212       21    0        0     2     0     2     2     0   inf    0
vnodepl      184     3277    0        0   149     0   149   149     0   inf    0

In use 6680K, total allocated 8638K; utilization 77.3%

===========================================================================

# xm dmesg
 __  __            _____  _____  ____  
 \ \/ /___ _ __   |___ / |___ / |___ \ 
  \  // _ \ '_ \    |_ \   |_ \   __) |
  /  \  __/ | | |  ___) | ___) | / __/ 
 /_/\_\___|_| |_| |____(_)____(_)_____|
                                       
(XEN) Xen version 3.3.2 (root@) (gcc version 4.1.3 20080704 prerelease (NetBSD 
nb2 20081120)) Wed Oct 28 18:11:20 MDT 2009
(XEN) Latest ChangeSet: unavailable
(XEN) Command line: dom0_mem=64M
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 2 seconds
(XEN) Disc information:
(XEN)  Found 1 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 000000000009f800 (usable)
(XEN)  000000000009f800 - 00000000000a0000 (reserved)
(XEN)  00000000000d8000 - 00000000000e0000 (reserved)
(XEN)  00000000000e4000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 00000000f7ef0000 (usable)
(XEN)  00000000f7ef0000 - 00000000f7efc000 (ACPI data)
(XEN)  00000000f7efc000 - 00000000f7f00000 (ACPI NVS)
(XEN)  00000000f7f00000 - 00000000f7f80000 (usable)
(XEN)  00000000f7f80000 - 00000000f8000000 (reserved)
(XEN)  00000000fec00000 - 00000000fec10000 (reserved)
(XEN)  00000000fee00000 - 00000000fee01000 (reserved)
(XEN)  00000000ff800000 - 00000000ffc00000 (reserved)
(XEN)  00000000fff00000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 0000000400000000 (usable)
(XEN) System RAM: 16255MB (16645180kB)
(XEN) ACPI: RSDP 000F6720, 0014 (r0 PTLTD )
(XEN) ACPI: RSDT F7EF8195, 0034 (r1 PTLTD    RSDT    6040000  LTP        0)
(XEN) ACPI: FACP F7EFBE60, 0074 (r1 INTEL  K_CANYON  6040000 PTL         8)
(XEN) ACPI: DSDT F7EF81C9, 3C97 (r1  INTEL   PLUMAS  6040000 MSFT  100000E)
(XEN) ACPI: FACS F7EFCFC0, 0040
(XEN) ACPI: APIC F7EFBED4, 00B4 (r1 PTLTD        APIC    6040000  LTP        0)
(XEN) ACPI: BOOT F7EFBF88, 0028 (r1 PTLTD  $SBFTBL$  6040000  LTP        1)
(XEN) ACPI: SPCR F7EFBFB0, 0050 (r1 PTLTD  $UCRTBL$  6040000 PTL         1)
(XEN) Xen heap: 9MB (9444kB)
(XEN) Domain heap initialised
(XEN) Processor #0 15:2 APIC version 20
(XEN) Processor #6 15:2 APIC version 20
(XEN) Processor #1 15:2 APIC version 20
(XEN) Processor #7 15:2 APIC version 20
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
(XEN) IOAPIC[1]: apic_id 3, version 32, address 0xfec80000, GSI 24-47
(XEN) IOAPIC[2]: apic_id 4, version 32, address 0xfec80400, GSI 48-71
(XEN) IOAPIC[3]: apic_id 5, version 32, address 0xfec81000, GSI 72-95
(XEN) IOAPIC[4]: apic_id 8, version 32, address 0xfec81400, GSI 96-119
(XEN) Enabling APIC mode:  Flat.  Using 5 I/O APICs
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 1996.662 MHz processor.
(XEN) CPU0: Intel(R) XEON(TM) CPU 2.00GHz stepping 04
(XEN) Booting processor 1/6 eip 8c000
(XEN) CPU1: Intel(R) XEON(TM) CPU 2.00GHz stepping 04
(XEN) Booting processor 2/1 eip 8c000
(XEN) CPU2: Intel(R) XEON(TM) CPU 2.00GHz stepping 04
(XEN) Booting processor 3/7 eip 8c000
(XEN) CPU3: Intel(R) XEON(TM) CPU 2.00GHz stepping 04
(XEN) Total of 4 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using new ACK method
(XEN) checking TSC synchronization across 4 CPUs: passed.
(XEN) Platform timer is 3.579MHz ACPI PM Timer
(XEN) Brought up 4 CPUs
(XEN) I/O virtualisation disabled
(XEN) mtrr: your CPUs had inconsistent fixed MTRR settings
(XEN) *** LOADING DOMAIN 0 ***
(XEN)  Xen  kernel: 32-bit, PAE, lsb
(XEN)  Dom0 kernel: 32-bit, PAE, lsb, paddr 0xc0100000 -> 0xc0ab54fc
(XEN)  Dom0 symbol map 0xc0ab54fc -> 0xc0b89dc8
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   000000003e000000->000000003f000000 (12288 pages to be 
allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: c0100000->c0b89dc8
(XEN)  Init. ramdisk: c0b8a000->c0b8a000
(XEN)  Phys-Mach map: c0b8a000->c0b9a000
(XEN)  Start info:    c0b9a000->c0b9a474
(XEN)  Page tables:   c0b9b000->c0ba8000
(XEN)  Boot stack:    c0ba8000->c0ba9000
(XEN)  TOTAL:         c0000000->c1000000
(XEN)  ENTRY ADDRESS: c0100000
(XEN) Dom0 has maximum 4 VCPUs
(XEN) Scrubbing Free RAM: 
................................................................................................................................................................done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to 
Xen)
(XEN) Freed 96kB init memory.
(XEN) ioapic_guest_write: apic=0, pin=2, old_irq=0, new_irq=-1
(XEN) ioapic_guest_write: old_entry=000009f0, new_entry=00010900
(XEN) ioapic_guest_write: Attempt to remove IO-APIC pin of in-use IRQ!


Home | Main Index | Thread Index | Old Index