port-xen/49549: 6.1.5 dom0 reports "debug event", followed by lvm volumes becoming non-responsive

[mbowie%rocket-space.com@localhost]

>Number:         49549
>Category:       port-xen
>Synopsis:       6.1.5 dom0 reports "debug event", followed by lvm volumes becoming non-responsive
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    port-xen-maintainer
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Jan 08 23:40:00 +0000 2015
>Originator:     Mike Bowie
>Release:        6.1.5
>Organization:
RocketSpace, Inc.
>Environment:
NetBSD mike03.restricted.sfo5.rocketstre.am 6.1.5 NetBSD 6.1.5 (XEN3_DOM0+AGR+VLAN+NMBCLUSTERS) #0: Thu Dec  4 01:34:45 UTC 2014  root%netbsd6-1-5-amd64.restricted.sfo3.rocketstre.am@localhost:/usr/obj/sys/arch/amd64/compile/XEN3_DOM0+AGR+VLAN+NMBCLUSTERS amd64
>Description:
Initial symptom was a selection of services stopping on some of the device's domU's. Some started reporting their primary volume as full, others would not respond at a console. Attempting to run "xl dest <domUid>" resulted in (null) "xl list" entries, and trying to restart the domain failed with a "Device busy" message in the xl debug for the domain.

Behavior began shortly after addition of a new vlan interface, and matching bridge. (Something which happens from time to time.) No output in syslog, but dmesg has a final entry reading:
debug event
ci_ilevel 0x0 ci_ipending 0x0 ci_idepth 0
evtchn_upcall_pending 1 evtchn_upcall_mask 1 evtchn_pending_sel 0x1
evtchn_mask f0fe387feb40003 ffffffffffff7ce0 ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffff
 ffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff
evtchn_pending 100200000140000 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

While attempting to produce further debug (over an hour period) additional domU's started to stall and ultimately the dom0 had to be hard reset to recover.

The device was installed about a month ago, and has been running smoothly since then. Hardware is a Dell PE 720, with dmesg below.

$ pkg_info | grep xen
xenkernel42-4.2.5nb1 Xen 4.2.x Kernel
xentools42-4.2.5    Userland Tools for Xen 4.2.x

$ dmesg
Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
    2006, 2007, 2008, 2009, 2010, 2011, 2012
    The NetBSD Foundation, Inc.  All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
    The Regents of the University of California.  All rights reserved.

NetBSD 6.1.5 (XEN3_DOM0+AGR+VLAN+NMBCLUSTERS) #0: Thu Dec  4 01:34:45 UTC 2014
        root%netbsd6-1-5-amd64.restricted.sfo3.rocketstre.am@localhost:/usr/obj/sys/arch/amd64/compile/XEN3_DOM0+AGR+VLAN+NMBCLUSTERS
total memory = 2048 MB
avail memory = 1968 MB
timecounter: Timecounters tick every 10.000 msec
mainbus0 (root)
cpu0 at mainbus0 apid 0
cpu1 at mainbus0 apid 32
cpu2 at mainbus0 apid 2
cpu3 at mainbus0 apid 34
cpu4 at mainbus0 apid 4
cpu5 at mainbus0 apid 36
cpu6 at mainbus0 apid 6
cpu7 at mainbus0 apid 38
cpu8 at mainbus0 apid 8
cpu9 at mainbus0 apid 40
cpu10 at mainbus0 apid 10
cpu11 at mainbus0 apid 42
cpu12 at mainbus0 apid 1
cpu13 at mainbus0 apid 33
cpu14 at mainbus0 apid 3
cpu15 at mainbus0 apid 35
cpu16 at mainbus0 apid 5
cpu17 at mainbus0 apid 37
cpu18 at mainbus0 apid 7
cpu19 at mainbus0 apid 39
cpu20 at mainbus0 apid 9
cpu21 at mainbus0 apid 41
cpu22 at mainbus0 apid 11
cpu23 at mainbus0 apid 43
ioapic0 at mainbus0 apid 0: pa 0xfec00000, version 20, 24 pins
ioapic1 at mainbus0 apid 1: pa 0xfec3f000, version 20, 24 pins
ioapic1: can't remap to apid 1
ioapic2 at mainbus0 apid 2: pa 0xfec7f000, version 20, 24 pins
ioapic2: can't remap to apid 2
ipmi0 at mainbus0
hypervisor0 at mainbus0: Xen version 4.2.5
VIRQ_DEBUG interrupt using event channel 1
vcpu0 at hypervisor0: Intel(R) Xeon(R) CPU E5-2630L 0 @ 2.00GHz, id 0x206d7
xenbus0 at hypervisor0: Xen Virtual Bus Interface
xencons0 at hypervisor0: Xen Virtual Console Driver
xencons0: console major 143, unit 0
xencons0: using event channel 2
acpi0 at hypervisor0: Intel ACPICA 20110623
acpi0: X/RSDT: OemId <DELL  ,PE_SC3  ,00000001>, AslId <DELL,00000001>
acpi0: SCI interrupting at int 9
timecounter: Timecounter "ACPI-Safe" frequency 3579545 Hz quality 900
WHEA (PNP0C33) at acpi0 not configured
PMI0 (ACPI000D) at acpi0 not configured
SPK (PNP0C01) at acpi0 not configured
TMR (PNP0100) at acpi0 not configured
COMA (PNP0501) at acpi0 not configured
COMB (PNP0501) at acpi0 not configured
MBIO (PNP0C01) at acpi0 not configured
NIPM (IPI0001) at acpi0 not configured
PEHB (PNP0C02) at acpi0 not configured
PEHB (PNP0C02) at acpi0 not configured
PEHB (PNP0C02) at acpi0 not configured
VTD1 (PNP0C02) at acpi0 not configured
VTD2 (PNP0C02) at acpi0 not configured
HPET (PNP0103) at acpi0 not configured
SRIO (PNP0C14) at acpi0 not configured
pci0 at hypervisor0 bus 0: configuration mode 1
pci0: i/o space, memory space enabled
pchb0 at pci0 dev 0 function 0: vendor 0x8086 product 0x3c00 (rev. 0x07)
ppb0 at pci0 dev 1 function 0: vendor 0x8086 product 0x3c02 (rev. 0x07)
ppb0: PCI Express 2.0 <Root Port of PCI-E Root Complex>
pci1 at ppb0 bus 1
pci1: i/o space, memory space enabled
wm0 at pci1 dev 0 function 0: I350 Gigabit Network Connection (rev. 0x01)
wm0: interrupting at ioapic1 pin 6, event channel 4
wm0: PCI-Express bus
wm0: 65536 word (16 address bits) SPI EEPROM
wm0: Ethernet address b8:ca:3a:70:fc:30
ukphy0 at wm0 phy 1: OUI 0x00aa00, model 0x003b, rev. 1
ukphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
wm1 at pci1 dev 0 function 1: I350 Gigabit Network Connection (rev. 0x01)
wm1: interrupting at ioapic1 pin 5, event channel 5
wm1: PCI-Express bus
wm1: 256 word (8 address bits) SPI EEPROM
wm1: Ethernet address b8:ca:3a:70:fc:31
ukphy1 at wm1 phy 1: OUI 0x00aa00, model 0x003b, rev. 1
ukphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
wm2 at pci1 dev 0 function 2: I350 Gigabit Network Connection (rev. 0x01)
wm2: interrupting at ioapic1 pin 4, event channel 6
wm2: PCI-Express bus
wm2: 256 word (8 address bits) SPI EEPROM
wm2: Ethernet address b8:ca:3a:70:fc:32
ukphy2 at wm2 phy 1: OUI 0x00aa00, model 0x003b, rev. 1
ukphy2: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
wm3 at pci1 dev 0 function 3: I350 Gigabit Network Connection (rev. 0x01)
wm3: interrupting at ioapic1 pin 2, event channel 7
wm3: PCI-Express bus
wm3: 256 word (8 address bits) SPI EEPROM
wm3: Ethernet address b8:ca:3a:70:fc:33
ukphy3 at wm3 phy 1: OUI 0x00aa00, model 0x003b, rev. 1
ukphy3: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
ppb1 at pci0 dev 2 function 0: vendor 0x8086 product 0x3c04 (rev. 0x07)
ppb1: PCI Express 2.0 <Root Port of PCI-E Root Complex>
pci2 at ppb1 bus 3
pci2: i/o space, memory space enabled
ppb2 at pci0 dev 2 function 2: vendor 0x8086 product 0x3c06 (rev. 0x07)
ppb2: PCI Express 2.0 <Root Port of PCI-E Root Complex>
pci3 at ppb2 bus 2
pci3: i/o space, memory space enabled
mfi0 at pci3 dev 0 function 0: Dell PERC H710P Mini (blades)
mfi0: interrupting at ioapic1 pin 10, event channel 8
mfi0: PERC H710P Mini version 21.3.0-0009
mfi0: logical drives 1, 1024MB RAM, BBU type BBU, status good
scsibus0 at mfi0: 64 targets, 8 luns per target
ppb3 at pci0 dev 3 function 0: vendor 0x8086 product 0x3c08 (rev. 0x07)
ppb3: PCI Express 2.0 <Root Port of PCI-E Root Complex>
pci4 at ppb3 bus 4
pci4: i/o space, memory space enabled
ppb4 at pci0 dev 3 function 2: vendor 0x8086 product 0x3c0a (rev. 0x07)
ppb4: PCI Express 2.0 <Root Port of PCI-E Root Complex>
pci5 at ppb4 bus 5
pci5: i/o space, memory space enabled
vendor 0x8086 product 0x3c28 (miscellaneous system, revision 0x07) at pci0 dev 5 function 0 not configured
vendor 0x8086 product 0x3c2a (miscellaneous system, revision 0x07) at pci0 dev 5 function 2 not configured
ppb5 at pci0 dev 17 function 0: vendor 0x8086 product 0x1d3e (rev. 0x05)
ppb5: PCI Express 2.0 <Root Port of PCI-E Root Complex>
pci6 at ppb5 bus 6
pci6: i/o space, memory space enabled
vendor 0x8086 product 0x1d3a (miscellaneous communications, revision 0x05) at pci0 dev 22 function 0 not configured
vendor 0x8086 product 0x1d3b (miscellaneous communications, revision 0x05) at pci0 dev 22 function 1 not configured
ehci0 at pci0 dev 26 function 0: vendor 0x8086 product 0x1d2d (rev. 0x05)
ehci0: interrupting at ioapic0 pin 23, event channel 9
ehci0: BIOS has given up ownership
ehci0: EHCI version 1.0
usb0 at ehci0: USB revision 2.0
ppb6 at pci0 dev 28 function 0: vendor 0x8086 product 0x1d10 (rev. 0xb5)
ppb6: PCI Express 2.0 <Root Port of PCI-E Root Complex>
pci7 at ppb6 bus 7
pci7: i/o space, memory space enabled
ppb7 at pci0 dev 28 function 7: vendor 0x8086 product 0x1d1e (rev. 0xb5)
ppb7: PCI Express 2.0 <Root Port of PCI-E Root Complex>
pci8 at ppb7 bus 8
pci8: i/o space, memory space enabled
ppb8 at pci8 dev 0 function 0: vendor 0x1912 product 0x0013 (rev. 0x00)
ppb8: PCI Express 2.0 <Upstream Port of PCI-E Switch>
pci9 at ppb8 bus 9
pci9: i/o space, memory space enabled
ppb9 at pci9 dev 0 function 0: vendor 0x1912 product 0x0013 (rev. 0x00)
ppb9: PCI Express 2.0 <Downstream Port of PCI-E Switch>
pci10 at ppb9 bus 10
pci10: i/o space, memory space enabled
ppb10 at pci10 dev 0 function 0: vendor 0x1912 product 0x0012 (rev. 0x00)
ppb10: PCI Express 2.0 <PCI-E to PCI/PCI-X Bridge>
pci11 at ppb10 bus 11
pci11: i/o space, memory space enabled
vga0 at pci11 dev 0 function 0: vendor 0x102b product 0x0534 (rev. 0x00)
wsdisplay0 at vga0 kbdmux 1
wsmux1: connecting to wsdisplay0
drm at vga0 not configured
ppb11 at pci9 dev 1 function 0: vendor 0x1912 product 0x0013 (rev. 0x00)
ppb11: PCI Express 2.0 <Downstream Port of PCI-E Switch>
pci12 at ppb11 bus 12
pci12: i/o space, memory space enabled
ehci1 at pci0 dev 29 function 0: vendor 0x8086 product 0x1d26 (rev. 0x05)
ehci1: interrupting at ioapic0 pin 22, event channel 10
ehci1: EHCI version 1.0
usb1 at ehci1: USB revision 2.0
ppb12 at pci0 dev 30 function 0: vendor 0x8086 product 0x244e (rev. 0xa5)
pci13 at ppb12 bus 13
pci13: i/o space, memory space enabled
pcib0 at pci0 dev 31 function 0: vendor 0x8086 product 0x1d41 (rev. 0x05)
ahcisata0 at pci0 dev 31 function 2: vendor 0x8086 product 0x1d02 (rev. 0x05)
ahcisata0: interrupting at ioapic0 pin 20, event channel 11
ahcisata0: 64-bit DMA
ahcisata0: AHCI revision 1.30, 6 ports, 32 slots, CAP 0xe720ff45<EMS,PSC,SSC,PMD,ISS=0x2=Gen2,SCLO,SAL,SALP,SSNTF,SNCQ,S64A>
atabus0 at ahcisata0 channel 0
atabus1 at ahcisata0 channel 1
atabus2 at ahcisata0 channel 2
atabus3 at ahcisata0 channel 3
atabus4 at ahcisata0 channel 4
atabus5 at ahcisata0 channel 5
isa0 at pcib0
com1 at isa0 port 0x2f8-0x2ff irq 3: ns16550a, working fifo
pci14 at hypervisor0 bus 63
pci14: i/o space, memory space enabled
vendor 0x8086 product 0x3c80 (miscellaneous system, revision 0x07) at pci14 dev 8 function 0 not configured
vendor 0x8086 product 0x3c90 (miscellaneous system, revision 0x07) at pci14 dev 9 function 0 not configured
vendor 0x8086 product 0x3cc0 (miscellaneous system, revision 0x07) at pci14 dev 10 function 0 not configured
vendor 0x8086 product 0x3cc1 (miscellaneous system, revision 0x07) at pci14 dev 10 function 1 not configured
vendor 0x8086 product 0x3cc2 (miscellaneous system, revision 0x07) at pci14 dev 10 function 2 not configured
vendor 0x8086 product 0x3cd0 (miscellaneous system, revision 0x07) at pci14 dev 10 function 3 not configured
vendor 0x8086 product 0x3ce0 (miscellaneous system, revision 0x07) at pci14 dev 11 function 0 not configured
vendor 0x8086 product 0x3ce3 (miscellaneous system, revision 0x07) at pci14 dev 11 function 3 not configured
vendor 0x8086 product 0x3ce8 (miscellaneous system, revision 0x07) at pci14 dev 12 function 0 not configured
vendor 0x8086 product 0x3ce8 (miscellaneous system, revision 0x07) at pci14 dev 12 function 1 not configured
vendor 0x8086 product 0x3ce8 (miscellaneous system, revision 0x07) at pci14 dev 12 function 2 not configured
vendor 0x8086 product 0x3cf4 (miscellaneous system, revision 0x07) at pci14 dev 12 function 6 not configured
vendor 0x8086 product 0x3cf6 (miscellaneous system, revision 0x07) at pci14 dev 12 function 7 not configured
vendor 0x8086 product 0x3ce8 (miscellaneous system, revision 0x07) at pci14 dev 13 function 0 not configured
vendor 0x8086 product 0x3ce8 (miscellaneous system, revision 0x07) at pci14 dev 13 function 1 not configured
vendor 0x8086 product 0x3ce8 (miscellaneous system, revision 0x07) at pci14 dev 13 function 2 not configured
vendor 0x8086 product 0x3cf5 (miscellaneous system, revision 0x07) at pci14 dev 13 function 6 not configured
vendor 0x8086 product 0x3ca0 (miscellaneous system, revision 0x07) at pci14 dev 14 function 0 not configured
vendor 0x8086 product 0x3c46 (Time and Frequency DASP, revision 0x07) at pci14 dev 14 function 1 not configured
vendor 0x8086 product 0x3ca8 (miscellaneous system, revision 0x07) at pci14 dev 15 function 0 not configured
vendor 0x8086 product 0x3c71 (miscellaneous system, revision 0x07) at pci14 dev 15 function 1 not configured
vendor 0x8086 product 0x3caa (miscellaneous system, revision 0x07) at pci14 dev 15 function 2 not configured
vendor 0x8086 product 0x3cab (miscellaneous system, revision 0x07) at pci14 dev 15 function 3 not configured
vendor 0x8086 product 0x3cac (miscellaneous system, revision 0x07) at pci14 dev 15 function 4 not configured
vendor 0x8086 product 0x3cad (miscellaneous system, revision 0x07) at pci14 dev 15 function 5 not configured
vendor 0x8086 product 0x3cae (miscellaneous system, revision 0x07) at pci14 dev 15 function 6 not configured
vendor 0x8086 product 0x3cb0 (miscellaneous system, revision 0x07) at pci14 dev 16 function 0 not configured
vendor 0x8086 product 0x3cb1 (miscellaneous system, revision 0x07) at pci14 dev 16 function 1 not configured
vendor 0x8086 product 0x3cb2 (miscellaneous system, revision 0x07) at pci14 dev 16 function 2 not configured
vendor 0x8086 product 0x3cb3 (miscellaneous system, revision 0x07) at pci14 dev 16 function 3 not configured
vendor 0x8086 product 0x3cb4 (miscellaneous system, revision 0x07) at pci14 dev 16 function 4 not configured
vendor 0x8086 product 0x3cb5 (miscellaneous system, revision 0x07) at pci14 dev 16 function 5 not configured
vendor 0x8086 product 0x3cb6 (miscellaneous system, revision 0x07) at pci14 dev 16 function 6 not configured
vendor 0x8086 product 0x3cb7 (miscellaneous system, revision 0x07) at pci14 dev 16 function 7 not configured
vendor 0x8086 product 0x3cb8 (miscellaneous system, revision 0x07) at pci14 dev 17 function 0 not configured
vendor 0x8086 product 0x3ce4 (miscellaneous system, revision 0x07) at pci14 dev 19 function 0 not configured
vendor 0x8086 product 0x3c43 (Time and Frequency DASP, revision 0x07) at pci14 dev 19 function 1 not configured
vendor 0x8086 product 0x3ce6 (Time and Frequency DASP, revision 0x07) at pci14 dev 19 function 4 not configured
vendor 0x8086 product 0x3c44 (Time and Frequency DASP, revision 0x07) at pci14 dev 19 function 5 not configured
vendor 0x8086 product 0x3c45 (miscellaneous system, revision 0x07) at pci14 dev 19 function 6 not configured
pci15 at hypervisor0 bus 64
pci15: i/o space, memory space enabled
ppb13 at pci15 dev 1 function 0: vendor 0x8086 product 0x3c02 (rev. 0x07)
ppb13: PCI Express 2.0 <Root Port of PCI-E Root Complex>
pci16 at ppb13 bus 65
pci16: i/o space, memory space enabled
ppb14 at pci15 dev 2 function 0: vendor 0x8086 product 0x3c04 (rev. 0x07)
ppb14: PCI Express 2.0 <Root Port of PCI-E Root Complex>
pci17 at ppb14 bus 66
pci17: i/o space, memory space enabled
ppb15 at pci15 dev 3 function 0: vendor 0x8086 product 0x3c08 (rev. 0x07)
ppb15: PCI Express 2.0 <Root Port of PCI-E Root Complex>
pci18 at ppb15 bus 67
pci18: i/o space, memory space enabled
ppb16 at pci15 dev 3 function 2: vendor 0x8086 product 0x3c0a (rev. 0x07)
ppb16: PCI Express 2.0 <Root Port of PCI-E Root Complex>
pci19 at ppb16 bus 68
pci19: i/o space, memory space enabled
vendor 0x8086 product 0x3c28 (miscellaneous system, revision 0x07) at pci15 dev 5 function 0 not configured
vendor 0x8086 product 0x3c2a (miscellaneous system, revision 0x07) at pci15 dev 5 function 2 not configured
pci20 at hypervisor0 bus 127
pci20: i/o space, memory space enabled
vendor 0x8086 product 0x3c80 (miscellaneous system, revision 0x07) at pci20 dev 8 function 0 not configured
vendor 0x8086 product 0x3c90 (miscellaneous system, revision 0x07) at pci20 dev 9 function 0 not configured
vendor 0x8086 product 0x3cc0 (miscellaneous system, revision 0x07) at pci20 dev 10 function 0 not configured
vendor 0x8086 product 0x3cc1 (miscellaneous system, revision 0x07) at pci20 dev 10 function 1 not configured
vendor 0x8086 product 0x3cc2 (miscellaneous system, revision 0x07) at pci20 dev 10 function 2 not configured
vendor 0x8086 product 0x3cd0 (miscellaneous system, revision 0x07) at pci20 dev 10 function 3 not configured
vendor 0x8086 product 0x3ce0 (miscellaneous system, revision 0x07) at pci20 dev 11 function 0 not configured
vendor 0x8086 product 0x3ce3 (miscellaneous system, revision 0x07) at pci20 dev 11 function 3 not configured
vendor 0x8086 product 0x3ce8 (miscellaneous system, revision 0x07) at pci20 dev 12 function 0 not configured
vendor 0x8086 product 0x3ce8 (miscellaneous system, revision 0x07) at pci20 dev 12 function 1 not configured
vendor 0x8086 product 0x3ce8 (miscellaneous system, revision 0x07) at pci20 dev 12 function 2 not configured
vendor 0x8086 product 0x3cf4 (miscellaneous system, revision 0x07) at pci20 dev 12 function 6 not configured
vendor 0x8086 product 0x3cf6 (miscellaneous system, revision 0x07) at pci20 dev 12 function 7 not configured
vendor 0x8086 product 0x3ce8 (miscellaneous system, revision 0x07) at pci20 dev 13 function 0 not configured
vendor 0x8086 product 0x3ce8 (miscellaneous system, revision 0x07) at pci20 dev 13 function 1 not configured
vendor 0x8086 product 0x3ce8 (miscellaneous system, revision 0x07) at pci20 dev 13 function 2 not configured
vendor 0x8086 product 0x3cf5 (miscellaneous system, revision 0x07) at pci20 dev 13 function 6 not configured
vendor 0x8086 product 0x3ca0 (miscellaneous system, revision 0x07) at pci20 dev 14 function 0 not configured
vendor 0x8086 product 0x3c46 (Time and Frequency DASP, revision 0x07) at pci20 dev 14 function 1 not configured
vendor 0x8086 product 0x3ca8 (miscellaneous system, revision 0x07) at pci20 dev 15 function 0 not configured
vendor 0x8086 product 0x3c71 (miscellaneous system, revision 0x07) at pci20 dev 15 function 1 not configured
vendor 0x8086 product 0x3caa (miscellaneous system, revision 0x07) at pci20 dev 15 function 2 not configured
vendor 0x8086 product 0x3cab (miscellaneous system, revision 0x07) at pci20 dev 15 function 3 not configured
vendor 0x8086 product 0x3cac (miscellaneous system, revision 0x07) at pci20 dev 15 function 4 not configured
vendor 0x8086 product 0x3cad (miscellaneous system, revision 0x07) at pci20 dev 15 function 5 not configured
vendor 0x8086 product 0x3cae (miscellaneous system, revision 0x07) at pci20 dev 15 function 6 not configured
vendor 0x8086 product 0x3cb0 (miscellaneous system, revision 0x07) at pci20 dev 16 function 0 not configured
vendor 0x8086 product 0x3cb1 (miscellaneous system, revision 0x07) at pci20 dev 16 function 1 not configured
vendor 0x8086 product 0x3cb2 (miscellaneous system, revision 0x07) at pci20 dev 16 function 2 not configured
vendor 0x8086 product 0x3cb3 (miscellaneous system, revision 0x07) at pci20 dev 16 function 3 not configured
vendor 0x8086 product 0x3cb4 (miscellaneous system, revision 0x07) at pci20 dev 16 function 4 not configured
vendor 0x8086 product 0x3cb5 (miscellaneous system, revision 0x07) at pci20 dev 16 function 5 not configured
vendor 0x8086 product 0x3cb6 (miscellaneous system, revision 0x07) at pci20 dev 16 function 6 not configured
vendor 0x8086 product 0x3cb7 (miscellaneous system, revision 0x07) at pci20 dev 16 function 7 not configured
vendor 0x8086 product 0x3cb8 (miscellaneous system, revision 0x07) at pci20 dev 17 function 0 not configured
vendor 0x8086 product 0x3ce4 (miscellaneous system, revision 0x07) at pci20 dev 19 function 0 not configured
vendor 0x8086 product 0x3c43 (Time and Frequency DASP, revision 0x07) at pci20 dev 19 function 1 not configured
vendor 0x8086 product 0x3ce6 (Time and Frequency DASP, revision 0x07) at pci20 dev 19 function 4 not configured
vendor 0x8086 product 0x3c44 (Time and Frequency DASP, revision 0x07) at pci20 dev 19 function 5 not configured
vendor 0x8086 product 0x3c45 (miscellaneous system, revision 0x07) at pci20 dev 19 function 6 not configured
timecounter: Timecounter "clockinterrupt" frequency 100 Hz quality 0
timecounter: Timecounter "xen_system_time" frequency 1000000000 Hz quality 10000
Xen clock: using event channel 13
xenbus0: using event channel 14
scsibus0: waiting 2 seconds for devices to settle...
uhub0 at usb0: vendor 0x8086 EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
uhub1 at usb1: vendor 0x8086 EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub1: 2 ports with 2 removable, self powered
ahcisata0 port 4: device present, speed: 1.5Gb/s
atapibus0 at atabus4: 1 targets
cd0 at atapibus0 drive 0: <TEAC DVD-ROM DV-28SW, 10051007220133, R.2A> cdrom removable
cd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 5 (Ultra/100)
cd0(ahcisata0:4:0): using PIO mode 4, DMA mode 2, Ultra-DMA mode 5 (Ultra/100) (using DMA)
sd0 at scsibus0 target 0 lun 0: <DELL, PERC H710P, 3.13> disk fixed
sd0: fabricating a geometry
sd0: 1394 GB, 1427840 cyl, 64 head, 32 sec, 512 bytes/sect x 2924216320 sectors
sd0: fabricating a geometry
sd0: tagged queueing
uhub2 at uhub0 port 1: vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2
uhub2: single transaction translator
uhub3 at uhub1 port 1: vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2
uhub3: single transaction translator
uhub2: 6 ports with 6 removable, self powered
uhub3: 8 ports with 8 removable, self powered
uhub4 at uhub2 port 6: no manufacturer Gadget USB HUB, class 9/0, rev 2.00/0.00, addr 3
uhub4: 6 ports with 6 removable, self powered
uhidev0 at uhub4 port 1 configuration 1 interface 0
uhidev0: Avocent Keyboard/Mouse Function, rev 2.00/0.00, addr 4, iclass 3/1
ukbd0 at uhidev0
wskbd0 at ukbd0 mux 1
wskbd0: connecting to wsdisplay0
uhidev1 at uhub4 port 1 configuration 1 interface 1
uhidev1: Avocent Keyboard/Mouse Function, rev 2.00/0.00, addr 4, iclass 3/1
ums0 at uhidev1: 3 buttons and Z dir
wsmouse0 at ums0 mux 0
uhidev2 at uhub4 port 1 configuration 1 interface 2
uhidev2: Avocent Keyboard/Mouse Function, rev 2.00/0.00, addr 4, iclass 3/1
ums1 at uhidev2: 3 buttons and Z dir
wsmouse1 at ums1 mux 0
Kernelized RAIDframe activated
boot device: sd0
root on sd0a dumps on sd0b
Your machine does not initialize mem_clusters; sparse_dumps disabled
root file system type: ffs
wm1: cannot assign link-local address
wm1: cannot assign link-local address
wm2: cannot assign link-local address
wm2: cannot assign link-local address
mfi0: normal state on 'mfi0:0' (online)
ipmi0: version 2.0 interface KCS iobase 0xca8/8 spacing 4
ipmi0:  irq 10
balloon0 at xenbus0 id 0: Xen Balloon driver
balloon0: current reservation: 2097152 KiB
balloon0: current reservation: 524288 pages => target: 524288 pages
xbd backend: attach device vg00-cdist01 (size 10485760) for domain 1
xvif1i0: Ethernet address 00:16:3e:36:fa:0f
xvif1i1: Ethernet address 00:16:3e:49:02:10
xbd backend: attach device vg00-cdist--tes (size 10485760) for domain 2
xbd backend domain 1 handle 0x1 (1) using event channel 21, protocol x86_64-abi
xvif2i0: Ethernet address 00:16:3e:49:54:59
xvif2i1: Ethernet address 00:16:3e:1e:5c:59
xbd backend: attach device vg00-kibana01 (size 104857600) for domain 4
xvif4i0: Ethernet address 00:16:3e:77:ca:aa
xbd backend: attach device vg00-netbsd6--1 (size 62914560) for domain 5
xvif5i0: Ethernet address 00:16:3e:28:97:da
xbd backend domain 2 handle 0x1 (1) using event channel 28, protocol x86_64-abi
xbd backend domain 4 handle 0x1 (1) using event channel 29, protocol x86_64-abi
xbd backend domain 5 handle 0x1 (1) using event channel 30, protocol x86_64-abi
xbd backend: attach device vg00-netbsd7--0 (size 62914560) for domain 6
xvif6i0: Ethernet address 00:16:3e:46:29:0c
xbd backend domain 6 handle 0x1 (1) using event channel 37, protocol x86_64-abi
xbd backend: attach device vg00-flexmaster (size 20971520) for domain 10
xvif10i0: Ethernet address 00:16:3e:80:2e:86
xbd backend domain 10 handle 0xca00 (51712) using event channel 41, protocol x86_64-abi
xbd backend: attach device vg00-noc01 (size 41943040) for domain 14
xvif14i0: Ethernet address 00:16:3e:11:5f:ae
xvif14i1: Ethernet address 00:16:3e:11:5f:af
xbd backend domain 14 handle 0xca00 (51712) using event channel 44, protocol x86_64-abi
>How-To-Repeat:
Did not reoccur following reboot.
>Fix:
n/a