Subject: Re: a process hanged?
To: None <tech-smp@netbsd.org>
From: Kazushi (Jam) Marukawa <jam@pobox.com>
List: tech-smp
Date: 11/18/2001 17:13:07
   On Nov 18, 16:24, Kazushi (Jam) Marukawa wrote:
   > Subject: Re: a process hanged?
   >    On Nov 18, 21:40, Martin Husemann wrote:
   >    > Subject: Re: a process hanged?
   >    > I've seen this. To track it down I added options LOCKDEBUG
   >    > to my kernel (and a few more debug options) and installed
   >    > a serial console - since then the machine is running rock solid
   > 
   > Thanks the tips.  I'll try to use GENERIC.MPDEBUG instead of
   > GENERIC.MP as the base of my configuration file to catch the
   > reason of this problem.

I made new kernel, but it doesn't work well.  It looks like
kernel diagnostic routines found deadlock or something, but
I'm not sure.  Where should I start to look over?  Thanks.

My configuration file is:

  include "arch/i386/conf/GENERIC.MPDEBUG"
  options 	NMBCLUSTERS=1024
  options 	GATEWAY		# packet forwarding
  pseudo-device	raid		4	# RAIDframe disk driver
  options 	RAID_AUTOCONFIG		# auto-configuration of RAID components

And console shows

>> NetBSD/i386 BIOS Boot, Revision 2.7
>> (jam@sou.nerv.org, Sat Aug 26 00:15:53 CDT 2000)
>> Memory: 634/785396 k
Use hd1a:netbsd to boot sd0 when wd0 is also installed
Press return to boot now, any other key for boot menu
booting wd0a:netbsd - starting in 0
5476160+107848+351040 [65+294480+247787]=0x62e8cc
[ using 542788 bytes of netbsd ELF symbol table ]
BIOS CFG: Model-SubMod-Rev: fc-01-00, 0x70<KBDINT,RTC,IC2>
Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001
    The NetBSD Foundation, Inc.  All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
    The Regents of the University of California.  All rights reserved.

NetBSD 1.5Y (sou.mpdebug) #0: Sun Nov 18 16:33:53 CST 2001
    jam@sou.nerv.org:/mnt/src/syssmp/arch/i386/compile/sou.mpdebug
total memory = 767 MB
avail memory = 696 MB
using 9850 buffers containing 39400 KB of memory
BIOS32 rev. 0 found at 0xf0530
mainbus0 (root)
mainbus0: scanning 0x9e400 to 0x9e7f0 for MP signature
mainbus0: scanning 0xf0000 to 0xffff0 for MP signature
mainbus0: MP floating pointer found in bios at 0xf6e90
mainbus0: MP config table at 0xf6a80, 244 bytes long
mainbus0: Intel MP Specification (Version 1.1) (OEM00000 PROD00000000)
cpu0 at mainbus0: apid 1 (boot processor)
cpu0: Intel Pentium III (Coppermine) Pentium III (686-class), 701.63 MHz
cpu0: features 383fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR>
cpu0: features 383fbff<PGE,MCA,CMOV,FGPAT,PSE36,MMX,FXSR,SSE>
cpu0: I-cache 16 KB 32b/line 4-way, D-cache 16 KB 32b/line 4-way
cpu0: L2 cache 256 KB 32b/line 8-way
cpu0: ITLB 32 4 KB entries 4-way, 2 4 MB entries fully associative
cpu0: DTLB 64 4 KB entries 4-way, 8 4 MB entries 4-way
cpu0: calibrating local timer
cpu0: apic clock running at 100 MHz
cpu0: 8 page colors
cpu0: kstack at 0xf2118000 for 8192 bytes
cpu0: idle pcb at 0xf2118000, idle sp at 0xf2119f98
cpu1 at mainbus0: apid 0 (application processor)
cpu1: starting
cpu1: Intel Pentium III (Coppermine) Pentium III (686-class), 701.60 MHz
cpu1: features 383fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR>
cpu1: features 383fbff<PGE,MCA,CMOV,FGPAT,PSE36,MMX,FXSR,SSE>
cpu1: I-cache 16 KB 32b/line 4-way, D-cache 16 KB 32b/line 4-way
cpu1: L2 cache 256 KB 32b/line 8-way
cpu1: ITLB 32 4 KB entries 4-way, 2 4 MB entries fully associative
cpu1: DTLB 64 4 KB entries 4-way, 8 4 MB entries 4-way
cpu1: kstack at 0xf211a000 for 8192 bytes
cpu1: idle pcb at 0xf211a000, idle sp at 0xf211bf98
mpbios: bus 0 is type PCI
mpbios: bus 1 is type ISA
ioapic0 at mainbus0 apid 2 (I/O APIC)
ioapic0: pa 0xfec00000, virtual wire mode, version 11, 24 pins
ioapic0: int0 attached to ExtINT (type 3<type=3=ExtINT> flags 0<pol=0,trig=0>)
ioapic0: int1 attached to isa0 irq 1 (type 0<type=0> flags 0<pol=0,trig=0>)
ioapic0: int2 attached to isa0 irq 0 (type 0<type=0> flags 0<pol=0,trig=0>)
ioapic0: int3 attached to isa0 irq 3 (type 0<type=0> flags 0<pol=0,trig=0>)
ioapic0: int4 attached to isa0 irq 4 (type 0<type=0> flags 0<pol=0,trig=0>)
ioapic0: int6 attached to isa0 irq 6 (type 0<type=0> flags 0<pol=0,trig=0>)
ioapic0: int7 attached to isa0 irq 7 (type 0<type=0> flags 0<pol=0,trig=0>)
ioapic0: int8 attached to isa0 irq 8 (type 0<type=0> flags 0<pol=0,trig=0>)
ioapic0: int9 attached to isa0 irq 9 (type 0<type=0> flags 0<pol=0,trig=0>)
ioapic0: int14 attached to isa0 irq 14 (type 0<type=0> flags 0<pol=0,trig=0>)
ioapic0: int15 attached to isa0 irq 15 (type 0<type=0> flags 0<pol=0,trig=0>)
ioapic0: int16 attached to isa0 irq 11 (type 0<type=0> flags f<pol=3=Act Lo,trig
=3=Level>)
ioapic0: int17 attached to isa0 irq 10 (type 0<type=0> flags f<pol=3=Act Lo,trig
=3=Level>)
ioapic0: int18 attached to isa0 irq 12 (type 0<type=0> flags f<pol=3=Act Lo,trig
=3=Level>)
ioapic0: int19 attached to isa0 irq 5 (type 0<type=0> flags f<pol=3=Act Lo,trig=
3=Level>)
local apic: int0 attached to ExtINT (type 3<type=3=ExtINT> flags 5<pol=1=Act Hi,
trig=1=Edge>)
local apic: int1 attached to NMI (type 1<type=1=NMI> flags 5<pol=1=Act Hi,trig=1
=Edge>)
pci0 at mainbus0 bus 0: configuration mode 1
pci0: i/o space, memory space enabled
pchb0 at pci0 dev 0 function 0
pchb0: Intel 82443BX Host Bridge/Controller (rev. 0x03)
agp at pchb0 not configured
ppb0 at pci0 dev 1 function 0: Intel 82443BX AGP Interface (rev. 0x03)
pci1 at ppb0 bus 1
pci1: i/o space, memory space enabled
pcib0 at pci0 dev 4 function 0
pcib0: Intel 82371AB PCI-to-ISA Bridge (PIIX4) (rev. 0x02)
pciide0 at pci0 dev 4 function 1: Intel 82371AB IDE controller (PIIX4) (rev. 0x0
1)
pciide0: bus-master DMA support present
pciide0: primary channel wired to compatibility mode
wd0 at pciide0 channel 0 drive 0: <Maxtor 92049U4>
wd0: drive supports 16-sector PIO transfers, LBA addressing
wd0: 19541 MB, 16383 cyl, 16 head, 63 sec, 512 bytes/sect x 40020624 sectors
wd0: 32-bit data port
wd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 4 (Ultra/66)
pciide0: primary channel interrupting at irq 14
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2 (Ultra/33) (using DMA data
transfers)
pciide0: secondary channel wired to compatibility mode
pciide0: disabling secondary channel (no drives)
uhci0 at pci0 dev 4 function 2: Intel 82371AB USB Host Controller (PIIX4) (rev.
0x01)
pci_intr_map: no MP mapping found
uhci0: interrupting at irq 5
usb0 at uhci0: USB revision 1.0
uhub0 at usb0
uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
Intel 82371AB Power Management Controller (PIIX4) (miscellaneous bridge, revisio
n 0x02) at pci0 dev 4 function 3 not configured
ahc1 at pci0 dev 6 function 0
pci_intr_map: no MP mapping found
ahc1: interrupting at irq 5
ahc1: aic7890/91 Wide Channel A, SCSI Id=7, 16/255 SCBs
scsibus0 at ahc1: 16 targets, 8 luns per target
ex0 at pci0 dev 9 function 0: 3Com 3c905-TX 10/100 Ethernet (rev. 0x0)
pci_intr_map: no MP mapping found
ex0: interrupting at irq 5
ex0: MAC address 00:10:4b:24:4a:b4
nsphy0 at ex0 phy 24: DP83840 10/100 media interface, rev. 1
nsphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp0 at pci0 dev 10 function 0: i82559S Ethernet, rev 9
pci_intr_map: no MP mapping found
fxp0: interrupting at irq 12
fxp0: detected 64 word EEPROM
fxp0: Ethernet address 00:d0:b7:8f:1b:08
inphy0 at fxp0 phy 1: i82555 10/100 media interface, rev. 4
inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
pciide1 at pci0 dev 11 function 0: Promise Ultra100/ATA Bus Master IDE Accelerat
or (rev. 0x02)
pciide1: bus-master DMA support present
pciide1: primary channel configured to native-PCI mode
pci_intr_map: no MP mapping found
pciide1: using irq 10 for native-PCI interrupt
wd1 at pciide1 channel 0 drive 0: <Maxtor 96147U8>
wd1: drive supports 16-sector PIO transfers, LBA addressing
wd1: 58623 MB, 16383 cyl, 16 head, 63 sec, 512 bytes/sect x 120060864 sectors
wd1: 32-bit data port
wd1: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 5 (Ultra/100)
wd1(pciide1:0:0): using PIO mode 4, Ultra-DMA mode 5 (Ultra/100) (using DMA data
 transfers)
pciide1: secondary channel configured to native-PCI mode
wd2 at pciide1 channel 1 drive 0: <WDC WD800AB-22BTA0>
wd2: drive supports 16-sector PIO transfers, LBA addressing
wd2: 76319 MB, 16383 cyl, 16 head, 63 sec, 512 bytes/sect x 156301488 sectors
wd2: 32-bit data port
wd2: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 5 (Ultra/100)
wd2(pciide1:1:0): using PIO mode 4, Ultra-DMA mode 5 (Ultra/100) (using DMA data
 transfers)
pciide2 at pci0 dev 12 function 0: Promise Ultra100/ATA Bus Master IDE Accelerat
or (rev. 0x02)
pciide2: bus-master DMA support present
pciide2: primary channel configured to native-PCI mode
pci_intr_map: no MP mapping found
pciide2: using irq 11 for native-PCI interrupt
wd3 at pciide2 channel 0 drive 0: <Maxtor 96147H8>
wd3: drive supports 16-sector PIO transfers, LBA addressing
wd3: 58623 MB, 16383 cyl, 16 head, 63 sec, 512 bytes/sect x 120060864 sectors
wd3: 32-bit data port
wd3: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 5 (Ultra/100)
wd3(pciide2:0:0): using PIO mode 4, Ultra-DMA mode 5 (Ultra/100) (using DMA data
 transfers)
pciide2: secondary channel configured to native-PCI mode
wd4 at pciide2 channel 1 drive 0: <Maxtor 96147U8>
wd4: drive supports 16-sector PIO transfers, LBA addressing
wd4: 58623 MB, 16383 cyl, 16 head, 63 sec, 512 bytes/sect x 120060864 sectors
wd4: 32-bit data port
wd4: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 5 (Ultra/100)
wd4(pciide2:1:0): using PIO mode 4, Ultra-DMA mode 5 (Ultra/100) (using DMA data
 transfers)
isa0 at pcib0
com0 at isa0 port 0x3f8-0x3ff irq 4: ns16550a, working fifo
com0: console
com1 at isa0 port 0x2f8-0x2ff irq 3: ns16550a, working fifo
pckbc0 at isa0 port 0x60-0x64
pckbdprobe: reset error 5
pmsprobe: reset error 5
pmsiprobe: reset error 5
lpt0 at isa0 port 0x378-0x37b irq 7
lptprobe: mask ff data 55 failed
lptprobe: mask ff data 55 failed
seaprobe: board type unknown at address 0xc8000
pcppi0 at isa0 port 0x61
midi0 at pcppi0: PC speaker
sysbeep0 at pcppi0
isapnp0 at isa0 port 0x279: ISA Plug 'n Play device support
npx0 at isa0 port 0xf0-0xff: using exception 16
fdc0 at isa0 port 0x3f0-0x3f7 irq 6 drq 2
fd0 at fdc0 drive 0: 1.44MB, 80 cyl, 2 head, 18 sec
isapnp0: no ISA Plug 'n Play devices found
biomask 0 netmask 0 ttymask 0
cpu0: prelint0 8700<vector=0,delmode=7,level,dest=0> 0<target=0>
cpu0: prelint1 400<vector=0,delmode=4,dest=0> 0<target=0>
cpu0: timer0 300d0<vector=d0,delmode=0,masked,dest=0> 0<target=0>
cpu0: pcint0 10000<vector=0,delmode=0,masked,dest=0> 0<target=0>
cpu0: lint0 10700<vector=0,delmode=7,masked,dest=0> 0<target=0>
cpu0: lint1 400<vector=0,delmode=4,dest=0> 0<target=0>
cpu0: err0 10000<vector=0,delmode=0,masked,dest=0> 0<target=0>
ioapic0: enabling
ioapic0: int0 10700<vector=0,delmode=7,masked,dest=0> 0<target=0>
ioapic0: int1 10100<vector=0,delmode=1,masked,dest=0> 0<target=0>
ioapic0: int2 10100<vector=0,delmode=1,masked,dest=0> 0<target=0>
ioapic0: int3 1e1<vector=e1,delmode=1,dest=0> 1000000<target=1>
ioapic0: int4 1e2<vector=e2,delmode=1,dest=0> 1000000<target=1>
ioapic0: int5 10000<vector=0,delmode=0,masked,dest=0> 0<target=0>
ioapic0: int6 170<vector=70,delmode=1,dest=0> 1000000<target=1>
ioapic0: int7 1a0<vector=a0,delmode=1,dest=0> 1000000<target=1>
ioapic0: int8 10100<vector=0,delmode=1,masked,dest=0> 0<target=0>
ioapic0: int9 10100<vector=0,delmode=1,masked,dest=0> 0<target=0>
ioapic0: int10 10000<vector=0,delmode=0,masked,dest=0> 0<target=0>
ioapic0: int11 10000<vector=0,delmode=0,masked,dest=0> 0<target=0>
ioapic0: int12 10000<vector=0,delmode=0,masked,dest=0> 0<target=0>
ioapic0: int13 10000<vector=0,delmode=0,masked,dest=0> 0<target=0>
ioapic0: int14 171<vector=71,delmode=1,dest=0> 1000000<target=1>
ioapic0: int15 10100<vector=0,delmode=1,masked,dest=0> 0<target=0>
ioapic0: int16 a172<vector=72,delmode=1,actlo,level,dest=0> 1000000<target=1>
ioapic0: int17 a173<vector=73,delmode=1,actlo,level,dest=0> 1000000<target=1>
ioapic0: int18 a181<vector=81,delmode=1,actlo,level,dest=0> 1000000<target=1>
ioapic0: WARNING: sharing interrupt between different IPLs (currently broken)
ioapic0: pin 19, ipls 70..80
ioapic0: int19 a182<vector=82,delmode=1,actlo,level,dest=0> 1000000<target=1>
ioapic0: int20 10000<vector=0,delmode=0,masked,dest=0> 0<target=0>
ioapic0: int21 10000<vector=0,delmode=0,masked,dest=0> 0<target=0>
ioapic0: int22 10000<vector=0,delmode=0,masked,dest=0> 0<target=0>
ioapic0: int23 10000<vector=0,delmode=0,masked,dest=0> 0<target=0>
scsibus0: waiting 2 seconds for devices to settle...
raidattach: Asked for 4 units
Kernelized RAIDframe activated
Searching for raid components...

pool_get(PR_WAITOK) with held simple_lock 0xc0645238 CPU 1 /usr/src/syssmp/arch/
i386/compile/sou.mpdebug/../../../../kern/vfs_subr.c:460
pool_get(c0698860,2,15f,3,0) at pool_get+0x7b
uvm_map(c0666e40,c0731c0c,1000,c0666d40,32968000,0,0,1727) at uvm_map+0x8a6
uvm_km_kmemalloc(c0666e40,c0666d40,1000,1,0) at uvm_km_kmemalloc+0x6d
uvm_km_alloc_poolpage1(c0666e40,c0666d40,0,c0207a9e,0) at uvm_km_alloc_poolpage1
+0x3d
pool_page_alloc_nointr(1000,0,19,c0215b7e,c0681770) at pool_page_alloc_nointr+0x
23
pool_get(c0681700,0,1cc,c03880b8,c04b4d00) at pool_get+0x1d0
getnewvnode(0,0,c185b300,c0731d1c) at getnewvnode+0xc8
getdevvp(3,c0731d98,3,1,c0731d68) at getdevvp+0x2f
bdevvp(3,c0731d98,3d9,d0) at bdevvp+0x16
rf_find_raid_components(c04dcfc0,c04dcf86,3,c01d89eb,c063ff30,c067e020,c067e228,
c0257882,c063ff10,c067e020,c0731f70,c01f3713,640,640,c0731fa0,c01fda4a,4,c067e02
0,c0731fa0,c01fda30,c072f010,72f000,736000,c00007ff,c089,0,0,c01002fa,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0) at rf_find_raid_compone
nts+0xda
raidattach(4,c067e020,c0731fa0,c01fda30) at raidattach+0x2da
main(0,0,0,0,0) at main+0x41a
Stopped in pid 0 (swapper) at   cpu_Debugger+0x4:       leave
db{1}> c
Component on: wd1e: 120060801
   Row: 0 Column: 0 Num Rows: 1 Num Columns: 4
   Version: 2 Serial Number: 2948234 Mod Counter: 1677
   Clean: Yes Status: 0
   sectPerSU: 128 SUsPerPU: 1 SUsPerRU: 1
   RAID Level: 5  blocksize: 512 numBlocks: 120060672
   Autoconfig: No
   Contains root partition: No
   Last configured as: raid0
Component on: wd2e: 156301425
   Row: 0 Column: 3 Num Rows: 1 Num Columns: 4
   Version: 2 Serial Number: 2948234 Mod Counter: 1677
   Clean: Yes Status: 0
   sectPerSU: 128 SUsPerPU: 1 SUsPerRU: 1
   RAID Level: 5  blocksize: 512 numBlocks: 120060672
   Autoconfig: No
   Contains root partition: No
   Last configured as: raid0
Component on: wd3e: 120060801
   Row: 0 Column: 2 Num Rows: 1 Num Columns: 4
   Version: 2 Serial Number: 2948234 Mod Counter: 1677
   Clean: Yes Status: 0
   sectPerSU: 128 SUsPerPU: 1 SUsPerRU: 1
   RAID Level: 5  blocksize: 512 numBlocks: 120060672
   Autoconfig: No
   Contains root partition: No
   Last configured as: raid0
Component on: wd4e: 120060801
   Row: 0 Column: 1 Num Rows: 1 Num Columns: 4
   Version: 2 Serial Number: 2948234 Mod Counter: 1677
   Clean: Yes Status: 0
   sectPerSU: 128 SUsPerPU: 1 SUsPerRU: 1
   RAID Level: 5  blocksize: 512 numBlocks: 120060672
   Autoconfig: No
   Contains root partition: No
   Last configured as: raid0
Found: wd1e at 0,0
Found: wd4e at 0,1
Found: wd3e at 0,2
Found: wd2e at 0,3
findroot: can't open dev wd1a (6)
findroot: can't open dev wd2a (6)
findroot: can't open dev wd3a (6)
findroot: can't open dev wd4a (6)
boot device: wd0
root on wd0a dumps on wd0b
mountroot: trying coda...
mountroot: trying msdos...
mountroot: trying cd9660...
mountroot: trying ntfs...
mountroot: trying nfs...
mountroot: trying lfs...
mountroot: trying ext2fs...
mountroot: trying ffs...
root file system type: ffs
cpu1: prelint0 10000<vector=0,delmode=0,masked,dest=0> 0<target=0>
cpu1: prelint1 10000<vector=0,delmode=0,masked,dest=0> 0<target=0>
cpu1: timer0 200d0<vector=d0,delmode=0,dest=0> 0<target=0>
cpu1: pcint0 10000<vector=0,delmode=0,masked,dest=0> 0<target=0>
cpu1: lint0 10700<vector=0,delmode=7,masked,dest=0> 0<target=0>
cpu1: lint1 400<vector=0,delmode=4,dest=0> 0<target=0>
cpu1: err0 10000<vector=0,delmode=0,masked,dest=0> 0<target=0>
init: copying out path `/sbin/init' 11
RAIDFRAME: protectedSectors is 64

malloc with held simple_lock 0xc065ebc8 CPU 1 /usr/src/syssmp/arch/i386/compile/
sou.mpdebug/../../../../dev/raidframe/rf_driver.c:356
malloc(30,5d,0,c0217fd2,c19f5000) at malloc+0x1f
rf_ConfigureMapModule(c065ebe4,40,c04dace0,167,c19f5000) at rf_ConfigureMapModul
e+0x16
rf_Configure(c19e0000,c19f5000,0,c0215f61,f299edc4) at rf_Configure+0x1b9
raidioctl(1203,80047201,f299eec0,3,f298e1cc) at raidioctl+0x6f6
spec_ioctl(f299edc4,1,f299edd0,c0207bf4,f2989090) at spec_ioctl+0x9e
VOP_IOCTL(f299c0f4,80047201,f299eec0,3,c186af00) at VOP_IOCTL+0x49
vn_ioctl(f2989090,80047201,f299eec0,f298e1cc,f299ef78) at vn_ioctl+0xb1
sys_ioctl(f298e1cc,f299ef80,f299ef78,c051a340,0) at sys_ioctl+0x34d
syscall_plain(1f,1f,1f,1f,bfbfccbc) at syscall_plain+0x10e
Stopped in pid 12 (raidctl) at  cpu_Debugger+0x4:       leave
db{1}> c

malloc with held simple_lock 0xc065ebc8 CPU 1 /usr/src/syssmp/arch/i386/compile/
sou.mpdebug/../../../../dev/raidframe/rf_driver.c:356
malloc(30,5d,0,c0217fd2,c19f5000) at malloc+0x1f
rf_ConfigureMapModule(c065ebe4,40,c04dace0,167,c19f5000) at rf_ConfigureMapModul
e+0xe2
rf_Configure(c19e0000,c19f5000,0,c0215f61,f299edc4) at rf_Configure+0x1b9
raidioctl(1203,80047201,f299eec0,3,f298e1cc) at raidioctl+0x6f6
spec_ioctl(f299edc4,1,f299edd0,c0207bf4,f2989090) at spec_ioctl+0x9e
VOP_IOCTL(f299c0f4,80047201,f299eec0,3,c186af00) at VOP_IOCTL+0x49
vn_ioctl(f2989090,80047201,f299eec0,f298e1cc,f299ef78) at vn_ioctl+0xb1
sys_ioctl(f298e1cc,f299ef80,f299ef78,c051a340,0) at sys_ioctl+0x34d
syscall_plain(1f,1f,1f,1f,bfbfccbc) at syscall_plain+0x10e
Stopped in pid 12 (raidctl) at  cpu_Debugger+0x4:       leave
db{1}> c

malloc with held simple_lock 0xc065ebc8 CPU 1 /usr/src/syssmp/arch/i386/compile/
sou.mpdebug/../../../../dev/raidframe/rf_driver.c:356
malloc(30,5d,0,c0217fd2,c19f5000) at malloc+0x1f
rf_ConfigureMapModule(c065ebe4,40,c04dace0,167,c19f5000) at rf_ConfigureMapModul
e+0x236
rf_Configure(c19e0000,c19f5000,0,c0215f61,f299edc4) at rf_Configure+0x1b9
raidioctl(1203,80047201,f299eec0,3,f298e1cc) at raidioctl+0x6f6
spec_ioctl(f299edc4,1,f299edd0,c0207bf4,f2989090) at spec_ioctl+0x9e
VOP_IOCTL(f299c0f4,80047201,f299eec0,3,c186af00) at VOP_IOCTL+0x49
vn_ioctl(f2989090,80047201,f299eec0,f298e1cc,f299ef78) at vn_ioctl+0xb1
sys_ioctl(f298e1cc,f299ef80,f299ef78,c051a340,0) at sys_ioctl+0x34d
syscall_plain(1f,1f,1f,1f,bfbfccbc) at syscall_plain+0x10e
Stopped in pid 12 (raidctl) at  cpu_Debugger+0x4:       leave
db{1}> c

malloc with held simple_lock 0xc065ebc8 CPU 1 /usr/src/syssmp/arch/i386/compile/
sou.mpdebug/../../../../dev/raidframe/rf_driver.c:356
malloc(10,5d,0,c0217fd2,c19f5000) at malloc+0x1f
rf_ConfigureMapModule(c065ebe4,40,c04dace0,167,c19f5000) at rf_ConfigureMapModul
e+0x480
rf_Configure(c19e0000,c19f5000,0,c0215f61,f299edc4) at rf_Configure+0x1b9
raidioctl(1203,80047201,f299eec0,3,f298e1cc) at raidioctl+0x6f6
spec_ioctl(f299edc4,1,f299edd0,c0207bf4,f2989090) at spec_ioctl+0x9e
VOP_IOCTL(f299c0f4,80047201,f299eec0,3,c186af00) at VOP_IOCTL+0x49
vn_ioctl(f2989090,80047201,f299eec0,f298e1cc,f299ef78) at vn_ioctl+0xb1
sys_ioctl(f298e1cc,f299ef80,f299ef78,c051a340,0) at sys_ioctl+0x34d
syscall_plain(1f,1f,1f,1f,bfbfccbc) at syscall_plain+0x10e
Stopped in pid 12 (raidctl) at  cpu_Debugger+0x4:       leave
db{1}> reboot
syncing disks... done
unmounting / (root_device)...

pool_get(PR_WAITOK) with held simple_lock 0xc065ebc8 CPU 1 /usr/src/syssmp/arch/
i386/compile/sou.mpdebug/../../../../dev/raidframe/rf_driver.c:356
pool_get(c0679740,2,f299e368,c0245216,c196e4c4) at pool_get+0x7b
wdc_get_xfer(0,70,6,c04edd00,c1971800) at wdc_get_xfer+0x35
wdc_exec_command(c196e508,f299e3c8,0,0) at wdc_exec_command+0x4c
wd_flushcache(c1971800,8,f2977514,c0207a9e) at wd_flushcache+0x54
wdclose(0,1,6000,f298e1cc,f2977514) at wdclose+0xde
spec_close(f299e484,f2977514,c19e9c00,c04edcc0,0) at spec_close+0x192
VOP_CLOSE(f2977514,1,ffffffff,f298e1cc,f2977514) at VOP_CLOSE+0x3b
ffs_unmount(c19e9e00,80000,f298e1cc,c04ec4a0,c19e9e00) at ffs_unmount+0xec
dounmount(c19e9e00,80000,f298e1cc,c0217fd2) at dounmount+0x116
vfs_unmountall(f298e1cc,0,0,246,0) at vfs_unmountall+0xa6
vfs_shutdown(0,10,f299e5c8,c01a86e8,74) at vfs_shutdown+0x255
cpu_reboot(0,0,f299e5d8,c01a6475,c04d5040) at cpu_reboot+0x3b
db_reboot_cmd(1,0,f299e62c,f299e610,0) at db_reboot_cmd+0x51
db_command(c0640f70,c04d5040,f299e6d8,c01a62ce,c04d52e0,1,f299e6d8,c01a6295) at
db_command+0x214
db_command_loop(c0374c70,c,f298e347,c03870c2) at db_command_loop+0x94
db_trap(1,0,f299e7a8,c0374ba7,1,c01a53eb,f299e7a8,c0374af5) at db_trap+0x143
kdb_trap(1,0,f299e800,0) at kdb_trap+0x137
trap() at trap+0x1bf
--- trap (number 1) ---
cpu_Debugger(0,0,0,c01ef1d8,0) at cpu_Debugger+0x4
simple_lock_only_held(0,c04e6c58,c1973180,2) at simple_lock_only_held+0x106
malloc(10,5d,0,c0217fd2,c19f5000) at malloc+0x1f
rf_ConfigureMapModule(c065ebe4,40,c04dace0,167,c19f5000) at rf_ConfigureMapModul
e+0x480
rf_Configure(c19e0000,c19f5000,0,c0215f61,f299edc4) at rf_Configure+0x1b9
raidioctl(1203,80047201,f299eec0,3,f298e1cc) at raidioctl+0x6f6
spec_ioctl(f299edc4,1,f299edd0,c0207bf4,f2989090) at spec_ioctl+0x9e
VOP_IOCTL(f299c0f4,80047201,f299eec0,3,c186af00) at VOP_IOCTL+0x49
vn_ioctl(f2989090,80047201,f299eec0,f298e1cc,f299ef78) at vn_ioctl+0xb1
sys_ioctl(f298e1cc,f299ef80,f299ef78,c051a340,0) at sys_ioctl+0x34d
syscall_plain(1f,1f,1f,1f,bfbfccbc) at syscall_plain+0x10e
Stopped in pid 12 (raidctl) at  cpu_Debugger+0x4:       leave
db{1}> reboot
rebooting...

-- Kazushi