Subject: possible causes for read(2) to return EFAULT?
To: NetBSD-current Discussion List <current-users@NetBSD.ORG>
From: Greg A. Woods <woods@weird.com>
List: current-users
Date: 04/29/2003 16:28:56
I've just been doing some benchmarks and tests on a new server with the
NetBSD snapshot from releng.netbsd.org/NetBSD-daily/current/200304250000/i386

You'll see why I'm testing with -current when you see which devices are
in the system from the dmesg output below.

Running bonnie (after patching some off_t bugs), and not on the first
run but rather after a half-dozen successful runs, I see it return an
EFAULT from read():

	Rewriting...
	Bonnie: drastic I/O error (rewrite read): Bad address

There are no messages from the kernel.  This machine supposedly has ECC
DRAM (though I've not yet actually opened it up to look :-).

The filesystem I was writing to, which was on the ATA drive, checks clean.
The filesystem was created by sysinst, and still has softdep disabled.

The machine ran FreeBSD-4.8 for over ten days doing very extensive and
intensive testing using bonnie, bonnie++, and postmark on various VINUM
volumes on the SCSI drives, as well as with comparison runs on the ATA
drive; all with narry ever a peep of a problem (except when VINUM
occasionally crashed because I was sometimes feeding it what turned out
to be impossible, though highly desirable, configurations).

The system has an Award BIOS which is configured to "log ECC errors"
(IIRC), so I'm going shut it down now and see if I can find where this
log is now and if there's anything unexpected in it.....

# dmesg
NetBSD 1.6R (GENERIC) #0: Fri Apr 25 23:30:16 UTC 2003
        autobuild@tgm.daemon.org:/autobuild/HEAD/i386/OBJ/autobuild/HEAD/src/sys/arch/i386/compile/GENERIC
total memory = 1023 MB
avail memory = 943 MB
using 6144 buffers containing 52508 KB of memory
BIOS32 rev. 0 found at 0xf14e0
mainbus0 (root)
cpu0 at mainbus0: (uniprocessor)
cpu0: Intel Pentium 4 (686-class), 2790.96 MHz, id 0xf27
cpu0: features bfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR>
cpu0: features bfebfbff<PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX>
cpu0: features bfebfbff<FXSR,SSE,SSE2,SS,HTT,TM,SBF>
cpu0: I-cache 12K uOp cache 8-way, D-cache 8 KB 64b/line 4-way
cpu0: L2 cache 512 KB 64b/line 8-way
cpu0: ITLB 4K/4M: 64 entries
cpu0: DTLB 4K/4M: 64 entries
cpu0: 16 page colors
pci0 at mainbus0 bus 0: configuration mode 1
pci0: i/o space, memory space enabled, rd/line, rd/mult, wr/inv ok
pchb0 at pci0 dev 0 function 0
pchb0: ServerWorks product 0x0014 (rev. 0x31)
pchb1 at pci0 dev 0 function 1
pchb1: ServerWorks product 0x0014 (rev. 0x00)
pci1 at pchb1 bus 3
pci1: no spaces enabled!
pchb2 at pci0 dev 0 function 2
pchb2: ServerWorks product 0x0014 (rev. 0x00)
pci2 at pchb2 bus 1
pci2: no spaces enabled!
wm0 at pci0 dev 2 function 0: Intel i82540EM 1000BASE-T Ethernet, rev. 2
wm0: interrupting at irq 12
wm0: Ethernet address 00:0c:6e:02:30:f2
makphy0 at wm0 phy 1: Marvell 88E1011 Gigabit PHY, rev. 3
makphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
vga1 at pci0 dev 3 function 0: ATI Technologies Rage XL (rev. 0x27)
wsdisplay0 at vga1 kbdmux 1: console (80x25, vt100 emulation)
wsmux1: connecting to wsdisplay0
pcib0 at pci0 dev 15 function 0
pcib0: ServerWorks CSB5 SouthBridge (rev. 0x93)
pciide0 at pci0 dev 15 function 1: ServerWorks CSB5 IDE Controller (rev. 0x93)
pciide0: bus-master DMA support present
pciide0: primary channel configured to compatibility mode
wd0 at pciide0 channel 0 drive 0: <WDC WD600JB-00CRA1>
wd0: drive supports 16-sector PIO transfers, LBA addressing
wd0: 57241 MB, 16383 cyl, 16 head, 63 sec, 512 bytes/sect x 117231408 sectors
wd0: 32-bit data port
wd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 5 (Ultra/100)
pciide0: primary channel interrupting at irq 14
wd0(pciide0:0:0): using PIO mode 4, DMA mode 2, Ultra-DMA mode 2 (Ultra/33) (using DMA data transfers)
pciide0: secondary channel configured to compatibility mode
atapibus0 at pciide0 channel 1: 2 targets
cd0 at atapibus0 drive 0: <ASUS    CD-S520/A, , 2.0L> cdrom removable
cd0: 32-bit data port
cd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 2 (Ultra/33)
pciide0: secondary channel interrupting at irq 15
cd0(pciide0:1:0): using PIO mode 4, DMA mode 2, Ultra-DMA mode 2 (Ultra/33) (using DMA data transfers)
ohci0 at pci0 dev 15 function 2: ServerWorks OSB4/CSB5 USB (rev. 0x05)
ohci0: interrupting at irq 11
ohci0: OHCI version 1.0, legacy support
usb0 at ohci0: USB revision 1.0
uhub0 at usb0
uhub0: ServerWorks OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 4 ports with 4 removable, self powered
pchb3 at pci0 dev 15 function 3
pchb3: ServerWorks product 0x0225 (rev. 0x00)
pchb4 at pci0 dev 16 function 0
pchb4: ServerWorks CIOB-X2 (rev. 0x03)
pci3 at pchb4 bus 3
pci3: i/o space, memory space enabled
wm1 at pci3 dev 2 function 0: Intel i82544GC (LOM) 1000BASE-T Ethernet, rev. 2
wm1: interrupting at irq 5
wm1: Ethernet address 00:0c:6e:02:30:f1
makphy1 at wm1 phy 1: Marvell 88E1000 Gigabit PHY, rev. 0
makphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
pchb5 at pci0 dev 16 function 2
pchb5: ServerWorks CIOB-X2 (rev. 0x03)
pci4 at pchb5 bus 4
pci4: i/o space, memory space enabled
pchb6 at pci0 dev 17 function 0
pchb6: ServerWorks CIOB-X2 (rev. 0x03)
pci5 at pchb6 bus 1
pci5: i/o space, memory space enabled
pchb7 at pci0 dev 17 function 2
pchb7: ServerWorks CIOB-X2 (rev. 0x03)
pci6 at pchb7 bus 2
pci6: i/o space, memory space enabled
mpt0 at pci6 dev 4 function 0: LSI Logic 53c1030 Ultra320 SCSI
mpt0: interrupting at irq 9
scsibus0 at mpt0: 16 targets, 8 luns per target
mpt1 at pci6 dev 4 function 1: LSI Logic 53c1030 Ultra320 SCSI
mpt1: interrupting at irq 9
scsibus1 at mpt1: 16 targets, 8 luns per target
isa0 at pcib0
com0 at isa0 port 0x3f8-0x3ff irq 4: ns16550a, working fifo
com1 at isa0 port 0x2f8-0x2ff irq 3: ns16550a, working fifo
pckbc0 at isa0 port 0x60-0x64
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
lpt0 at isa0 port 0x378-0x37b irq 7
pcppi0 at isa0 port 0x61
midi0 at pcppi0: PC speaker
sysbeep0 at pcppi0
isapnp0 at isa0 port 0x279: ISA Plug 'n Play device support
npx0 at isa0 port 0xf0-0xff: using exception 16
fdc0 at isa0 port 0x3f0-0x3f7 irq 6 drq 2
fd0 at fdc0 drive 0: 1.44MB, 80 cyl, 2 head, 18 sec
isapnp0: no ISA Plug 'n Play devices found
Kernelized RAIDframe activated
scsibus0: waiting 2 seconds for devices to settle...
scsibus1: waiting 2 seconds for devices to settle...
sd0 at scsibus0 target 0 lun 0: <FUJITSU, MAP3735NC, 0104> disk fixed
sd0: 70103 MB, 47996 cyl, 4 head, 747 sec, 512 bytes/sect x 143571316 sectors
sd0: sync (6.25ns offset 127), 16-bit (320.000MB/s) transfers, tagged queueing
sd1 at scsibus0 target 1 lun 0: <FUJITSU, MAP3735NC, 0104> disk fixed
sd1: 70103 MB, 47996 cyl, 4 head, 747 sec, 512 bytes/sect x 143571316 sectors
sd1: sync (6.25ns offset 127), 16-bit (320.000MB/s) transfers, tagged queueing
sd2 at scsibus0 target 2 lun 0: <FUJITSU, MAP3735NC, 0104> disk fixed
sd2: 70103 MB, 47996 cyl, 4 head, 747 sec, 512 bytes/sect x 143571316 sectors
sd2: sync (6.25ns offset 127), 16-bit (320.000MB/s) transfers, tagged queueing
sd3 at scsibus0 target 3 lun 0: <FUJITSU, MAP3735NC, 0104> disk fixed
sd3: 70103 MB, 47996 cyl, 4 head, 747 sec, 512 bytes/sect x 143571316 sectors
sd3: sync (6.25ns offset 127), 16-bit (320.000MB/s) transfers, tagged queueing
ses0 at scsibus0 target 11 lun 0: <SDR, GEM318, 0> processor fixed
ses0: SAF-TE Compliant Device
ses0: async, 8-bit transfers
boot device: wd0
root on wd0a dumps on wd0b
root file system type: ffs
wsdisplay0: screen 1 added (80x25, vt100 emulation)
wsdisplay0: screen 2 added (80x25, vt100 emulation)
wsdisplay0: screen 3 added (80x25, vt100 emulation)
wsdisplay0: screen 4 added (80x25, vt100 emulation)

-- 
								Greg A. Woods

+1 416 218-0098;            <g.a.woods@ieee.org>;           <woods@robohack.ca>
Planix, Inc. <woods@planix.com>; VE3TCP; Secrets of the Weird <woods@weird.com>