port-evbarm/49463: dma driver on rpi is toxic

[christos%zoulas.com@localhost]

>Number:         49463
>Category:       port-evbarm
>Synopsis:       using the dma driver on rpc corrupts the sd data
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    port-evbarm-maintainer
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Dec 10 17:55:00 +0000 2014
>Originator:     Christos Zoulas
>Release:        NetBSD 7.99.1
>Organization:
	Bit Corruption, Inc.
>Environment:
NetBSD rpi+ 7.99.1 NetBSD 7.99.1 (RPI) #0: Tue Dec  9 22:06:37 EST 2014  ljt@t61.localnet:/s/NetBSD/obj/sys/arch/evbarm/compile/RPI evbarm
System: NetBSD rpi+ 7.99.1 NetBSD 7.99.1 (RPI) #0: Tue Dec  9 22:06:37 EST 2014  ljt@t61.localnet:/s/NetBSD/obj/sys/arch/evbarm/compile/RPI evbarm
Architecture: arm
Machine: earm6hf
>Description:

Sources are 7.99.1-2014.11.17.11.50.00

The microSD card is a 32GB SanDisk Ultra Plus card.  I have it
partitioned as:

------------------------------------------------------------------------

# /dev/rld0c:
type: SCSI
disk: Card  Reader   =20
label: fictitious
flags: removable
bytes/sector: 512
sectors/track: 32
tracks/cylinder: 64
sectors/cylinder: 2048
cylinders: 30436
total sectors: 62333952
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0           # microseconds
track-to-track seek: 0  # microseconds
drivedata: 0=20

9 partitions:
#        size    offset     fstype [fsize bsize cpg/sgs]
 a:   1536000    122880     4.2BSD      0     0     0  # (Cyl.     60 -    =
809)
 b:    463872  61870080       swap                     # (Cyl.  30210 -  30=
436*)
 c:  62211072    122880     unused      0     0        # (Cyl.     60 -  30=
436*)
 d:  62333952         0     unused      0     0        # (Cyl.      0 -  30=
436*)
 e:    114688      8192      MSDOS                     # (Cyl.      4 -    =
 59)
 f:   5120000   1658880     4.2BSD      0     0     0  # (Cyl.    810 -   3=
309)
 g:   5120000   6778880     4.2BSD      0     0     0  # (Cyl.   3310 -   5=
809)
 h:  19456000  11898880     4.2BSD      0     0     0  # (Cyl.   5810 -  15=
309)
 i:  30515200  31354880     4.2BSD      0     0     0  # (Cyl.  15310 -  30=
209)

------------------------------------------------------------------------

Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
    2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014
    The NetBSD Foundation, Inc.  All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
    The Regents of the University of California.  All rights reserved.

NetBSD 7.99.1 (RPI) #1: Mon Dec  8 11:49:48 EST 2014
	ljt@t61.localnet:/s/NetBSD/obj/sys/arch/evbarm/compile/RPI
total memory = 448 MB
avail memory = 436 MB
sysctl_createv: sysctl_create(machine_arch) returned 17
kern.module.path=/stand/evbarm/7.99.1/modules
timecounter: Timecounters tick every 10.000 msec
mainbus0 (root)
cpu0 at mainbus0 core 0: 700 MHz ARM1176JZ-S r0p7 (ARM11J V6ZK core)
cpu0: DC enabled IC enabled WB enabled LABT
cpu0: isar: [0]=0x140011 [1]=0x12002111 [2]=0x11231121 [3]=0x1102131, [4]=0x1141, [5]=0
cpu0: mmfr: [0]=0x1130003 [1]=0x10030302 [2]=0x1222100 [3]=0
cpu0: pfr: [0]=0x111 [1]=0x11
cpu0: 16KB/32B 4-way L1 VIPT Instruction cache
cpu0: 16KB/32B 4-way write-back-locking-C L1 VIPT Data cache
vfp0 at cpu0: VFP11, rounding, exceptions
vfp0: mvfr: [0]=0x11111111 [1]=0
obio0 at mainbus0
bcmicu0 at obio0
bcmmbox0 at obio0 intr 65: VC mailbox
vcmbox0 at bcmmbox0
bcmtmr0 at obio0 intr 3: VC System Timer
vchiq0 at obio0 intr 66: BCM2835 VCHIQ
bcmpm0 at obio0: Power management, Reset and Watchdog controller
bcmdmac0 at obio0: DMA0 DMA2 DMA4 DMA5 DMA8 DMA9 DMA10
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
bcmrng0 at obio0: RNG
plcom0 at obio0 intr 57
plcom0: txfifo disabled
plcom0: console
genfb0 at obio0rpi_fb_init: addr = 0x1c006000 size = 3686400
rpi_fb_init: depth = 32
rpi_fb_init: pitch = 5120
rpi_fb_init: width = 1280 height = 720
rpi_fb_init: vwidth = 1280 vheight = 720
rpi_fb_init: pixelorder = 0

genfb0: framebuffer at 0x1c006000, size 1280x720, depth 32, stride 5120
wsdisplay0 at genfb0 kbdmux 1
wsmux1: connecting to wsdisplay0
wsdisplay0: screen 0-3 added (default, vt100 emulation)
sdhc0 at obio0 intr 62: SDHC controller
sdhc0: interrupting on intr 62
dwctwo0 at obio0 intr 9: USB controller
bcmspi0 at obio0 intr 54: SPI
spi0 at bcmspi0: SPI bus
bsciic0 at obio0 intr 53: BSC0
iic0 at bsciic0: I2C bus
bsciic1 at obio0 intr 53: BSC1
iic1 at bsciic1: I2C bus
bcmgpio0 at obio0: GPIO [0...31]
bcmgpio1 at obio0: GPIO [32...53]
gpio1 at bcmgpio1: 22 pins
usb0 at dwctwo0: USB revision 2.0
timecounter: Timecounter "clockinterrupt" frequency 100 Hz quality 0
timecounter: Timecounter "bcmtmr0" frequency 1000000 Hz quality 100
sdhc0: SD Host Specification 3.0, rev.153
sdhc0: using DMA transfer
sdmmc0 at sdhc0 slot 0
WARNING: module error: vfs load failed for `usbverbose', error 45
uhub0 at usb0WARNING: module error: vfs load failed for `usbverbose', error 45
: vendor 0000 DWC2 root hub, class 9/0, rev 2.00/1.00, addr 1
uhub0: 1 port with 1 removable, self powered
ld0 at sdmmc0: <0x03:0x5344:SL32G:0x80:0x28b5d86a:0x0ea>
ld0: 30436 MB, 7729 cyl, 128 head, 63 sec, 512 bytes/sect x 62333952 sectors
ld0: 4-bit width, bus clock 50.000 MHz
WARNING: module error: vfs load failed for `usbverbose', error 45
WARNING: module error: vfs load failed for `usbverbose', error 45
uhub1 at uhub0 port 1WARNING: module error: vfs load failed for `usbverbose', error 45
WARNING: module error: vfs load failed for `usbverbose', error 45
: vendor 0424 product 9514, class 9/0, rev 2.00/2.00, addr 2
uhub1: multiple transaction translators
uhub1: 5 ports with 4 removable, self powered
WARNING: module error: vfs load failed for `usbverbose', error 45
WARNING: module error: vfs load failed for `usbverbose', error 45
usmsc0 at uhub1 port 1
WARNING: module error: vfs load failed for `usbverbose', error 45
WARNING: module error: vfs load failed for `usbverbose', error 45
usmsc0: vendor 0424 product ec00, rev 2.00/2.00, addr 3
usmsc0: Ethernet address b8:27:eb:1c:d1:36
ukphy0 at usmsc0 phy 1WARNING: module error: vfs load failed for `miiverbose', error 45
: OUI 0x00800f, model 0x000c, rev. 3
ukphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
urtwn0 at uhub1 port 3
urtwn0: Realtek 802.11n WLAN Adapter, rev 2.00/2.00, addr 4
urtwn0: MAC/BB RTL8188CUS, RF 6052 1T1R, address 80:1f:02:b6:ee:f3
urtwn0: 1 rx pipe, 2 tx pipes
urtwn0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps
urtwn0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
boot device: ld0
root on ld0a dumps on ld0b
mountroot: trying nfs...
mountroot: trying msdos...
mountroot: trying ext2fs...
mountroot: trying ffs...
root file system type: ffs
vchiq: local ver 6 (min 3), remote ver 6.
vcaudio0 at vchiq0: auds
WARNING: no TOD clock present
WARNING: using filesystem time
WARNING: CHECK AND RESET THE DATE!
audio0 at vcaudio0: half duplex, playback, capture, independent
warning: no /dev/console
init: copying out path `/sbin/init' 11
WARNING: module error: vfs load failed for `com', error 2
WARNING: module error: vfs load failed for `com', error 2
usmsc0: link state DOWN (was UNKNOWN)
wsdisplay0: screen 4 added (default, vt100 emulation)

>How-To-Repeat:
The problem is easily triggered with:

------------------------------------------------------------------------
rpi+# newfs  /dev/ld0g
/dev/rld0g: 2500.0MB (5120000 sectors) block size 16384, fragment size 2048
        using 14 cylinder groups of 178.58MB, 11429 blks, 22528 inodes.
super-block backups (for fsck_ffs -b #) at:
     32,  365760,  731488, 1097216, 1462944, 1828672, 2194400, 2560128,
2925856, 3291584, 3657312, 4023040, 4388768, 4754496,
rpi+# fsck -f /dev/ld0g
** /dev/rld0g
** File system is already clean
** Last Mounted on=20
** Phase 1 - Check Blocks and Sizes
PARTIALLY ALLOCATED INODE I=3D264
CLEAR? [yn] n

PARTIALLY ALLOCATED INODE I=3D265
CLEAR? [yn] n

PARTIALLY ALLOCATED INODE I=3D266
CLEAR? [yn] ^C
rpi+#=20
>Fix:
Please
For now disable bcmdmac0