Re: Making port-alpha usable again

[Tobias Nygren <tnn%NetBSD.org@localhost>]

On Thu, 13 Jul 2017 16:25:27 +0000 (UTC)
John Klos <john%ziaspace.com@localhost> wrote:

> Most critically, when the filesystem has been in use for a while, it gets 
> nuked. Sometimes the disklabel goes away, sometimes the filesystem suffers 
> irreparable corruption, and occasionally it's OK (or seems OK) after an 
> fsck.
> 
> This happens both with IDE (aceride0) and SCSI (esiop0). It happens when 
> the kernel is booted off a small, read-only ffs filesystem at the 
> beginning of the disk and the root filesystem is the rest of the disk. It 
> happens with and without WAPBL (log), although it seems that with log it 
> happens sooner, although that's just an impression.
> 
> If anyone has any thoughts about how best to diagnose the disk corruption 
> issues, please do let me know. I'd love to see NetBSD 8 running properly 
> on Alpha :)

This is a DMA transfer corruption problem caused by insufficient memory
fencing somewhere in the md pci and/or page table code. Today I tried
an assortement of PCI add-on cards (satalink, wm, rtk, fxp, tlp) in the
CS20 and they all exhibited varying degrees of problems, ranging from
random packet loss to corrupt packets.

With the rtk(4) card I was able to reproduce the issue 100%
consistently. Sent packets only contained the 16 first bytes of the
mbuf, the rest being junk.

I found that for the network cards, the problem goes away if I sprinkle
a memory barrier before bus_dmamap_load. (See attached patch.)
This is not the correct fix though as satalink is still severly broken.
I'll try to narrow down the actual issue.

-Tobias

Index: tsp_dma.c
===================================================================
RCS file: /cvsroot/src/sys/arch/alpha/pci/tsp_dma.c,v
retrieving revision 1.13
diff -p -u -r1.13 tsp_dma.c
--- tsp_dma.c	23 Sep 2013 16:41:57 -0000	1.13
+++ tsp_dma.c	29 Jul 2017 14:24:45 -0000
@@ -280,6 +280,7 @@ tsp_bus_dmamap_load_sgmap(bus_dma_tag_t 
 {
 	int error;
 
+	alpha_mb();
 	error = pci_sgmap_pte64_load(t, map, buf, buflen, p, flags,
 	    t->_sgmap);
 	if (error == 0)
@@ -296,6 +297,7 @@ tsp_bus_dmamap_load_mbuf_sgmap(bus_dma_t
 {
 	int error;
 
+	alpha_mb();
 	error = pci_sgmap_pte64_load_mbuf(t, map, m, flags, t->_sgmap);
 	if (error == 0)
 		tsp_tlb_invalidate(t->_cookie);
@@ -311,6 +313,7 @@ tsp_bus_dmamap_load_uio_sgmap(bus_dma_ta
 {
 	int error;
 
+	alpha_mb();
 	error = pci_sgmap_pte64_load_uio(t, map, uio, flags, t->_sgmap);
 	if (error == 0)
 		tsp_tlb_invalidate(t->_cookie);
@@ -326,6 +329,7 @@ tsp_bus_dmamap_load_raw_sgmap(bus_dma_ta
 {
 	int error;
 
+	alpha_mb();
 	error = pci_sgmap_pte64_load_raw(t, map, segs, nsegs, size, flags,
 	    t->_sgmap);
 	if (error == 0)