Subject: Re: port-cats/18026 cats fails to dump kernel core
To: None <port-cats@netbsd.org>
From: Chris Gilbert <chris@dokein.co.uk>
List: port-arm
Date: 08/25/2007 20:16:09
This is a multi-part message in MIME format.
--------------000905070405010307010703
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Chris Gilbert wrote:
> Hi,
> 
> Just thought I'd let people know I've managed to dig into this one a bit.
> 
> The issue looks to be related to the memory map in some way.  When doing
> the memory dump cats use DMA to do the transfers, having enabled dma
> debugging and some pciide debugging, it seems that the hang probably
> isn't a bug in the dma code, but something else.
> 
> Basically when the code attempts to dump the memory page at 0xB8000 the
> system hangs.   It doesn't matter how big the DMAs are, if it's the
> first page that we attempted to dump, it always hangs.
> 
> Searching google seems to show that 0xb8000 is a common address for vga
> on x86.  I'm wondering if the M1543 chip has decided that address is
> special (as it's a x86 southbridge) and gets itself confused, and hangs
> the system.

Having done more digging into this issue, the range of effected
addresses is 0xb8000-0xc0000.  Searching this is a common vga range.

My best guess as to the cause of the hangs is that the vga card has
decided to respond to those addresses on the pci bus, and so the IDE DMA
 gets confused.

Handily the footbridge has the ability to shift the SDRAM window to
elsewhere in the PCI space.

attached is a patch that does just that, the SDRAM window is moved from
being at 0x00000000 to 0x20000000.  (0x10000000 is taken by the CSR
registers)  I'm not sure what a sensible choice would actually be for
the window base.

Having made the change my cats is happily running, and doing a memory dump.

I would note that due to a different PR (port-arm/24077) we're not
creating a kcore header, and so savecore doesn't think there's a core
dump there.

I'll look into fixing the kcore issue next.

Please could people try the attached diff and see if it works for them.

Thanks,
Chris

--------------000905070405010307010703
Content-Type: text/plain;
 name="fb_sdram_move.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="fb_sdram_move.diff"

Index: dc21285reg.h
===================================================================
RCS file: /cvsroot/src/sys/arch/arm/footbridge/dc21285reg.h,v
retrieving revision 1.4
diff -u -p -r1.4 dc21285reg.h
--- dc21285reg.h	17 Jan 2003 22:29:43 -0000	1.4
+++ dc21285reg.h	25 Aug 2007 19:01:19 -0000
@@ -46,6 +46,9 @@
 #define DC21285_DEVICE_ID	0x1065
 #define REVISION		0x08
 #define CLASS			0x0A
+#define CSR_BASE_MEMORY_ADDR	0x10
+#define CSR_BASE_IO_ADDR	0x14
+#define SDRAM_MEMORY_ADDR	0x18
 
 /* Other PCI control / status registers */
 
Index: footbridge.c
===================================================================
RCS file: /cvsroot/src/sys/arch/arm/footbridge/footbridge.c,v
retrieving revision 1.17.24.1
diff -u -p -r1.17.24.1 footbridge.c
--- footbridge.c	11 Aug 2007 21:14:49 -0000	1.17.24.1
+++ footbridge.c	25 Aug 2007 19:01:19 -0000
@@ -43,11 +43,13 @@ __KERNEL_RCSID(0, "$NetBSD: footbridge.c
 #include <sys/conf.h>
 #include <sys/malloc.h>
 #include <sys/device.h>
+#include <uvm/uvm_extern.h>
 
 #include <dev/pci/pcivar.h>
 #define _ARM32_BUS_DMA_PRIVATE
 #include <machine/bus.h>
 #include <machine/intr.h>
+#include <machine/bootconfig.h>
 
 #include <arm/cpuconf.h>
 #include <arm/cpufunc.h>
@@ -86,6 +88,7 @@ struct bus_space footbridge_pci_io_bs_ta
 struct bus_space footbridge_pci_mem_bs_tag;
 extern struct arm32_pci_chipset footbridge_pci_chipset;
 extern struct arm32_bus_dma_tag footbridge_pci_bus_dma_tag;
+extern struct arm32_dma_range footbridge_dma_ranges[1];
 
 /* Used in footbridge_clock.c */
 struct footbridge_softc *clock_sc;
@@ -208,6 +211,46 @@ footbridge_attach(parent, self, aux)
 
 	/* calibrate the delay loop */
 	calibrate_delay();
+
+	/* it seems that the default of the memory being visible from 0 upwards
+	 * on the PCI bus doesn't work.  As the address that break are
+	 * traditional vga addresses, it suggests that the vga card is
+	 * taking them for itself, and the system gets confused.
+	 * For an S3 card DMAing to disk 0xb8000-0xc0000 causes a hang.
+	 *
+	 * To workaround this the SDRAM window on the PCI bus is shifted
+	 * to 0x20000000.  0x10000000 is taken by the CSR.
+	 */
+	{
+		/*
+		 * first setup the correct base address mask
+		 */
+		int memory_size = bootconfig.dram[0].pages * PAGE_SIZE;
+		uint32_t mask;
+
+		/* window has to be at least 256KB, and up to 256MB */
+		for (mask = 0x00040000; mask < 0x10000000; mask <<= 1)
+			if (mask >= memory_size)
+				break;
+		mask--;
+		mask &= SDRAM_MASK_256MB;
+		
+		bus_space_write_4(sc->sc_iot, sc->sc_ioh, SDRAM_BA_MASK, mask);
+		bus_space_write_4(sc->sc_iot, sc->sc_ioh, SDRAM_BA_OFFSET, 0);
+
+		/*
+		 * Now we need to shift the PCI window to the SDRAM
+		 */
+		bus_space_write_4(sc->sc_iot, sc->sc_ioh, SDRAM_MEMORY_ADDR, 0x20000000);
+
+		/* 
+		 * configure the dma range for the footbridge to match
+		 */
+		footbridge_dma_ranges[0].dr_sysbase = bootconfig.dram[0].address;
+		footbridge_dma_ranges[0].dr_busbase = 0x20000000;
+		footbridge_dma_ranges[0].dr_len = memory_size;
+	}
+
 	/* Attach the PCI bus */
 	fba.fba_pba.pba_pc = &footbridge_pci_chipset;
 	fba.fba_pba.pba_iot = &footbridge_pci_io_bs_tag;
Index: footbridge_pci.c
===================================================================
RCS file: /cvsroot/src/sys/arch/arm/footbridge/footbridge_pci.c,v
retrieving revision 1.13
diff -u -p -r1.13 footbridge_pci.c
--- footbridge_pci.c	25 Feb 2007 18:42:00 -0000	1.13
+++ footbridge_pci.c	25 Aug 2007 19:01:19 -0000
@@ -95,13 +95,15 @@ struct arm32_pci_chipset footbridge_pci_
 	footbridge_pci_intr_disestablish
 };
 
+struct arm32_dma_range footbridge_dma_ranges[1];
+
 /*
  * PCI doesn't have any special needs; just use the generic versions
  * of these functions.
  */
 struct arm32_bus_dma_tag footbridge_pci_bus_dma_tag = {
-	0,
-	0,
+	footbridge_dma_ranges,
+	1,
 	NULL,
 	_bus_dmamap_create, 
 	_bus_dmamap_destroy,

--------------000905070405010307010703--