Re: odroid-c1's awge0 on 9?

[Christoph Badura <bad%bsd.de@localhost>]

On Thu, Oct 31, 2019 at 06:18:03PM -0400, Andrew Cagney wrote:
> I was reminded (off list) of the old bug where the network only worked
> if plugged in after the OS had booted.
> That helped - if the cable is plugged in after booting things seem to work.
> Interestingly, the old bug only occurred with 100mb  -  another work
> around was to connect to a 1gb hub.

Meanwhile I obtained an odroid-c1 and I see the same behavior on -current:
Booting the kernel with the ethernet cable plugged in the odroid doesn't
see the DHCP responses.  Connecting the cable right after the kernel has
mounted the root file system makes the problem go away.

This happens regardless whether connected to the Zyxel GS2200 GbE switch
and negotiating 1G link speed or a Netgear FS108 switch and negotiating
100Mbit link speed.

With DWC_GMAC_DEBUG enabled and a crude debug change I see that when
booting with cable connected the following happens early:

/soc/bus@c4300000/scu@0 at simplebus4 not configured
awge0: setting MAC conf register: 0060208c
awge0: INTR status: 00000001, DMA status: 0
cpu_topology_init: info bogus, faking it

So we're getting an AWIN_GMAC_MII_IRQ early.  And it sets the MAC conf
register for non-FDX.  Forcing FDX doesn't change the behavior, though.

Later dma_status never indicates an RX interrupt when booting with the
cable connected.

I also tested powering on the odroid with the cable not connected and
plugging the cable in after uboot has loaded the kernel.  Same failure
mode.

Plugging in the cable after the kernel has mounted the root file system
the odroid gets  RX interrupts.

I didn't do load tests of the network.  Just basic DHCP client config,
IPv6 RA processing, DNS lookups and pinging some hosts (both IPv4 and
IPv6).

--chris

Index: dwc_gmac.c
===================================================================
RCS file: /cvsroot/src/sys/dev/ic/dwc_gmac.c,v
retrieving revision 1.68
diff -u -r1.68 dwc_gmac.c
--- dwc_gmac.c	19 Oct 2019 06:40:20 -0000	1.68
+++ dwc_gmac.c	16 Jan 2020 10:10:12 -0000
@@ -43,7 +43,7 @@
 
 __KERNEL_RCSID(1, "$NetBSD: dwc_gmac.c,v 1.68 2019/10/19 06:40:20 tnn Exp $");
 
-/* #define	DWC_GMAC_DEBUG	1 */
+#define	DWC_GMAC_DEBUG	1 /* */
 
 #ifdef _KERNEL_OPT
 #include "opt_inet.h"
@@ -177,6 +177,7 @@
 static void dwc_gmac_dump_rx_desc(struct dwc_gmac_softc *);
 static void dwc_dump_and_abort(struct dwc_gmac_softc *, const char *);
 static void dwc_dump_status(struct dwc_gmac_softc *);
+static void dwc_print_status(struct dwc_gmac_softc *, uint32_t, uint32_t);
 static void dwc_gmac_dump_ffilt(struct dwc_gmac_softc *, uint32_t);
 #endif
 
@@ -1230,6 +1231,11 @@
 
 	mutex_enter(&sc->sc_rxq.r_mtx);
 	for (i = sc->sc_rxq.r_cur; ; i = RX_NEXT(i)) {
+#ifdef DWC_GMAC_DEBUG
+		aprint_normal_dev(sc->sc_dev,
+		    "dwc_gmac_rx_intr: checking desc #%d (r_cur: %d)\n",
+		    i, sc->sc_rxq.r_cur);
+#endif
 		bus_dmamap_sync(sc->sc_dmat, sc->sc_dma_ring_map,
 		    RX_DESC_OFFSET(i), sizeof(*desc),
 		    BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE);
@@ -1445,7 +1451,14 @@
 
 	dma_status = bus_space_read_4(sc->sc_bst, sc->sc_bsh,
 	    AWIN_GMAC_DMA_STATUS);
-
+#ifdef DWC_GMAC_DEBUG
+	dwc_print_status(sc, status, dma_status);
+#if 0
+	aprint_normal_dev(sc->sc_dev,
+	    "dwc_gmac_intr: status %08x dma_status %08x\n",
+	    status, dma_status);
+#endif
+#endif
 	if (dma_status & (GMAC_DMA_INT_NIE | GMAC_DMA_INT_AIE))
 		rv = 1;
 
@@ -1681,6 +1694,12 @@
 	     AWIN_GMAC_MAC_INTR);
 	uint32_t dma_status = bus_space_read_4(sc->sc_bst, sc->sc_bsh,
 	     AWIN_GMAC_DMA_STATUS);
+	dwc_print_status(sc, status, dma_status);
+}
+
+static void
+dwc_print_status(struct dwc_gmac_softc *sc, uint32_t status, uint32_t dma_status)
+{
 	char buf[200];
 
 	/* print interrupt state */