Alright, I found the problem and turned a previously unuseable mc
interface into a 980KB/s rip roaring 0 packet loss mc interface. The
problem turned out to be pretty simple and the temporary fix was only
one line.

In if_mc.c, in mc_attach, there is the comment:
        /*
         * allocate memory for receive buffer and mark it non-cacheable
         * XXX This should use the bus_dma interface, since the buffer
         * needs to be physically contiguous. However, it seems that
         * at least on my system, malloc() does allocate contiguous
         * memory. If it's not, suggest reducing the number of buffers
         * to 2, which will fit in one 4K page.
         */
sc->sc_rxbuf = malloc(MC_NPAGES * PAGE_SIZE, M_DEVBUF, M_WAITOK);
sc->sc_rxbuf_phys = kvtop(sc->sc_rxbuf);
memset(sc->sc_rxbuf, 0, MC_NPAGES * PAGE_SIZE);

Well, apparently on my 7300 the memory is not contiguous, or at least
it's not being written to. I examined the contents of the buffers during
transmission losses and found the upper 2 buffers to be empty,
correlating to the 50% packet loss in ping. I reduced the number of
buffers to 2 as the comment suggests and this immediately fixed the
problem.

Feedback requested from testers, as this should get committed as quickly
as others can confirm this works. Later someone can write a proper
bus_dma allocation, and I'll clean up the media type patch I have for
if_mc.c, and maybe we can close that PR.

tim

Index: if_mcvar.h
===================================================================
RCS file: /cvsroot/src/sys/arch/macppc/dev/if_mcvar.h,v
retrieving revision 1.7
diff -d -u -r1.7 if_mcvar.h
--- if_mcvar.h  9 Apr 2003 01:54:45 -0000       1.7
+++ if_mcvar.h  13 Jan 2005 17:49:14 -0000
@@ -45,7 +45,7 @@
                                    (sc)->sc_regh, MACE_REG(reg),(val)))

 #ifndef        MC_RXDMABUFS
-#define        MC_RXDMABUFS    4
+#define        MC_RXDMABUFS    2 /* 4 */
 #endif
 #if (MC_RXDMABUFS < 2)
 #error Must have at least two buffers for DMA!
@@ -80,6 +80,8 @@
        void            (*sc_mediastatus) __P((struct mc_softc *,
                                struct ifmediareq *));

+       u_int           sc_defaultmedia;
+
        bus_space_tag_t sc_regt;
        bus_space_handle_t sc_regh;