Subject: Re: /netbsd: ti0: no free jumbo buffers -> how to tweak it off
To: None <bouyer@antioche.lip6.fr>
From: Havard Eidnes <he@netbsd.org>
List: current-users
Date: 03/09/2004 11:41:36
> > Sporadically I see a lot of
> > 
> >   /netbsd: ti0: no free jumbo buffers
> >   /netbsd: ti0: jumbo allocation failed -- packet dropped!
> > 
> > complaints from my
> > 
> >   ti0 at pci0 dev 10 function 0: Netgear GA620 1000BASE-SX Ethernet (rev. 0x01)
> >   ti0: interrupting at irq 10
> >
> > if setting mtu to higher values (> 1500).
>
> In sys/dev/pci/if_tireg.h you can bump TI_JSLOTS

Yes, you can (these could be enclosed by #ifndef / #endif, could
they not?), but even then if you do e.g. ttcp with large windows,
you may drive the driver (primarily the receiver) into
starvation.  In my setup that happened when using around or
slightly above 128KB windows in a back-to-back configuration with
1GHz P3 machines.

I see the FreeBSD driver has TI_JSLOTS at 384, but simply trying
to use that value with our driver just causes those messages
above to appear when configuring the interface up.

Hm... Further digging reveals if_ti.c FreeBSD revision 1.18 and
if_tireg.h revision 1.31, which changes ti_init_rx_ring_jumbo()
from allocating JSLOTS-20 to instead to use TI_JUMBO_RX_RING_CNT
buffers with (part of) the attached commit log message being

   Also changed the jumbo buffer allocation scheme just a little to avoid
   'failed to allocate jumbo buffer' conditions in certain cases.

This made the driver a whole lot happier; the messages above are
gone in my case now where they would appear earlier -- I can now do
back-to-back ttcp with 300KB windows, and I could set TI_JSLOTS to
384.  My current diffs follow below.  Additionally, I have in my
kernel config file:

options         TI_SSLOTS=256
options         TI_MSLOTS=256
options         TI_JSLOTS=384

These are the values used by the FreeBSD if_tireg.h, BTW -- should
we perhaps track these?

Regards,

- Havard

--------------------

Index: if_tireg.h
===================================================================
RCS file: /cvsroot/src/sys/dev/pci/if_tireg.h,v
retrieving revision 1.12
diff -u -r1.12 if_tireg.h
--- if_tireg.h	2003/05/14 13:03:36	1.12
+++ if_tireg.h	2004/03/09 10:34:12
@@ -985,9 +985,15 @@
  * allocated for the standard, mini and jumbo receive rings.
  */
 
+#ifndef TI_SSLOTS
 #define TI_SSLOTS	64 /* 256 */
+#endif
+#ifndef TI_MSLOTS
 #define TI_MSLOTS	64 /* 256 */
+#endif
+#ifndef TI_JSLOTS
 #define TI_JSLOTS	64 /* 256 */
+#endif
 #define TI_RSLOTS	128
 
 #define TI_JRAWLEN (ETHER_MAX_LEN_JUMBO + ETHER_ALIGN + sizeof(u_int64_t))
Index: if_ti.c
===================================================================
RCS file: /cvsroot/src/sys/dev/pci/if_ti.c,v
retrieving revision 1.58
diff -u -r1.58 if_ti.c
--- if_ti.c	2004/02/24 15:05:55	1.58
+++ if_ti.c	2004/03/09 10:34:12
@@ -976,7 +976,7 @@
 	int		i;
 	struct ti_cmd_desc	cmd;
 
-	for (i = 0; i < (TI_JSLOTS - 20); i++) {
+	for (i = 0; i < TI_JUMBO_RX_RING_CNT; i++) {
 		if (ti_newbuf_jumbo(sc, i, NULL) == ENOBUFS)
 			return(ENOBUFS);
 	};