On Fri, Mar 10, 2000 at 12:48:49AM -0800, Hal Murray wrote:

 > What's the downside of making that bigger?  I assume it's allocating 
 > memory.  How big is each cluster?  What's the impact of using jumbo 
 > packets on GBE? 

It's really a high-water mark on how much memory can be allocated by
mbuf clusters (which are really sort-of misnamed.. they're actually large
buffers used as external storage for mbufs).

Clusters are MCLBYTES in size.  This defaults to 2K on all NetBSD platforms.

The behavior of making NMBCLUSTERS buffer depends on which port you're using.

	- On ports that use direct-mapped pool pages (Alpha, MIPS, PowerPC,
	  VAX), this merely places an upper-bound on the number of pages
	  which can be allocated to mbuf clusters.  On these platforms,
	  the limit can be adjusted dynamically with kern.mbuf.nmbclusters.

	- On ports that do not use direct-mapped pool pages (everything but
	  the platforms listed above), NMBCLUSTERS is used to size a kernel
	  virtual memory map `mb_map'.  This map CANNOT be resized at run
	  time.  Explaining why is beyond the scope of this message.  Just
	  trust me on this one.

Now, in the latter case, making NMBCLUSTERS too big wastes kernel virtual
address space.  On many platforms, this is a rare and precious resource,
not to be squandered aimlessly.

In the case of jumbo frames for Gig-E, there are a couple of options:

	- Make MCLBYTES larger on your system, say, 9k.  Unfortunately,
	  this does not currently work, as the pool allocator is not able
	  to allocate objects larger than a page size.  Besides, most of
	  the code in the tree assumes that an mbuf cluster will be
	  physically contigous, so you have to be careful about this.  In
	  any case, you could make it the same size of a page, which is
	  always bigger than a standard Ethernet frame, but won't be as
	  big as the biggest jumbo frame you could send/receive.

	- Manage a pool of larger DMA-safe buffers in the driver, and use
	  the mbuf external storage facility to attach them to mbufs.  This
	  is what is currently done in the Alteon ACEnic driver (`ti').

 > Would it be reasonable to add a boot-time switch to make it bigger 
 > during initialization?  I'm interested in systems that can't increase 
 > it on the fly? 

Possibly, but there's currently not a good way to do this, short of booting
into DDB and patching the nmbclusters variable (or patching it in the
kernel image with gdb and rebooting).

-- 
        -- Jason R. Thorpe <thorpej@shagadelic.org>