Subject: Re: SMP re-entrancy in kernel drivers/"bottom half?"
To: Daniel Carosone <dan@geek.com.au>
From: Jonathan Stone <jonathan@dsg.stanford.edu>
List: tech-kern
Date: 12/17/2003 18:30:43
In message <20031218021053.GK6077@bcd.geek.com.au>Daniel Carosone writes

>Which the smp-safe driver will have to aquire (via traditional splX?)
>before maniupulating. mbuf chains seem the logical example.

Allocating new mbuf chains (or freeing existing ones), yes.

If the mbuf is already hanging off a NIC structure, then the
per-softc spinlock is all you need.

[...]

>It certainly sounds like a reasonable compromise, but whether its
>useful or not will need to be measured (ie, do they spend most of
>their time holding the global locks on mbufs anyway).

Most of the drivers I've profiled spend most of their time either:
  a) banging registers on busses at 33Mhz to 133Mhz (*very* compared to GHz
     CPU cycles), or 
  b)  walking through internal softc structures.

Other operations can be combined (instead of freeing Tx-done mbufs one
at a time, link them into a monster chain and free the lot in one
`monster' free() call, which acquires and releases the spinklock once.

Or start adding per-CPU caches of free objects. Which is worth doing
anyway, once you get past very small UMA SMP systems.

>Fortunately, it sounds like implementing it for the purposes of the
>experiment should be relatively easy.

Yep, I thought that was an important consideration. :-/.