Re: Dynamically allocated locks

[Andrew Doran <ad%netbsd.org@localhost>]

On Mon, Mar 24, 2008 at 10:10:07AM -0700, Matt Thomas wrote:

> On Mar 23, 2008, at 10:52 AM, Andrew Doran wrote:
> 
> >Hi,
> >
> >This patch allows for dynamically allocated locks.
> >
> >http://www.netbsd.org/~ad/lockobj.diff
> >
> >Each lock gets its own cache line, which can reduce cache coherency  
> >costs
> >when the lock is very active. The locks are reference counted so  
> >can be
> >shared between objects. That could be used for the two ends of a  
> >Unix socket
> >connection, or to share a lock between clone()d processes in order to
> >protect common state like signal actions, etc.
> >
> >Any objections?
> 
> Yes.  One / cacheline size is a dramatic waste of space.  Some  
> systems have
> 128 bytes per cacheline.  On systems with small caches or UP systems  
> it's
> extremely expensive.

On UP systems it's unnecessary. How about introducing a global variable that
is tuned during early boot by MD code? Maybe "coherency_unit"? I'm not
proposing to do this for all locks in the system, only those that are likely
to be particularly busy.
 
> Other than that its fine.  I can see not crossing a cacheline boundary,
> but only 1 / cacheline is just wrong.

It really does matter on MP systems. If you were to take a machine and have
all CPUs doing nothing but modifying operations on data within a single
cache line, then in the best case the machine would in effect be single
threaded: it would do no better than a single CPU system. In the worst case
it could be a lot slower due to (for example) costly writebacks to main
memory.

Andrew