Re: mutexes, locks and so on...

[Adam Hoka <adam.hoka%gmail.com@localhost>]

On Thu, 11 Nov 2010 17:22:03 +0100
Johnny Billquist <bqt%softjar.se@localhost> wrote:

> I've been looking through the netbsd mutex, and lock implementations the 
> last few days, with the intent to try and improve the performance on the 
> VAX, which has suffered somewhat in performance with NetBSD (maybe other 
> platforms as well, but they might not feel the pain that much :-) ).
> 
> While doing this, I have stumbled upon a few questions, as well as some 
> general observations.
> 
> A few observations first.
> The mutex implementation in place now, is nice in many ways, as it is 
> rather open to different implementations based on what the hardware can 
> do. However, I found that only one platform (hppa) is currently using 
> this. All others rely on the __HAVE_SIMPLE_MUTEXES implementation, which 
> utilize a CAS function. Obviously the VAX does not have a CAS, and it is 
> rather costly to simulate it, so I'm working on getting away from this. 
> (Does really all other platforms have a CAS?)
> 
> What would be nice here is if it had been possible to inline the 
> mutex_lock and mutex_unlock (as well as their spin equivalents) 
> functions, but I found out (the hard way) that this is not possible.
> 
> I also found out that mutex spin locks are more or less equivalent to 
> the old scheme of using splraise() and splx(). However, I noticed one 
> big difference. In the old days, splraise() and splx() were always 
> handled as a pair. You did not do:
> 
> a = splraise(..)
> b = splraise(..)
> splx(a)
> splx(b)
> 
> which would have been very broken.
> 
> With mutex_spin, you instead store the original spl at the first 
> mutex_spin_enter, and later calls to mutex_spin_enter can only possibly 
> raise the ipl further. At mutex_spin_exit, we do not lower the spl 
> again, until the final mutex_spin_exit, which resets the spl to the 
> value as before any mutex was held.
> The cause a slightly different behaviour, as the spl will continue to 
> possibly be very high, even though you are only holding a low ipl mutex. 
> While it obviously don't cause a system to fail, it can introduce delays 
> which might not be neccesary, and could in theory cause interrupts to be 
> dropped that needn't be.
> 
> Is this a conscious design? Do we not expect/enforce mutexes to be 
> released in the reverse order they were acquired?

Isnt that because of "priority inversion"?

> 
> Moving on to locks:
> the simple lock defined in lock.h is easy enough, and I haven't found 
> any problems with it. The rwlock, however, is written with the explicit 
> assumption that there is a CAS function. It would be great if that code 
> were a little more like the mutex code, so that alternative 
> implementations could be done for architectures where you'd like to do 
> it in other ways. Is the reason for this not being the case just an 
> oversight, a lack of time and resources, or is there some underlying 
> reason why this could not be done?
> 
> Also, there are a few places in the code in the kernel, where atomic_cas 
> is called explicitly. Wouldn't it be better if such code were abstracted 
> out, so we didn't depend on specific cpu instructions in the MI code?
> 
> Well, just a few thoughts, and a question for all of you... I'll happily 
> listed to your wisdom as I try to get my alternative locking on the VAX 
> running...
> 
>       Johnny


-- 
NetBSD - Simplicity is prerequisite for reliability