Re: CVS commit: src/sys/kern

[Andrew Doran <ad%netbsd.org@localhost>]

On Sun, Feb 10, 2008 at 01:56:17PM +0000, Andrew Doran wrote:

> Log Message:
> Disable the event counters by default, since they create terrible cache
> behaviour on MP systems.

The problem is: to update the counters the CPU needs to write to a cache
line - or even two lines, since ev_count is 64 bit. In order to do that, the
cache line first has to be made private to the CPU that is making the
update. To get the cache line into that state, the updating CPU forces the
cache line away from any other CPUs that hold it - the others could have an
exclusive or shared hold on it. That can mean a writeback to main memory if
another CPU holds the line modified - I'm not sure if they can be
transferred directly. On a busy MP system, the chances are that the cache
traffic around the event counters is going to occur on nearly every call to
get the time, so it means hundreds if not thousands of clock cycles added to
every call.

We have the 'percpu' allocator now so it would be possible have a per-CPU
component for every event counter so that updates don't suffer from this
problem. The difficult bit is (I think) figuring out how to use that in
assembly language, and how to deal with bootstrap when the VM system and the
percpu allocator aren't working yet.

Thanks,
Andrew