Re: event counting vs. the cache

[David Laight <david%l8s.co.uk@localhost>]

On Thu, Jan 17, 2013 at 05:25:44PM -0600, David Young wrote:
> 
> We can end up with silly values with the status quo, too, can't we?  On
> 32-bit architectures like i386, x++ for uint64_t x compiles to
> 
>       addl $0x1, x
>       adcl $0x0, x
> 
> If the addl carries, then reading x between the addl and adcl will show
> a silly value.
> 
> I think that you can avoid the silly values.  Say you're using per-CPU
> counters.  If counter x belongs to CPU p, then avoid silly values by
> reading x in a low-priority thread, t, that's bound to p and reads hi(x)
> then lo(x) then hi(x) again.  If hi(x) changed, then t was preempted by
> a thread or an interrupt handler that wrapped lo(x), so t has to restart
> the sequence.

You don't actually need to restart, the value new_hi:0 happened while
the function was running - so is a valid response.

I think there is another problem with that scheme - but I can't remember it!

There are other schemes that handle the case of a single writer (guaranteed
by something else) and occaisional readers that don't want to acquire
whatever context single-threads the writes.

One is (I think), add an extra 32bit counter. The writer increments it
before updating the stats block, and again afterwards. The reader spins
until it is even, reads all the stats, then checks the value hasn't changed.

Another involves writing a 3rd value that contains the middle bits of the
value (from both the high and low parts). The reader checks consistency.

Or, assume a 63 bit counter will also not wrap and replcate the high bit
of the low word into the low bit od the high word - reader verifies.

On 64bit systems I sometimes wonder whether it is necessary for stats to
be 100% accurate - so not using locked increments may be ok.

There are also issues making stats per-cpu.
While not unreasonable for 2, 4 or 8 cpus it gets a bit silly when there
are 1024 or more.

The differences between common and uncommon (eg error) stats also needs to
be considered.

        David

-- 
David Laight: david%l8s.co.uk@localhost