jtc@acorntoolworks.com (J.T. Conklin) writes:
> David Laight <david@l8s.co.uk> writes:
>>> I've been running these implementations on my main development
>>> box for many months, without problems.
>> Also the optimisation '(word32 - 0x01010101) & 0x80808080' is only
>> valid for 7-bit ascii data.  For 8-bit data it is rather pointless.
>
> Even for 8 bit data, it only fails a small percentage of the time.  In
> my measurements, it still outperformed the current implementations
> with random 8 bit data.

I rechecked this by running both ((word32 - 0x01010101) & 0x80808080)
and ((word32 - 0x01010101) & ~word & 0x80808080) through all possible
values, and discovered the false positive rate is much higher than I
remembered.  It looks likely I inverted the ratio.

Considering 8 bit chars, adding the &~word to the expression may be
worthwhile.  I'll experiment with the costs of adding &~word to the
loop versus the costs of checking each byte individually after a 
false positive.

    --jtc

-- 
J.T. Conklin