pthread_rwlock starvation

[Sad Clouds <cryintothebluesky%googlemail.com@localhost>]

Hi, I'm running NetBSD 5.0.2 on dual x86 Pentium 3 machine. It seems
there are issues with pthread_rwlock, lock starvation.

I was testing a concurrent hash table, which uses pthread_rwlock, since
most of the time hash table lookups are read-only.

Below are hash table statistics when ASCII strings
from /usr/share/dict/words are hashed. There are 1024 read/write locks,
striped over 524288 buckets.

p3smp$ ./a.out
Hash table stat
entries      = 234976
buckets      = 524288
locks        = 1024
buckets used = 189791
buckets free = 334497

Hash table chaining stat
2 to 3        = 38457
4 to 7        = 604
8 to 15       = 0
16 to 31      = 0
32 to 63      = 0
64 to 127     = 0
128 to 255    = 0
256 to 511    = 0
512 to 1023   = 0
1024 and over = 0

To test concurrency my test program creates two threads, each of the
threads runs a tight loop:

lookup hash entry
remove entry from hash table
reinsert entry back into hash table

Both CPUs start in 100% busy state, but after a while one CPU becomes
100% idle, while another CPU stays in 100% busy state. The idle CPU
stays blocked for about 20-30 sec. after that it goes back to 100% busy
state.

I tested the same program on the same machine, but booted into Solaris.
In Solaris all CPUs are busy 100% all the time. This seems to suggest
issues with NetBSD read/write locks, or something else.

Any ideas?