> Please try the attached test program:
> ./tst /dev/rwd0c 10000
> 
> this will report the speed of the bus (data transfers between the disk's
> cache and main memory). Maybe you're hitting the limit of what the hardware
> can do here.

I added 3 lines to get rid of some compiler warnings:

diff -c2 -r1.1 read_disk_cache_speed.c
*** read_disk_cache_speed.c     2005/08/22 15:28:29     1.1
--- read_disk_cache_speed.c     2005/08/22 15:29:22
***************
*** 17,21 ****
--- 17,24 ----
  #include <unistd.h>
  #include <sys/time.h>
+ #include <stdio.h>
+ #include <stdlib.h>
  
+ int
  main(int argc, char **argv)
  {

I ran it a few times, the fastest result on a mostly idle system:

17471975 us, 35.771571 MB/s

Again, results are about the same at the beginning and the end
of the drive.

I/O on a SCSI drive at the same time drops the speed slightly,
to about 33 MB/s, perhaps due to PCI bus contention.

I/O on the other SATA drive on the same controller drops the
speed significantly, to 19-20 MB/s per drive.

It looks to me like the Sil 3512 controller is maxing out.

Another result that appears a bit odd, is that running

	dd | hexdump -C

reduces the throughput from 36 MB/s to 13 MB/s.  The CPU
still has a significant %idle as reported by "systat vmstat":

   9.9% Sy  47.1% Us   0.2% Ni   0.3% In  42.5% Id

The whole disk has the same data, so hexdump only generates
3 lines of output total.

I haven't yet figured out how to blame this result on the
3512 controller.  :-)