Hi,

I'm still playing with a server-to-be, equipped with a 3ware 9550SXU RAID
controller.  I'm having performance problems when manipulating lots of small
files (like extracting pkgsrc.tar.gz).  With 4 GB of RAM, data fits nicely
in the buffer cache and when the time comes, it is flushed to disk.  This is
when the problems begin: The disk (especially RAID5) becomes 100% busy and
all other disk accesses are severely affected - loading a small binary from
system disk can take 10...100 seconds, so the system is pretty much dead
until the cache flush is completed.

Is there a way to limit the number of transfers to disk per second?  Or
better yet, a way to limit the disk load ("%busy" in systat) to, say, 90%?
Ideally, this should probably affect only ioflush, so other disk accesses
would still be possible.  Or maybe there's an even better way of preventing
the cache flush from overloading the disk(s)?

I may try an Areca RAID controller (which is rumoured to be faster) when
the driver is completed, but with a large buffer cache and a fast CPU, there
may not be a controller and/or disk fast enough.  I guess ioflush can
initiate hundreds of thousands of disk requests per second and nothing
is fast enough to keep the queue empty..

Limiting the ioflush speed may slow down the buffer flush a bit, but if
the system is still usable during that time, it's more than worth the
cost.  At the moment, all disk I/O through the RAID controller comes to
a halt which kills streaming and stops virtual machines trying to access
the disk.

Thoughts?

  -jm