Re: Help with low raid5 performance

[Chuck Swiger <cswiger%mac.com@localhost>]

On Nov 10, 2010, at 11:22 AM, Peter Kerwien wrote:
> [ ... ]
> # dd if=/dev/zero of=/mnt/vol1/pub/zero bs=100m count=1k
> 1024+0 records in
> 1024+0 records out
> 107374182400 bytes transferred in 4056.655 secs (26468650 bytes/sec)
> => 25MB/s
> 
> I still not understand why I cannot reach higher write performance. I
> saw that the [system] process consumed like 10% (have a C2D @ 2.4GHz)
> CPU. Is the write speed limited due to CPU bound? Is there a limit how
> much CPU the process that calculates the xor checksums is allowed to use?

No, your CPU is sufficiently fast to keep up with the XOR calculations, and 
very probably is not the bottleneck.

One problem with a 3-disk RAID-5 setup is that writing 100MB of data to a bare 
drive will end up sending at least 150MB across your three disks.  The overhead 
of parity goes down as you add more drives, up until you reach a saturation 
point for the chipset (ie, limitation on the bandwidth and # of SATA available, 
or PCIe channels if you have RAID hardware).  However, that is a best-case 
situation, where you are writing blocks at least as big as the RAID stripe 
size; if you are doing smaller writes, you have to do a read/recalculate 
XOR/write cycle, which will be very slow.

It's normal for a RAID-5 volume to exhibit performance which is a factor of ten 
worse than a bare drive when doing lots of very small writes (ie, some database 
access patterns, revision control system changes from CVS/SVN/etc).  You can 
improve this somewhat by using RAID-50 instead of just RAID-5, and/or by using 
a real hardware RAID controller with plenty of onboard cache memory and battery 
backup.

If you value performance more than cost, RAID-1 or RAID-10 gives much better 
performance than RAID-5.

Regards,
-- 
-Chuck