Re: Help with low raid5 performance

[Stephen Borrill <netbsd%precedence.co.uk@localhost>]

On Tue, 11 Jan 2011, Greg Oster wrote:
> On Tue, 11 Jan 2011 08:43:12 -0500
> Thor Lancelot Simon <tls%panix.com@localhost> wrote:
> > > > > I'm using a 4-disk RAID 5 with the partition starting at 45416448
> > > > > (which is very well aligned (on a 65536 boundary!).
> >
> > This should not be aligned on a 64k boundary with 3 data disks.  It
> > should be aligned on a 48K boundary!

Actually 45416448 is both on a 48k boundary and a 64k boundary.

> > Greg, with RAIDframe sector 0 should always be at the beginning of a
> > stripe, so 48K alignment with no remainder is correct, yes?
>
> Yes, and yes. (in this case)

Which it is (by luck in this case!).

> > > Changing to 32 1 1 5 gave:
> > >       -------Sequential Output-------- ---Sequential Input--
> > > --Random-- -Per Char- --Block--- -Rewrite-- -Per Char- --Block---
> > > --Seeks--- MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec
> > > %CPU  /sec %CPU 2048  9137  9.6  7440  3.3  3045  1.3 61801 82.3
> > > 149558 37.1 228.2  2.4
> > >
> > > These were formatted with: newfs -O 2 -f 4096 -b 32768
> >
> > This newfs command line is wrong.  32 doesn't go evenly into 48.
> > Remember that the RAIDframe sectors-per-stripe is in half-kilobyte
> > units...
> >
> > You probably want newfs -f 2048 -b 16384 -a 3.
>
> I'm curious to see the results of the above.

newfs(8) says -a is obsolete...

I've run the tests through and it doesn't help much. RAID5 on 3 disks _is_ 
significantly quicker than on 4 (4x quicker for character writes and 7x 
for block writes). But contig 3 vs contig 4 and using -f 2048 -b 16384 
make little difference (a handful of % at best).

I've ensured all partitions are 48k aligned.

http://projects.precedence.co.uk/netbsd/RAIDtest.pdf

--
Stephen