Re: RAIDframe performance (RAID-5)

[Greg Oster <oster%cs.usask.ca@localhost>]

Johan Ihren writes:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> 
> On 28 Aug 2008, at 23:26, Greg Oster wrote:
> 
> > Aha...  In a previous post (when I wasn't paying attention ;) ) you  
> > said:
> 
> >> RAID5 w/ following parameters:
> >> * START layout
> >> * # sectPerSU SUsPerParityUnit SUsPerReconUnit RAID_level_1
> >> * 128 1 1 5
> >
> > Change that 128 to 64 or even 32.
> >
> > You have 3 disks.  That means for a given stripe you have "2 disks
> > that will have data, 1 disk with parity".  For a given stripe then,
> > that's 128+128 blocks of data... which is 128K of data -- which is
> > larger than the 64K MAXPHYS value (the largest amount RAIDframe will
> > ever be handed for one IO :( ).  So what's happening here is that
> > you're only ever given a max of 64K, and so RAIDframe is *always*
> > doing small-stripe-writes.  Change the 128 to 64, and now your
> > stripes will have 64K of data, which at least will give it a fighting
> > chance.
> 
> Ok, I interpret that as:
> 
> ($stripesize * 2 * ($disks -1)) < MAXPHYS

($stripewidth / 2 * ($disks - 1)) == MAXPHYS  :)
 
> I.e. the stripe size should go down with an increasing number of disks  
> in the RAID set. 

Right.

> Previously I thought it was the other way around.  
> Things are finally getting more understandable.
> 
> > A stripe width of 32 *may* perform better (can't say for
> > sure), so if you're wanting to play that's another thing you can
> > try...
> 
> I've done six tests, for bsize/fsize 16K/2K-->32K/4K-->64K/8K  
> respectively, and both 64K and 32K stripe sizes. Very interesting.  
> Three winners and three losers with absolutely nothing in between.

That's the impact of the "Small Write" penalty...  If you have to pay 
it, it hurts bad.  If you can avoid it, you can do quite well.

> The  
> 64K stripe size with 64K bsize probably hits the absolute optimal  
> performance (this is seems to be very close to 2x the raw single disk  
> performance) but 32K/32K and 32K/64K are both also very good at 80+MB/ 
> s both read AND write.

[snip]

Yes, these numbers are much better...  (It's fun to tweak things and 
watch performance get better...)

> > Oh.. and don't worry about rebuilding parity before doing the tests
> > -- parity only matters if you care about the data in the event of a 
> > failure ;)
> 
> Right, I actually knew that, but I was not thinking straight. Thanks  
> for pointing it out. And many thanks for helping me to understand this  
> much better than before.

No problem... glad to have helped. :)

Later...

Greg Oster