Re: RAIDframe performance vs. stripe size

[Eduardo Horvath <eeh%NetBSD.org@localhost>]

On Sat, 12 May 2012, Edgar Fuß wrote:

> > In general it won't access just one filesystem block.
> > It will try to readahead 64KB
> Oh, so this declustering seems to make matters even more
> complicated^Winteresting.
> 
> Staying with my example of a 16K fsbsize FFS on a 4+1 disc Level 5
> RAIDframe with a stripe size of 4*16k=64k:
> 
> Suppose a process does something that could immediately be satisfied
> by reading one fs block (probably it matters whether that's a small file,
> a small portion of a large file, a small directory, a portion of a large
> directory, inodes, free list or whatever?). Now, if that, as I understand,
> always causes FFS to in fact issue a 64k request to RAIDframe, this would
> need to read a full stripe and so need all but one disc. So it can't be
> parallelised with another process' request, can it? Does this mean I'm better
> off with a stripe size of 4*64k if I'm after low latency for concurrent
> access?

The problem here is NFS, which requires writes to be persistent before 
returning status to the caller.

Under normal operation, ufs will attempt to use the buffer cache in the 
most efficient manner, doing readahead and delaying writes as much as 
possible to be able to do maximize the number of clustered operations as 
it can.  

Now if NFS does not do similar clustering on writes (I don't know NFS that 
well, especially V3 and V4 which allegedly have write optimizations) then 
you get the situation where the underlying ufs will try to cluster reads 
(satisfying reads out of the buffer cache is much faster than hitting the 
platters) but write out only single filesystem blocks (to satisfy the NFS 
consistency requirements.)  My understanding is that later versions of NFS 
(v3+) have a mechanism for the client side to request writes without the 
consistency guarantee and a separate explicit sync operation.  But using 
those is the responsibility of the NFS client machine.  Of course, if all 
the files are on the order of one filesystem block, clustering won't 
happen at all.

I think you should attempt to characterize your workload here to determine 
the size of the I/O operations the clients are requesting so you can 
decide if clustering is a benefit to you, and if not, turn it off.  (I 
think it can be tweaked with tunefs(8).)

Eduardo