Re: Beating a dead horse

[Greg Oster <oster%netbsd.org@localhost>]

On Tue, 24 Nov 2015 21:57:50 -0553.75
"William A. Mahaffey III" <wam%hiwaay.net@localhost> wrote:

> On 11/24/15 19:08, Robert Elz wrote:
> >      Date:        Mon, 23 Nov 2015 11:18:48 -0553.75
> >      From:        "William A. Mahaffey III" <wam%hiwaay.net@localhost>
> >      Message-ID:  <5653492E.1090102%hiwaay.net@localhost>
> >

> >    | The machine works well except for horribly slow I/O to the
> > RAID5 I setup
> >
> > What is your definition of "horribly slow" and are we talking read
> > or write or both ?
> 
> 4256EE1 # time dd if=/dev/zero of=/home/testfile bs=16k count=32768
> 32768+0 records in
> 32768+0 records out
> 536870912 bytes transferred in 22.475 secs (23887471 bytes/sec)
>         23.28 real         0.10 user         2.38 sys
> 4256EE1 #

Just to recap: You have a RAID set that is not 4K aligned with the
underlying disks.  Fixing that will certainly help, but there is a bit
more tweaking to be done:

 1) Your stripe size is 32 blocks - 16K.  So if you are writing in 16K
 chunks (as with the dd above), at best you're guaranteed to be
 re-reading the old data, re-reading parity, writing new data, and
 writing new data for each block.  That's the 'small write problem'
 with RAID 5.  (at worst, that 16K might span two components, in which
 case you'll have to wait for three reads and three writes to complete,
 instead of two reads and two writes)

 2) Your filesystem block size is 32k, meaning a filesystem block write
 will never give you a full stripe write either.  See 1) for
 performance implications.

 3) If you want more write speed, then:
   a) get the RAID components and RAID 4K-aligned with the underlying
      disks.  Your 32 block stripe width is perfect. The number
      of data components is good.
   b) get the filesystem 4K-aligned
   c) use a 64K block size for the filesystem
   d) Use 'bs=64k' (or 'bs=10m') to see better IO performance
  
 4) As others have said, if you need high-performance writes, RAID 5 is
 probably not what you want, especially if you're not streaming writes
 in 64K chunks.

Later...

Greg Oster