Re: NetBSD Raid5, slow write speeds, using big disks?!

[Robert Elz <kre%munnari.OZ.AU@localhost>]

    Date:        Tue, 23 Jun 2026 16:03:42 +0200
    From:        smurfd <smurf.daemon%mail.smurfd.me@localhost>
    Message-ID:  <095e31e2-0627-405c-82eb-a57e2ccefb0c%mail.smurfd.me@localhost>

  | Its tiresome though, doing a change then waiting 8 to 72 hours and 
  | seeing that the change had no effect.

While you're experimenting, make a much smaller raid array, one that is
just 10GB or so (ok, at least a couple of multiples of the amount of RAM
in the system) which will init, newfs, ... all very quickly, and is big
enough to test performance.  You don't need to do the whole thing for that.
(Doing it just on the hope that this will be the last time, will almost
certainly have Murphy intervene, and once you have the basics correct,
just making the number of blocks bigger won't change anything except its
size.)   So for the raid partitions on wd0 (etc) use 1/2 of the size you
want (for testing) for each (for a 3 drive raid5).

  | My raid experience is, as you have guessed, small... coming from zfs, 
  | where it was kind of plug and play, i expected some push back here... 
  | not this much though.

Beyond what you seem to be doing, just make sure that the filesystems
inside the raid are correctly aligned on raid stripes (which will be
64 blocks, 32KB, assuming you used the same params as before, except
with a 3 drive set instead of 4).   That is, everything should be aligned
on (and sized to be) a multiple of 64 blocks within the raid, and your
filesystem block size (-b to newfs) should be 32768 or 65536 (you can set
a smaller frag size, -f, any binary fraction of the block size, down to 1/8,
ie: 4096 for 32K blocksize, is OK, and uses less space for small files,
including directories.)   You could also use SectPerSU as 16 if you're
needing to conserve space (at the expense of some speed) - if you do that
all the numbers (bigger than 4) above (in this paragraph) can be halved
as well.  You'd use less RAM for the raidset, and with a smaller blocksize
potentially have less wasted space in the filesystem (though that is a
complex thing to evaluate.)  I wouldn't recommend < 16 though, the performance
loss then is likely to be more noticeable, but you can certainly test it,
it needs to be a power of 2, and is a number of blocks - the actual raidset
stripe size is then the number of data drives (don't count the parity)
multiplied by that number.   [Yes, I know, with raid5, all drives are data
drives technically, all are also parity drives, but it is simpler to forget
that optimisation, and just say that 1 of the drives is parity, the remainder
are data .. just don't try and speculate which drive is which, as they
really aren't.]

kre