NetBSD-Users archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: NetBSD Raid5, slow write speeds, using big disks?!



On 6/23/26 8:30 PM, Robert Elz wrote:
     Date:        Tue, 23 Jun 2026 16:03:42 +0200
     From:        smurfd <smurf.daemon%mail.smurfd.me@localhost>
     Message-ID:  <095e31e2-0627-405c-82eb-a57e2ccefb0c%mail.smurfd.me@localhost>

   | Its tiresome though, doing a change then waiting 8 to 72 hours and
   | seeing that the change had no effect.

While you're experimenting, make a much smaller raid array, one that is
just 10GB or so (ok, at least a couple of multiples of the amount of RAM
in the system) which will init, newfs, ... all very quickly, and is big
enough to test performance.  You don't need to do the whole thing for that.
(Doing it just on the hope that this will be the last time, will almost
certainly have Murphy intervene, and once you have the basics correct,
just making the number of blocks bigger won't change anything except its
size.)   So for the raid partitions on wd0 (etc) use 1/2 of the size you
want (for testing) for each (for a 3 drive raid5).

   | My raid experience is, as you have guessed, small... coming from zfs,
   | where it was kind of plug and play, i expected some push back here...
   | not this much though.

Beyond what you seem to be doing, just make sure that the filesystems
inside the raid are correctly aligned on raid stripes (which will be
64 blocks, 32KB, assuming you used the same params as before, except
with a 3 drive set instead of 4).   That is, everything should be aligned
on (and sized to be) a multiple of 64 blocks within the raid, and your
filesystem block size (-b to newfs) should be 32768 or 65536 (you can set
a smaller frag size, -f, any binary fraction of the block size, down to 1/8,
ie: 4096 for 32K blocksize, is OK, and uses less space for small files,
including directories.)   You could also use SectPerSU as 16 if you're
needing to conserve space (at the expense of some speed) - if you do that
all the numbers (bigger than 4) above (in this paragraph) can be halved
as well.  You'd use less RAM for the raidset, and with a smaller blocksize
potentially have less wasted space in the filesystem (though that is a
complex thing to evaluate.)  I wouldn't recommend < 16 though, the performance
loss then is likely to be more noticeable, but you can certainly test it,
it needs to be a power of 2, and is a number of blocks - the actual raidset
stripe size is then the number of data drives (don't count the parity)
multiplied by that number.   [Yes, I know, with raid5, all drives are data
drives technically, all are also parity drives, but it is simpler to forget
that optimisation, and just say that 1 of the drives is parity, the remainder
are data .. just don't try and speculate which drive is which, as they
really aren't.]

kre

A bit of an update. I did not see any big difference in raid5, if i use 3 or 4 drives.
Not either if i use like 64k blocks / align or 1m... this has me worried...
So, Tried to do a raid10, and got blazing speeds! with these parameters, cutting away some parts... and yes i know i can remove the -s 1g for the log, if im not going ot use it... and yes labels are incorrectly named raid5 even with the raid10... Tried to do a raid5 with these parameters aswell, but no real difference, i think it is the speed i pasted below for raid5...
gpt add -t raid -l raid5@wd0 -b 1m -a 1m -s 1000g wd0 ****
gpt add -a 1m -b 1m -t ffs -l island raid1
gpt add -a 1m -b 1m -t ffs -l island raid2
newfs -O2 -a 1m -b 64k -s -1g /dev/dk7
newfs -O2 -a 1m -b 64k -s -1g /dev/dk8

gpt add -a 1m -b 1m -t ffs -l island raid0
dkctl raid0 addwedge island3 1048576 4191152128 ffs
newfs -O2 -a 1m -b 64k -s -1g /dev/dk9

Where for both raid1 and raid0 i used
# sectPerSU SUsPerParityUnit SUsPerReconUnit RAID_level_0
128 1 1 0/1 depeding on raid type

START queue
fifo 127

4194304000 bytes transferred in 8.255 secs (508092549 bytes/sec) CHAZAAAM!
The best i got for raid5 so far, is:
4194304000 bytes transferred in 107.170 secs (39136922 bytes/sec)
So raid10 is the backup plan :)

after re-reading your mail, i will try raid5 a few more attempts...
In raid5.conf you suggest: 16 1 1 5 that would have me use these commands while doing the disks and raid
gpt add -t raid -l raid5@wd0 -b 64k -a 32k -s 1000g wd0 ****
gpt add -a 32k -b 64k -t ffs -l island raid5
newfs -O2 -a 32k -b 64k -s -1g /dev/dk7

or did i missunderstand something?
Many many thanks again, for bearing with me!!
br smurfd



Home | Main Index | Thread Index | Old Index