Re: NetBSD Raid5, slow write speeds, using big disks?! - Resolved

[Robert Elz <kre%munnari.OZ.AU@localhost>]

    Date:        Thu, 2 Jul 2026 19:13:48 +0200
    From:        smurfd <smurf.daemon%mail.smurfd.me@localhost>
    Message-ID:  <24d1ea91-49dd-4477-86e9-a0d67964b28c%mail.smurfd.me@localhost>


  | START disks
  | /dev/dk3
  | /dev/dk4
  | /dev/dk5
  |
  | START spare
  | /dev/dk6

Those partitions should have labels as well (each a distinct one, which
I see you have done) and the config file should have NAME=raid5@wd0

This is less important if you turn on raid autoconfig

	raidctl -A yes raid5

in which case the kernel will just look for type raid partitions
and attempt to reconfigure them (it makes no difference what the
device names are in that case, the raid labels are all that is
required).

The one disadvantage to that is that, at the minute anyway (and I
am not aware of any current work to change this) spare drives cannot
be autoconfigured, you'd need to add the spare back after each reboot

	raidctl -a NAME=raid5_spare raid5

(again using whatever you like for the label).

You can turn autoconfig on & off whenever you like, it will take
effect at the next reboot (it is also possible to have the root
filesys on the raid if autoconfig is on, but your setup - and mine -
doesn't want that, so I won't mention it again).

If autoconfig is on, on all you raid arrays (which might be just this
one, doesn't matter, the all is important, not the number) you can turn
raid off in /etc/rc.conf - that is needed only to have /etc/rc configure
the raid partitions at boot time - if the kernel has already done it,
that isn't needed - but nor does it do any harm, if raid5 is already
configured, the rc script won't attempt to do it again.

  | # destroy possible previous raid5 config, also important while testing
  | gpt destroy raid5
  | raidctl -u raid5
  | dd if=/dev/zero of=/dev/rraid5 bs=1024k count=1� # i saw this was needed 
  | to really clear its parameters

First "raidctl -u" isn't intended to destroy anything, it just removes the
raid from the kernel (kind of like umount for a filesystem) - it can be a
good idea if you're not going to be using the raid for a while, to make sure
everything is left in a good state - just in case the system crashes.
(then just
	raidctl -c /etc/raid5.conf raid5
to bring it back again - note no -C and definitely no raidctl -I or raidctl -i)

But you shouldn't need to do that, the 3 raidctl commands
	raidctl -C ...
	raidctl -I ...
	raidctl -i ...

should simply ignore anything that is there, and make a new one.

(You also shouldn't be able to do the dd after the raidctl -u, as raid5
won't be known by the kernel any more).

On the other hand, you do need the "gpt destroy" if you are going to
be recreating a different sized raid, and you need to do it on the old
one, before it is unconfigured, so the secondary partition tables, at
the end, can be destroyed as well.

  | Tried the above configs, but instead of 3 disk + 1 spare, i used 4 
  | disks. Speeds were alot slower,

Yes, they would be, as now (3 discs) the (full) stripe size is 32KB (2 * 16KB).
That is either the same, or in your case half, which is fine, your filesystem
block size.   With 3 data discs you get 3 * 16KB (48KB) stripe size.   That's
not easy to make a block the same size (or twice) as block sizes are powers
of two, and 3 isn't!

  | gpt create -Af raid5
  | gpt add -a 32k -b 64k -t ffs -l island raid5
  | dkctl raid5 addwedge island <last two values from above commands output> ffs

I'm not sure if you said which version of NetBSD you're using, but in
anything relatively modern, that dkctl is no longer required (gpt does
it for you, unless you specifically tell it not to).

  | /etc/fstab add row, once done, since /dev/dk7 might change at reboot. 
  | Where NAME= is the same value you
  | use in the -l parameter for the raid.
  | NAME=island� � �/mnt/island� � �ffs� � �rw,-m1666,log� � �1 1

As above, it isn't just dk7 that might change, any dkN might (wdN numbers
might as well, but that usually requires a hardware change to make happen).

Using /dev/dkN (or /dev/rdkN as appropriate) on the command line is fine,
as long as you use the right value for N, but you really want to avoid
embedding those names in files anywhere at all if you can possibly avoid it.

  | Check dmesg after each command. When you create the raid with the 
  | raidctl -C,
  | it might complain with fatal errors, but they are ignored. that seems to 
  | be okey!

Hmm, fatal errors are rarely OK!   But whether what you saw was really
a fatal error or not I can't say, as you didn't say what it complained
about.

  | To save time on the raidctl -iv, start with smaller disks, like 1TB.

Not smaller discs, necessarily, just smaller partitions on those discs.

  | ps. no commentary needed! :)

Sorry...

kre