Re: NetBSD Raid5, slow write speeds, using big disks?!

[Stephen Borrill <netbsd%precedence.co.uk@localhost>]

On Sat, 20 Jun 2026, J. Hannken-Illjes wrote:
> Others will know better, but I always thought a Raid5 with four disks spends most time with read-modify-write cycles as usually no requests will fit the stripe size.

Yes, with RAIDframe RAID-5 you want (2^n)+1 disks because of parity being 
separate, so your MAXPHYS data needs to split neatly across the remaining 
data disks. You are trying to split your data across 3 disks...

Also make sure you align everything for 4K sectors.

But, yes, I vote for using ZFS too.

> > On 20. Jun 2026, at 18:55, smurfd <smurf.daemon%mail.smurfd.me@localhost> wrote:
> >
> > Hey,
> >
> > This will be a long one, sorry for this.
> > I have created a RAID5 NAS using NetBSD 10.1, and im experiencing slow writing speeds. Or is my expectations all wrong? :)
> > I earlier had FreeBSD and ZFS zraid on the same Ugreen DXP4800 plus (https://nas-eu.ugreen.com/products/ugreen-nasync-dxp4800-plus-nas-storage)
> > The NAS drives are 4x7TB, less than a year old Segate Ironwolf NAS 3.5".
> > NetBSD is installed first on the SSD, using a base install. Then the Raid is created manually...
> >
> > Have asked here:
> > https://www.unitedbsd.com/d/1646-raid5-slow-write-on-big-disks/8
> > Just now redid the raid, for the Xth time, to compare the speeds i got using FreeBSD (because i didnt remember/had that written down)
> > I know, comparing zraid vs raid5 is like comparing apples to oranges, right?...
> >
> > What write speeds should i expect?! is probably the most important question...
> > Or where did i go wrong, because user error is VERY likely.
> >
> > First, i tried to follow the Summary section in the https://man.netbsd.org/raidctl.8, realizing that Disklabel supports no larger than 2TB disks.
> > So found this: https://wiki.netbsd.org/users/mlelstv/using-large-disks/ and tried to follow the Raidframe section.
> > Then i got the correct size i expected...
> >
> > The speed though, is slow! (again compared to zfs zraid)
> > When i had FreeBSD installed, i did the following. Installed the system to the SSD.
> > Then, did this, where **** is repeaded 4 times for the disks...
> > # gpart destroy -F ada0 ****
> > # gpart create -s gpt ada0 ****
> > # gpart add -t freebsd-zfs ada0 ****
> > # zpool create -f island raidz /dev/ada0p1 /dev/ada1p1 /dev/ada2p1 /dev/ada3p1 (this was instant)
> > island/scumm           21T    140K     21T     0% /zstorage/scumm
> >
> > This is the speed...
> > # dd if=/dev/zero bs=1024k count=1000 of=/zstorage/scumm/test.txt
> > 1048576000 bytes transferred in 0.259733 secs (4037123661 bytes/sec)
> > # dd if=/dev/zero bs=4096k count=1000 of=/zstorage/scumm/test.txt
> > 4194304000 bytes transferred in 1.534966 secs (2732505282 bytes/sec)
> > # dd if=/dev/zero bs=1024 count=1000 of=/zstorage/scumm/test.txt
> > 1024000 bytes transferred in 0.004667 secs (219403356 bytes/sec)
> >
> > Then on NetBSD, installed on the SSD i did this:
> >
> > # gpt destroy wd0 ****
> > # gpt create -Af wd0 ****
> > # gpt add -t raid -l raid5@wd0 -b $(( 2048 )) -s 15628051053 wd0 ****
> >
> > # raidctl -C /etc/raid5.conf raid5
> > # raidctl -I 13371337 raid5
> > # raidctl -iv raid5 (This took around 72+ hours)
> > # gpt create -Af raid5
> > # gpt add -a 1024 -t ffs -l island raid5
> > /dev/rraid5: Partition 1 added: 49f48d5a-b10e-11dc-b99b-0019d1879648 34 46884152860
> > # dkctl raid5 addwedge island 34 46884152860 ffs
> > # newfs -O2 -b 65536 -s -1g /dev/dk7 (trying to enable https://man.netbsd.org/wapbl.4)
> > # tunefs -l 1g /dev/dk7
> > # mount -o log /dev/dk7 /mnt
> > # umount /mnt
> > # mount /dev/dk7 /mnt/island/
> > /dev/dk7        22T   8.0K    21T   0% /mnt/island
> >
> > from /etc/fstab
> > /dev/dk7        /mnt/island     ffs     rw,noatime,log   1 1
> >
> > This one, was done before initializing the raid, on one disk
> > nas1# dd if=/dev/zero bs=1024k count=1000 of=/mnt/test.txt
> > 1048576000 bytes transferred in 2.201 secs (476408905 bytes/sec)
> >
> > Writing to the raid:
> > nas1# dd if=/dev/zero bs=1024k count=1000 of=/mnt/island/test.txt
> > 1048576000 bytes transferred in 68.984 secs (15200278 bytes/sec)
> > nas1# dd if=/dev/zero bs=4096k count=1000 of=/mnt/island/test.txt
> > 4194304000 bytes transferred in 478.951 secs (8757271 bytes/sec)
> > nas1# dd if=/dev/zero bs=1024 count=1000 of=/mnt/island/test.txt
> > 1024000 bytes transferred in 0.054 secs (18962962 bytes/sec)
> >
> > If i tried bs=512k and count=1000, i get around 1.6Billion bytes per sek write, woohoo...
> > Then try the same with count=2000 and get 1.3 million bytes per sek write
> > hmmm ....
> >
> > This is /etc/raid5.conf
> > START array
> > # numRow numCol numSpare
> > 1 4 0
> >
> > START disks
> > /dev/dk3
> > /dev/dk4
> > /dev/dk5
> > /dev/dk6
> >
> > START layout
> > # sectPerSU SUsPerParityUnit SUsPerReconUnit RAID_level_5
> > 32 1 1 5
> >
> > START queue
> > fifo 100
> >
> >
> > Summary:
> > It seems like this is 200 times slower than FreeBSD zraid, is that reasonable? or where did i go wrong?!
> > Again, only curious. Was thinking about installing OpenBSD, they have a more similar raid system compared to netbsd(more or less exactly the same:))
> >
> > Could it be worth adding a Big disk section to the raidctl manpage: https://man.netbsd.org/raidctl.8
> >
> > Thanks in advance!
> > Br Nicklas