Re: NetBSD Raid5, slow write speeds, using big disks?!

["J. Hannken-Illjes" <hannken%mailbox.org@localhost>]

Others will know better, but I always thought a Raid5 with four disks spends most time with read-modify-write cycles as usually no requests will fit the stripe size.

Did you try ZFS/raidz on NetBSD?

--
J. Hannken-Illjes - hannken%mailbox.org@localhost



> On 20. Jun 2026, at 18:55, smurfd <smurf.daemon%mail.smurfd.me@localhost> wrote:
> 
> Hey,
> 
> This will be a long one, sorry for this.
> I have created a RAID5 NAS using NetBSD 10.1, and im experiencing slow writing speeds. Or is my expectations all wrong? :)
> I earlier had FreeBSD and ZFS zraid on the same Ugreen DXP4800 plus (https://nas-eu.ugreen.com/products/ugreen-nasync-dxp4800-plus-nas-storage)
> The NAS drives are 4x7TB, less than a year old Segate Ironwolf NAS 3.5".
> NetBSD is installed first on the SSD, using a base install. Then the Raid is created manually...
> 
> Have asked here:
> https://www.unitedbsd.com/d/1646-raid5-slow-write-on-big-disks/8
> Just now redid the raid, for the Xth time, to compare the speeds i got using FreeBSD (because i didnt remember/had that written down)
> I know, comparing zraid vs raid5 is like comparing apples to oranges, right?...
> 
> What write speeds should i expect?! is probably the most important question...
> Or where did i go wrong, because user error is VERY likely.
> 
> First, i tried to follow the Summary section in the https://man.netbsd.org/raidctl.8, realizing that Disklabel supports no larger than 2TB disks.
> So found this: https://wiki.netbsd.org/users/mlelstv/using-large-disks/ and tried to follow the Raidframe section.
> Then i got the correct size i expected...
> 
> The speed though, is slow! (again compared to zfs zraid)
> When i had FreeBSD installed, i did the following. Installed the system to the SSD.
> Then, did this, where **** is repeaded 4 times for the disks...
> # gpart destroy -F ada0 ****
> # gpart create -s gpt ada0 ****
> # gpart add -t freebsd-zfs ada0 ****
> # zpool create -f island raidz /dev/ada0p1 /dev/ada1p1 /dev/ada2p1 /dev/ada3p1 (this was instant)
> island/scumm           21T    140K     21T     0% /zstorage/scumm
> 
> This is the speed...
> # dd if=/dev/zero bs=1024k count=1000 of=/zstorage/scumm/test.txt
> 1048576000 bytes transferred in 0.259733 secs (4037123661 bytes/sec)
> # dd if=/dev/zero bs=4096k count=1000 of=/zstorage/scumm/test.txt
> 4194304000 bytes transferred in 1.534966 secs (2732505282 bytes/sec)
> # dd if=/dev/zero bs=1024 count=1000 of=/zstorage/scumm/test.txt
> 1024000 bytes transferred in 0.004667 secs (219403356 bytes/sec)
> 
> Then on NetBSD, installed on the SSD i did this:
> 
> # gpt destroy wd0 ****
> # gpt create -Af wd0 ****
> # gpt add -t raid -l raid5@wd0 -b $(( 2048 )) -s 15628051053 wd0 ****
> 
> # raidctl -C /etc/raid5.conf raid5
> # raidctl -I 13371337 raid5
> # raidctl -iv raid5 (This took around 72+ hours)
> # gpt create -Af raid5
> # gpt add -a 1024 -t ffs -l island raid5
> /dev/rraid5: Partition 1 added: 49f48d5a-b10e-11dc-b99b-0019d1879648 34 46884152860
> # dkctl raid5 addwedge island 34 46884152860 ffs
> # newfs -O2 -b 65536 -s -1g /dev/dk7 (trying to enable https://man.netbsd.org/wapbl.4)
> # tunefs -l 1g /dev/dk7
> # mount -o log /dev/dk7 /mnt
> # umount /mnt
> # mount /dev/dk7 /mnt/island/
> /dev/dk7        22T   8.0K    21T   0% /mnt/island
> 
> from /etc/fstab
> /dev/dk7        /mnt/island     ffs     rw,noatime,log   1 1
> 
> This one, was done before initializing the raid, on one disk
> nas1# dd if=/dev/zero bs=1024k count=1000 of=/mnt/test.txt
> 1048576000 bytes transferred in 2.201 secs (476408905 bytes/sec)
> 
> Writing to the raid:
> nas1# dd if=/dev/zero bs=1024k count=1000 of=/mnt/island/test.txt
> 1048576000 bytes transferred in 68.984 secs (15200278 bytes/sec)
> nas1# dd if=/dev/zero bs=4096k count=1000 of=/mnt/island/test.txt
> 4194304000 bytes transferred in 478.951 secs (8757271 bytes/sec)
> nas1# dd if=/dev/zero bs=1024 count=1000 of=/mnt/island/test.txt
> 1024000 bytes transferred in 0.054 secs (18962962 bytes/sec)
> 
> If i tried bs=512k and count=1000, i get around 1.6Billion bytes per sek write, woohoo...
> Then try the same with count=2000 and get 1.3 million bytes per sek write
> hmmm ....
> 
> This is /etc/raid5.conf
> START array
> # numRow numCol numSpare
> 1 4 0
> 
> START disks
> /dev/dk3
> /dev/dk4
> /dev/dk5
> /dev/dk6
> 
> START layout
> # sectPerSU SUsPerParityUnit SUsPerReconUnit RAID_level_5
> 32 1 1 5
> 
> START queue
> fifo 100
> 
> 
> Summary:
> It seems like this is 200 times slower than FreeBSD zraid, is that reasonable? or where did i go wrong?!
> Again, only curious. Was thinking about installing OpenBSD, they have a more similar raid system compared to netbsd(more or less exactly the same:))
> 
> Could it be worth adding a Big disk section to the raidctl manpage: https://man.netbsd.org/raidctl.8
> 
> Thanks in advance!
> Br Nicklas
>