Re: Help with low raid5 performance

[Juergen Hannken-Illjes <hannken%eis.cs.tu-bs.de@localhost>]

On Tue, Nov 09, 2010 at 04:38:59PM +0100, Peter Kerwien wrote:
> Hi,
> 
> I'm testing a RAID-5 configuration with RAIDframe, but cannot get 
> decent performance. I have read the following thread:
> 
> http://mail-index.netbsd.org/netbsd-users/2010/04/09/msg006043.html
> 
> But cannot understand what I'm doing wrong. I'm experimenting with 3 x 
> WD 500GB RE2 drives, which give approx. 55-60MB/s when using them as 
> separate disks (measured with iostat during a file copy to the drive). 
> But when setting up a RAID-5 configuration with all three drives, I'm 
> seeing something between 10-20 MB/s. Never above 20MB/s. The latest 
> setup I have tried is the following:
> 
> Raid configuration:
> 
> START array
> 1 3 0
> 
> START disks
> /dev/wd1e
> /dev/wd2e
> /dev/wd3e
> 
> START layout
> 64 1 1 5
> 
> START queue
> fifo 100
> 
> So what I understand gives a stripe size of 64 sectors = 32kBytes. So 
> now it is important that the partition on raid0 (my raid-5 device) is 
> aligned with this. I have then created the following gpt:
> 
> # gpt show raid0
>        start        size  index  contents
>            0           1         PMBR
>            1           1         Pri GPT header
>            2          32         Pri GPT table
>           34          30
>           64  1953546015      1  GPT part - NetBSD UFS/UFS2
>   1953546079          32         Sec GPT table
>   1953546111           1         Sec GPT header
> 
> and the corresponding wedge:
> 
> # dkctl raid0 addwedge dk0 64 1953546015 raid0
> 
> and finally a filesystem with:
> 
> # newfs -O2 -s -64m /dev/rdk0
> 
> (In my case: block size = 16kB => fragment size = 2kB).
> 
> What important thing have I missed or done wrong??

You should try "block size == stripe size" to avoid read-modify-write.
What throughput do you get from "newfs -b 32k ..."?

-- 
Juergen Hannken-Illjes - hannken%eis.cs.tu-bs.de@localhost - TU Braunschweig 
(Germany)