Subject: Re: Massive read/write performance problems with 3.0
To: Tonnerre LOMBARD <tonnerre@bsdprojects.net>
From: Manuel Bouyer <bouyer@antioche.eu.org>
List: port-amd64
Date: 05/09/2006 12:06:42
On Tue, May 09, 2006 at 07:31:35AM +0200, Tonnerre LOMBARD wrote:
> Salut,
> 
> On Mon, May 08, 2006 at 01:23:22PM +0200, Manuel Bouyer wrote:
> > Can you try the attached program ? It will benchmark the IDE bus's bandwidth:
> > ./tst /dev/rwd0d 10000
> > 
> > it should run for a few seconds to give usefull results; if it doesn't
> > use something larger than 10000
> 
> I added the O_SYNC parameter to your open() and it sanitized the output
> quite a bit:

As the tool is using the raw device it shouldn't matter. What problem did
you notice ?

> 
> vic# ./tst /dev/rwd0d 10000                                                     
> 5409749 us, 115.532162 MB/s                                                     
> vic# 
> 
> Bonnie gives the following output:
> 
>     -------Sequential Output-------- ---Sequential Input-- --Random--
>     -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
>  MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
> 256  4802  3.5  5217  1.8  6430  1.8 105399 75.6 627604 100.0 5970.5 31.6
> 
> The worst thing is that sometimes disk I/O operations hang for quite a
> while during heavy disk I/O (especially when the I/O takes place from two
> or more programs, so it seems that the I/O bandwidth isn't really
> distributed equally). I might bring up more test cases on that tonight.
> 
> It looks to me that either the VM can't stand much parallel I/O or that
> there is a speed issue in the file system layer.

I don't have such issues with others disk subsystems:
tango# /tmp/tst /dev/rwd0d 10000
5175742 us, 120.755633 MB/s
tango# bonnie -c 256
    -------Sequential Output-------- ---Sequential Input-- --Random--
    -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
 MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
256 43852 28.9 42751  8.2 28485  6.5 60864 42.9 1057795 100.0 34763.0 71.0
The same with -s 1024 (the box has 512Mb RAM):
     -------Sequential Output-------- ---Sequential Input-- --Random--
     -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
  MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
1024 46547 32.0 46896  8.8 20691  4.3 44786 32.4 47385  5.7 146.6  0.5

So the problem is probably with either the disk or the controller.
It seems that the controller can read from disk at full speed, so
it's probably setup properly, at last for communications with the disk.
You don't have the disk's write cache turned off, do you ?

BTW, you could also try to benchmark raw write speed: boot single user and
use:
dd if=/dev/zero of=/dev/rwd0b bs=64k (assuming your swap partition is wd0b)

-- 
Manuel Bouyer <bouyer@antioche.eu.org>
     NetBSD: 26 ans d'experience feront toujours la difference
--