[ On Tuesday, May 6, 2003 at 13:09:48 (-0400), I wrote: ]
> Subject: Re: some performance diff with getc()/putc() between FreeBSD and NetBSD?
>
> [ On Tuesday, May 6, 2003 at 22:41:08 (+1000), Paul Ripke wrote: ]
> > Subject: Re: some performance diff with getc()/putc() between FreeBSD and NetBSD?
> >
> > OK, what concerns me is those "random seeks/sec" rates. There's a huge
> > difference there, and that could hurt other stats. It shouldn't be CPU
> > bound (obviously), the only thing that comes to mind is in disk
> > I/O scheduling - disksort. I wonder what the difference is.
> 
> Yes, that concerns me a bit too.
> 
> I'm not sure how to quantify the difference in seeks/sec though.

I'm not sure if I've discovered something even worse, or just an
artifact in the way bonnie's measurements work.

Observe what happens when I more than double the size of the file:

              -------Sequential Output-------- ---Sequential Input-- --Random--
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU

NBSD-ATA 3000 41865 34.7 40128 13.2 10464  2.8 40413 34.4 40832  6.0  93.9  0.5
NBSD-ATA 8192 39055 29.5 33559 11.5  9421  2.6 33608 27.7 33738  5.0 427.4  0.9

NBSDRF10 3000 78534 66.6 78926 26.5 13895  5.9 87662 81.2 119517 26.2 276.9  2.9
NBSDRF10 8192 77872 61.2 78070 26.0 13961  6.0 92764 83.9 122623 27.3 1014.1  3.0

Note these measurements are all taken with a block size of 16384 bytes,
the default for Netbsd pkgsrc/benchmarks/bonnie prior to my patches
adding a chunk-size command-line option.  FreeBSD ports/benchmarks/bonnie
used a default of 8192 bytes.  However reducing that size to 8192 on
NetBSD does not significantly affect the seeks/sec nor the rewrite
throughput in any way.

While the rewrite throughput doesn't change, the seeks/sec improves very
considerably.....  On first glance this seems very counter-intuitive.
Normally the bigger the file the closer the seek time approaches the
actual seek time of the underlying hardware.  While that may be exactly
what's happening here, it's still odd because it's essentially backwards
here since the smaller the file then the longer the average seek time.

One more different test may help.  My kernel boots saying:

	total memory = 1023 MB
	avail memory = 943 MB
	using 6144 buffers containing 52508 KB of memory

Let's see what happens with smaller files (these on the RAID-0+1):

              -------Sequential Output-------- ---Sequential Input-- --Random--
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
All-Cached 52 98690 84.1 114556 37.8 88082 28.9 109563 94.4 721331 100.0 17050.3 51.6
          100 94715 80.7 94866 30.7 79674 26.6 113946 96.9 720360 100.0 19444.0 63.3
         1000 78164 66.8 79449 26.9 13869  5.9 83543 77.7 108555 23.8 1241.2  6.4

Hmmm, well it seems there may be more than one "knee" in the graph --
I'm just not quite sure where they are yet.....  If I had more time and
energy for this I'd script up a series of tests to graph the full curve,
but I guess that doesn't have to be done immediately on this very
machine I've been testing on.

(hmmm.... there's also more evidence here of the getc() peak rate, but
perhaps it's higher than I thought....)

Perhaps, in combination with the bizzare slow-down in rewrite that has
been apparently caused by the unified buffer cache, these results help
confirm my notion that the unified buffer cache is sometimes much better
in NetBSD than whatever buffer caching FreeBSD has, and yet obviously
for some types of jobs much worse.  Or maybe not.  :-)

-- 
								Greg A. Woods

+1 416 218-0098;            <g.a.woods@ieee.org>;           <woods@robohack.ca>
Planix, Inc. <woods@planix.com>; VE3TCP; Secrets of the Weird <woods@weird.com>