Subject: Re: Performance of SCSI writing.
To: None <port-pmax@NetBSD.ORG>
From: Michael L. Hitch <mhitch@lightning.oscs.montana.edu>
List: port-pmax
Date: 06/07/1997 09:54:10
On Jun  6,  1:56pm, Simon Burge wrote:
> On Thu, 5 Jun 1997 10:47:22 -0600  Michael L. Hitch wrote:
> 
> > On Jun  5,  6:57pm, Takuya Koumoto wrote:
> > > an RZ25(400MB) HDD. The iozone gives the following results
> > > on 100MB sequential I/O test. Thanks to Mr. Nishimura, the
> > > old kernel(2 months ago) gave 1.6MB/sec for writing. Is that
> > > a known problem?
> > > 
> > >         347844 bytes/second for writing the file
> > >         1360373 bytes/second for reading the file
...
> > Hmm - on my DS5000/25 and an RZ25 (but using a 99MB file), I get:
> > 
> >         1255855 bytes/second for writing the file
> >         1337400 bytes/second for reading the file
> > 
> > This is with a kernel built from sources around May 29 (and I don't know of
> > any significant changes since then).  This was with the DS5000_25 config
> > file with the DEBUG option enabled.
...
> As another data point, I've got a 5000/240 with a RZ26N, and I get these
> figures with IOZONE:
> 
> 	393461 bytes/second for writing the file
> 	1757389 bytes/second for reading the file

  This is getting very, very strange.

  Compiling without DEBUG gives me the reduced writing rate, compiling with
DEBUG triples the writing rate.  Further investigation showed that compiling
only pmax/machdep.c with DEBUG results in the faster writing rate.  Going
even  deeper, it looks like one small section of data makes the difference.
Removing the "#ifdef DEBUG" around the signal debug data in machdep.c seems
to make the difference.

  How and why including two initialized variables and a few #defines would
make a difference in the writing speed is (at the moment) a big mystery to
me.

> similar box (5000/240, 32MB RAM, 6.4MB buffer cache, RZ26N) running
> Ultrix 4.4, I get these results for IOZONE:
> 
> 	1853201 bytes/second for writing the file
> 	2612520 bytes/second for reading the file
...
> Seems the SCSI driver isn't as fast as it could be...

  I can think of a couple of things contributing to this.  The first one is
that the NetBSD driver currently allocates a "DMA buffer" for the IOASIC
SCSI.  All data transfers have to be copied to/from this buffer.  The IOASIC
SCSI is capable of doing DMA directly to memory, but the NetBSD driver does
not currently utilize this capability.  The second reason is related to
copying data to/from the DMA buffer memory:  the DMA buffer memory is in
uncached memory space.  Utilizing cached memory for the DMA buffer seems
to help the iozone reading rate a bit, presumably because the data copies
are a little quicker.  I've been running my kernels with cached memory
with no apparent ill effects for a while now.  [I had done this while getting
the R4000 mods working].  If anyone wants to test this, just make the
following change in sys/dev/tc/asc_ioasic.c:

--- /c/src/sys/dev/tc/asc_ioasic.c	Sun May 25 05:23:22 1997
+++ ./asc_ioasic.c	Fri Jun  6 05:38:45 1997
@@ -104,7 +104,7 @@
 	 * (2) timing based on turbochannel frequency
 	 */
 
-	asc->buff = (u_char *)MACH_PHYS_TO_UNCACHED(asc_iomem);
+	asc->buff = (u_char *)MACH_PHYS_TO_CACHED(asc_iomem);
 	bufsiz = 8192;
 	*((volatile int *)IOASIC_REG_SCSI_DMAPTR(ioasic_base)) = -1;
 	*((volatile int *)IOASIC_REG_SCSI_DMANPTR(ioasic_base)) = -1;

Michael

-- 
Michael L. Hitch			mhitch@montana.edu
Computer Consultant
Information Technology Center
Montana State University	Bozeman, MT	USA