Subject: Re: Performance of SCSI writing.
To: Michael L. Hitch <mhitch@lightning.oscs.montana.edu>
From: Simon Burge <simonb@telstra.com.au>
List: port-pmax
Date: 06/10/1997 12:54:17
On Sat, 7 Jun 1997 09:54:10 -0600  Michael L. Hitch wrote:

>   I can think of a couple of things contributing to this.  The first one is
> that the NetBSD driver currently allocates a "DMA buffer" for the IOASIC
> SCSI.  All data transfers have to be copied to/from this buffer.  The IOASIC
> SCSI is capable of doing DMA directly to memory, but the NetBSD driver does
> not currently utilize this capability.  The second reason is related to
> copying data to/from the DMA buffer memory:  the DMA buffer memory is in
> uncached memory space.  Utilizing cached memory for the DMA buffer seems
> to help the iozone reading rate a bit, presumably because the data copies
> are a little quicker.  I've been running my kernels with cached memory
> with no apparent ill effects for a while now.  [I had done this while getting
> the R4000 mods working].  If anyone wants to test this, just make the
> following change in sys/dev/tc/asc_ioasic.c:
> 
> --- /c/src/sys/dev/tc/asc_ioasic.c	Sun May 25 05:23:22 1997
> +++ ./asc_ioasic.c	Fri Jun  6 05:38:45 1997
> @@ -104,7 +104,7 @@
>  	 * (2) timing based on turbochannel frequency
>  	 */
>  
> -	asc->buff = (u_char *)MACH_PHYS_TO_UNCACHED(asc_iomem);
> +	asc->buff = (u_char *)MACH_PHYS_TO_CACHED(asc_iomem);
>  	bufsiz = 8192;
>  	*((volatile int *)IOASIC_REG_SCSI_DMAPTR(ioasic_base)) = -1;
>  	*((volatile int *)IOASIC_REG_SCSI_DMANPTR(ioasic_base)) = -1;

I've done a bit more testing (on the 5900 with a RZ26), and here's the
results:

"standard" kernel:
	IOZONE performance measurements:
		392431 bytes/second for writing the file
		1733183 bytes/second for reading the file

	bonnie:
	      -------Sequential Output-------- ---Sequential Input-- --Random--
	      -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
	   MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
	   20   519 33.8   519  5.6   416  6.4  1027 98.2  1767 16.3  58.1  4.6

kernel with Michael's one line patch:

	IOZONE performance measurements:
		393388 bytes/second for writing the file
		2032124 bytes/second for reading the file

	bonnie:
	      -------Sequential Output-------- ---Sequential Input-- --Random--
	      -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
	   MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
	   20   516 33.5   519  5.1   428  6.1  1308 96.9  2021 13.9  63.0  5.0

kernel with Michael's one line patch and "options DEBUG":

	IOZONE performance measurements:
		1593580 bytes/second for writing the file
		1908236 bytes/second for reading the file

	bonnie:
	      -------Sequential Output-------- ---Sequential Input-- --Random--
	      -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
	   MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
	   20  1439 97.1  1491 13.2   890 13.2  1298 95.7  2018 13.7  63.0  5.1


So Michael's patch give an approx 15% read speed improvement, and
"options DEBUG" gives about 4x write speed improvement.

This is still slower than Ultrix (2MB/s writing, 2.6MB/s reading), but a
lot better than before.

Simon.