Re: Scsipi guru needed...

[Manuel Bouyer <bouyer%antioche.eu.org@localhost>]

On Sun, Nov 30, 2008 at 07:41:05PM +0100, Anders Magnusson wrote:
> Hi,
> 
> I have been trying to find out why the performance of the ciss 
> controller (Compaq raid) in
> DL38x is really bad, but as I don't understand anything about the scsipi
> layer (are there any docs?)

there is scsipi(9).

> I must ask around.  All this is really strange.
> 
> Writing data to the raw sd device (bs=64k) gets just over 7MB/s, which 
> is quite slow.
> Some debugging showed that:
> - A write transfer (from given to the ctlr to interrupt) takes 8ms.  Ok, 
> no write cache on the ctlr maybe?

So, with 64k every 8ms, the max write speed would be 8MB/s when using
the raw device.

> - Only one (1) active transfer at a time, even though the ctlr can have 
> 1024.

Note that when using the raw device, you won't have more than one command
submitted to sd(4) at a time. 

> 
> Looking in sdstart it seems that periph_openings is 1, which means only 
> one at a time.
> Increasing it to 40 (using ddb) makes the machine a little faster, but 
> not much.
> 
> The config stuff in ciss.c to setup this seems correct to me, but 
> someone that understands
> it may have some ideas...? 

One thing looks wrong in ciss:
sc->sc_adapter.adapt_max_periph = sc->maxunits;
adapt_max_periph is "number of commands the adapter can handle per device".
So if the adater can handle 1024 commands per unit, it should probably be
sc->sc_adapter.adapt_max_periph = sc->sc_adapter.adapt_openings

Also our scsipi layer supports only 256 tag per device, and it's not
clamped by the scsipi layer si it should probably really be
sc->sc_adapter.adapt_max_periph = min(sc->sc_adapter.adapt_openings, 256)

> It should be noted that FreeBSD works very well with this controller, 
> while NetBSD doesn't :-/

Do you know what MAXPHYS does FreeBSD use ? 64k is very low for modern
disk systems. I think the right thing for us would be to kill MAXPHYS
and use per-device MAXPHYS, but it's not trivial.

-- 
Manuel Bouyer <bouyer%antioche.eu.org@localhost>
     NetBSD: 26 ans d'experience feront toujours la difference
--