Re: Scsipi guru needed...

[Anders Magnusson <ragge%ludd.ltu.se@localhost>]

Manuel Bouyer wrote:
> On Sun, Nov 30, 2008 at 07:41:05PM +0100, Anders Magnusson wrote:
>> Hi,
>>
>> I have been trying to find out why the performance of the ciss 
>> controller (Compaq raid) in
>> DL38x is really bad, but as I don't understand anything about the scsipi
>> layer (are there any docs?)
> 
> there is scsipi(9).
> 
Yes, but I still got lost in all terminology.  Better to ask :-)

>> I must ask around.  All this is really strange.
>>
>> Writing data to the raw sd device (bs=64k) gets just over 7MB/s, which 
>> is quite slow.
>> Some debugging showed that:
>> - A write transfer (from given to the ctlr to interrupt) takes 8ms.  Ok, 
>> no write cache on the ctlr maybe?
> 
> So, with 64k every 8ms, the max write speed would be 8MB/s when using
> the raw device.
> 
Exactly, that is as expected.

>> - Only one (1) active transfer at a time, even though the ctlr can have 
>> 1024.
> 
> Note that when using the raw device, you won't have more than one command
> submitted to sd(4) at a time. 
> 
Of course. I should have mentioned that this is a common behavior, not when
using the raw device.


>> Looking in sdstart it seems that periph_openings is 1, which means only 
>> one at a time.
>> Increasing it to 40 (using ddb) makes the machine a little faster, but 
>> not much.
>>
>> The config stuff in ciss.c to setup this seems correct to me, but 
>> someone that understands
>> it may have some ideas...? 
> 
> One thing looks wrong in ciss:
> sc->sc_adapter.adapt_max_periph = sc->maxunits;
> adapt_max_periph is "number of commands the adapter can handle per device".
> So if the adater can handle 1024 commands per unit, it should probably be
> sc->sc_adapter.adapt_max_periph = sc->sc_adapter.adapt_openings
> 
> Also our scsipi layer supports only 256 tag per device, and it's not
> clamped by the scsipi layer si it should probably really be
> sc->sc_adapter.adapt_max_periph = min(sc->sc_adapter.adapt_openings, 256)
> 
Ok, thanks, I try that tomorrow.

>> It should be noted that FreeBSD works very well with this controller, 
>> while NetBSD doesn't :-/
> 
> Do you know what MAXPHYS does FreeBSD use ? 64k is very low for modern
> disk systems. I think the right thing for us would be to kill MAXPHYS
> and use per-device MAXPHYS, but it's not trivial.
> 
No, I don't know, but it is far more than 64k.  But that doesn't matter;
using bs=64k when dd:ing to the raw device gives much better performance
anyway.  Besides,  MAXPHYS should be fixed anyway.


-- Ragge