On Sun, 30 Nov 2008, Anders Magnusson wrote:
Manuel Bouyer wrote:On Sun, Nov 30, 2008 at 07:41:05PM +0100, Anders Magnusson wrote:Looking in sdstart it seems that periph_openings is 1, which means only one at a time. Increasing it to 40 (using ddb) makes the machine a little faster, but not much. The config stuff in ciss.c to setup this seems correct to me, but someone that understands it may have some ideas...?One thing looks wrong in ciss: sc->sc_adapter.adapt_max_periph = sc->maxunits; adapt_max_periph is "number of commands the adapter can handle per device". So if the adater can handle 1024 commands per unit, it should probably be sc->sc_adapter.adapt_max_periph = sc->sc_adapter.adapt_openings Also our scsipi layer supports only 256 tag per device, and it's not clamped by the scsipi layer si it should probably really be sc->sc_adapter.adapt_max_periph = min(sc->sc_adapter.adapt_openings, 256)Ok, thanks, I try that tomorrow.
Manuel is correct about the sc->sc_adapter.adapt_max_periph being incorrect. This was very likely due to the original port of the OpenBSD driver and neither Tonnerre Lombard nor I understood the adapter/channel configuration stuff.
I recently acquired a DL360 G4p with the Smart Array 6i controller with no battery backup, and started investigating this once again. Even with the adapter settings changed to more reasonable values, I was only seeing one command queued on the ciss adapter at one time. After several iterations of adding stats at the sd and scsipi layers, I have finally figured out why this was.
It appears that PERIPH_UNTAG gets set in periph_flags, and scsipi_run_queue() only sends one command at a time to the adapter. If I comment out the check for PERIPH_UNTAG, then I see the adapter able to accept the full number of commands. I'm in the process of trying to figure out why PERIPH_UNTAG is being set. The inquiry flags3 does show CmdQue set, and sd.c appears to set XS_CTL_SIMPLE_TAG.
It should be noted that FreeBSD works very well with this controller, while NetBSD doesn't :-/
I did some testing of FreeBSD vs NetBSD a little while ago (relating the the thread about untarring pkgsrc on FreeBSD being twice as fast as NetBSD). It appeared to me that the reason for the speed increase was that FreeBSD only did 1/2 the number of disk writes as NetBSD. Mounting the filesystem with the log option made NetBSD speed much better. (That was with an older version of FreeBSD.) With my patch above, the pkgsrc untar on NetBSD is much speedier (about 3 times, I think it was).
No, I don't know, but it is far more than 64k. But that doesn't matter; using bs=64k when dd:ing to the raw device gives much better performance anyway. Besides, MAXPHYS should be fixed anyway.
Does FreeBSD write directly to the device when writing to the raw device? While testing with a Knoppix Live CD, I noticed that copying /dev/zero to the disk resulted in around 150 commands queued to the ciss adapter, and many pdflush processes were running. I'm guessing Linux may buffer the output in the disk cache and pdflush writes them out to the physical drive asynchronously. It does seem to keep up a good transfer rate, so how that is working with no write cache on the controller, I don't really know. I'll probably try my FreeBSD Live CD sometime again with this DL360 and see how it does.
-- Michael L. Hitch mhitch%montana.edu@localhost Computer Consultant Information Technology Center Montana State University Bozeman, MT USA