On Wed, 4 Aug 2010, Michael L. Hitch wrote:
It appears that PERIPH_UNTAG gets set in periph_flags, and scsipi_run_queue() only sends one command at a time to the adapter. If I comment out the check for PERIPH_UNTAG, then I see the adapter able to accept the full number of commands. I'm in the process of trying to figure out why PERIPH_UNTAG is being set. The inquiry flags3 does show CmdQue set, and sd.c appears to set XS_CTL_SIMPLE_TAG.
I think I have figured out what is happening here. The results of the inquiry command does have CmdQue set in flags3, but the version field is set to 0, and the device probed only checks the flags3 bits if the version is >= 2. The capabilities of the device don't get the tagged queuing, and all commands are untagged. I'm trying to figure out a clean way to enable tagged queueing in ciss(4). I've got a somewhat nasty hack at the moment to set periph_cap to PERIPH_CAP_TQING, which works.
[There's a thread on port-amd64 about slow disk writes on a Dell R310 using mpt(4), which I suspect may have the same type of problem.]
No, I don't know, but it is far more than 64k. But that doesn't matter; using bs=64k when dd:ing to the raw device gives much better performance anyway. Besides, MAXPHYS should be fixed anyway.Does FreeBSD write directly to the device when writing to the raw device? While testing with a Knoppix Live CD, I noticed that copying /dev/zero to the disk resulted in around 150 commands queued to the ciss adapter, and many pdflush processes were running. I'm guessing Linux may buffer the output in the disk cache and pdflush writes them out to the physical drive asynchronously. It does seem to keep up a good transfer rate, so how that is working with no write cache on the controller, I don't really know. I'll probably try my FreeBSD Live CD sometime again with this DL360 and see how it does.
With my FreeSBIE Live CD, I only see a /dev/da0 and /dev/da1 for the two drives I had configured. Doing a dd to /dev/da1 (an empty disk I was using for testing), I only got about 9MB/sec with 64k blocks. Increasing to 128K or higher blocks gives me an increase in write speed, but less than double that. An interesting note is that with a hacked ciss(4) to enable tagged queing, I get the 9MB/sec to /dev/rsd1d with 64k, and increases in speed as I double the block size up to about 1MB. Above that, the speed stays the same. NetBSD appears to submit multiple 64K byte writes, up to 16 commands, increasing the write speeds.
-- Michael L. Hitch mhitch%montana.edu@localhost Computer Consultant Information Technology Center Montana State University Bozeman, MT USA
-- Michael L. Hitch mhitch%montana.edu@localhost Computer Consultant Information Technology Center Montana State University Bozeman, MT USA