Subject: drivers for 5380/9224 on KA410/KA43 (rather long)
To: None <port-vax@NetBSD.ORG>
From: Bertram Barth <bertram@gummo.bbb.sub.org>
List: port-vax
Date: 03/03/1998 10:02:40
Ok, there seems to be some interest in how the current drivers are
implemented and how they interact with the underlying hardware.
Maybe this will help to correct/rewrite these drivers.

First a few excuses/statements/historical facts:
- At the time these drivers were written, I had access to an 2000 and
  an 3100/76. Thus I tried to make it work on these machines.
- I started with the drivers for 2000 since I had some manuals for 
  this machine. Thanks again to those who gave it to me.
- Once the 2000 was more or less running I tried to modify the 5380
  driver in a way, it'll work for both 2000 and 3100/76. Since I don't
  have docs for 3100/76, this was mere trial and error.
- I tried to keep the door open for support of other models of 3100,
  but without access to such machines it's little more than an intention.
- I'm not at all a "hardware person", thus many details have been and
  still are unclear to me.

In the following I'll quote/refer to files in sys/arch/vax/{include,vsa}

------------------------------

On KA410 the drivers for NCR and HDC use a shared 16K buffer for DMA
(ie. data transfer). According to the KA410's manual
	- access to this buffer
	- access to the registers of NCR and/or HDC
	- NCR or HDC doing DMA operations
have to be mutually exclusive. In the meantime and according to the 
discussions on port-vax to me it looks like there are two DMA controllers
on KA410, one internal to the HDC9224 and one external somehow combined
with NCR5380. The problems arising from two DMA logics accessing the same
buffer would explain most of the KA410's limitations in DMA.

In vsbus.{h,c} there's implemented some quick-and-dirty software locking
which is mainly used to block one driver from the other. These routines 
are named "vsbus_lockDMA()" and "vsbus_unlockDMA()". Both hdc9224.c and
ncr.c also implement some internal locking, since ncr.c needs more/finer
granularity in locking.

In general each driver works like that:
1. call vsbus_lockDMA() to block the other driver
2. do internal work
3. call vsbus_unlockDMA() to release the lock

Reading data from device means:
1. setup internal registers
2. issue command to controller
3. wait for interrupt
4. read data from 16K buffer

Writing data to device means:
1. write data to 16K buffer
2. setup internal registers
3. issue command to controller
4. wait for interrupt

Since accessing the buffer has to be exclusive, I've decided that each
driver uses the complete 16K, always starting at offset zero. It would 
have been possible to reserve lower 8K for one and upper 8K for the other,
but I didn't see any real gains in that. Thus I've implemented that
"brute-force" attempt which locks the complete buffer and blocks the
other driver during the complete operation.

Since it's forbidden to access/check the registers during DMA operations,
the only way to find out if operation has completed is checking the "intreq"
flag on KA*'s interrupt register. Thus (at least in this implementation)
the only difference between "polled mode" and "DMA mode" is polling the
"intreq" flag vs. having an ISR being called. [There's an additional
"poll mode" in the MI part of the ncr-driver which is activated if the
amount of bytes to transfer is smaller than "ncr_sc->sc_min_dma_len".
But this is rather independent from the actual implementation of ncr.c]

BTW: From the docs I have it seems that HDC9224 is at the same address
     for all VAXen, same holds for NCR-1 and NCR-2, for SCSI-DMA and for
     interrupt-registers. Thus at least some parts of the existing drivers
     should be usable for other models 3100...

------------------------------

The HDC driver is written from scratch and doesn't have an ISR yet, it's
pure polled. Two local variables "haveLock" and "keepLock" are currently
used for controlling the state of locking. The main routine is named 
"hdc_command()" which ensures blocking, writes preset values into registers,
issues command, polls/waits for completion, evtl. releases lock, checks
status and returns. Most of the registers and flags are documented in
hdc9224.h, on error-free RD-disks it's rather stable. In "hdcmatch()"
it would be fairly easy (IMHO) to add support for HDC9224 on other VAXen,
what would be needed is an entry in "struct confargs ka4??_devs[]" in
vsbus.c and a real probe-routine in hdc9224.c; that should be most of it.

What's also missing from the driver is bad-block-handling. This is mainly
due to fact that my disk didn't have bad spots, so I couldn't check/test
anything. At least parts of the on-disk structure neccessary for that
are already there.

------------------------------

The NCR driver is a completely different beast, since machine-independent
support for 5380 has already been there. Thus I modified one of the existing
MD implementation (sun3) which is kind of "front-end" to the MI code.
ncr.h is an unmodified copy of the sun3 version, all modifications are
in ncr.c; as stated above it's been KA410 only at the beginning, later
modified to support KA43 also.

Most of the ideas ("si_dma_handle", "si_softc"  etc.) are stolen from the 
sun3 driver and modified to fit my imagination (at that time) of SCSI and
DMA on KA410 and KA43. ncr.c implements additional internal locking which
is intended to prevent eg. accessing 5380's registers while doing DMA etc.

si_match() is doing a brute-force attempt to check for the existence of
the chip by writing -1 at the location of ncr's registers. Since these
registers are 8-bit wide, any value "bigger" than 0xff indicates 5380
not being there.

si_set_portid() and si_attach() are rather straight-forward, same holds
for si_minphys() etc.

The ISR si_intr() shows some differences between KA410 and KA43. For KA410
the DMA counter is "correct" while for KA43 it has to be "corrected".
One way to correct the off-by-one problems with KA43 is:
   resid = *sc->sc_dcreg;
   if (resid == 1 && sc->sc_xflags) {
     debug(("correcting resid...\n"));
     resid = 0;
   }
   ntrans = dh->dh_xlen + resid;

Another way (esp. for KA43) is:
   resid = *sc->sc_dcreg;
   if (resid)
     resid |= 0xfffe0000;  /* sign-extend this 17-bit value */
   if (dh->dh_flags & SIDH_OUT) {
     /*
      * correct previous manipulations...
      */
     resid |= 0xfffe0000;  /* sign-extend this 17-bit value */
     resid += 1;           /* Account for predecrement and byte lost in pipeline */
   }
   ntrans = dh->dh_xlen + resid;

Here "ntrans" is the number of bytes actually transfered, if ntrans
is not equal to the requested size, then the well known message about
"incomplete DMA" is output. The reasons for the differences are still 
unknown to me.

following in ncr.c are some routines which handle locking wrt. DMA
and which are handed over to the MI code which in turn uses them
to prevent accessing registers/buffer etc. against each other.
I suspect many problems arising from that and/or from incompatibilities
in the way MI code expects them to work and the actual implementation.

BTW: Througout the complete file comments wrt. sun3/VME/etc. are at
     least in parts misleading; I've kept them to keep the similarity
     in the structure of the file visible. Eg. "VME functions for DMA"
     is just a text-mark when comparing ncr.c for vax/sun3. What's
     really done in these sections is completely different for sun3/vax.

si_dma_setup() is NOP for KA410/KA43 since there's no need to reserve
space for DMA, the DMA buffer is already there.

si_dma_start() initializes DMA-counter from "xlen" (size to transfer);
By trial and error I found that above error messages vanish if byte-count
is calculated differently for read/write:
                *sc->sc_ddreg = DMA_DIR_OUT;
                *sc->sc_dcreg = 0 - (xlen + 2); /* bertram XXX */
vs.
                *sc->sc_ddreg = DMA_DIR_IN;
                *sc->sc_dcreg = 0 - xlen; /* bertram XXX */
Don't ask me why this seems to work that way.
Writing into the send/recv register starts the transfer:
                *ncr_sc->sci_dma_send = 0;      /* start it */
vs.
                *ncr_sc->sci_irecv = 0; /* start it */

Around line 1211 there's one of the most problematic/annoying things
in that driver: It seems that there's a collision somewhere either in
the MI or MD code which causes DMA to be interrupted. Thus I've inserted
a absolute ugly busy-wait here, which "avoids" this problem; It really
slows down the driver, but I haven't found another way to prevent these
"DMA incomplete" messages all the time. [BTW: Maybe this problem is
somehow related to the watchdog or some other checks in the MI code,
but I never managed to trace down this problem; adding debug-messages
usually slows down execution enough for this problem not to appear.]

BTW: I just found out that I've two versions of ncr.c at home where
     I don't know offhand which made it into the official tree. One
     holds the additional comments:

        char    *sc_dbase;      /* base address of DMA area */
        int     sc_dsize;       /* size of DMA area, vs2000:16K vs3100:128K */

        /*
         * Some more problems/differences :-/
         *
         * VS2000 implements DMA-Count-Register (dcreg) as 16-bit register
         * while VS3100 implements this as 31-bit register (17 bit effective).
         * VS2000's DMA area is 16K, DMA Address register (dareg) is a
         * 8-bit register, offset into DMA area is written as two bytes.
         * VS3100's DMA area is 128K, DMA Address register is a 32-bit
         * register, offset into DMA area is written as one longword.
         * (Since currently we're using offset 0x00 only, it somehow works)
         */

Also one file holds "sc->sc_xflags" which indicates KA410 or KA43 and
is used for correcting the off-by-one problem in the DMA-counter. This
flag is set to 1 for KA43 and set to 0 for KA410. The corresponding lines
read:
                resid = *sc->sc_dcreg;
                if (resid == 1 && sc->sc_xflags) {
                  debug(("correcting resid...\n"));
                  resid = 0;
                }

------------------------------

While re-reading the above stuff I realized that it hasn't become as
technical as I originally intended. Anyway I hope it made some details
clear. If there are more detailed/technical questions I'm willing to
answer [as/if time permits] them here in the mailing list or (if others 
think it's getting too off-topic) in private eMail.

Ciao,
	bertram

PS: In the meantime I think it could have been better to have to different
    implementations for NCR-drivers, one for KA410 and one for KA43. But 
    due to lack of spare time I never tried that. Also I have to say that
    because of my lack of knowledge/experience/time I can't rewrite these
    drivers esp. since I don't have my 2000 anymore. If I can be of any
    help to the one rewriting these drivers I'm willing to help.