Source-Changes-HG archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

[src/trunk]: src/sys/dev Major restructuring for swizzling to the request que...



details:   https://anonhg.NetBSD.org/src/rev/88f7ff15a466
branches:  trunk
changeset: 519251:88f7ff15a466
user:      mjacob <mjacob%NetBSD.org@localhost>
date:      Fri Dec 14 00:13:44 2001 +0000

description:
Major restructuring for swizzling to the request queue and unswizzling from
the response queue. Instead of the ad hoc ISP_SWIZZLE_REQUEST, we now have
a complete set of inline functions in isp_inline.h. Each platform is
responsible for providing just one of a set of ISP_IOX_{GET,PUT}{8,16,32}
macros.

The reason this needs to be done is that we need to have a single set of
functions that will work correctly on multiple architectures for both little
and big endian machines. It also needs to work correctly in the case that
we have the request or response queues in memory that has to be treated
specially (e.g., have ddi_dma_sync called on it for Solaris after we update
it or before we read from it).

One thing that falls out of this is that we no longer build requests in the
request queue itself. Instead, we build the request locally (e.g., on the
stack) and then as part of the swizzling operation, copy it to the request
queue entry we've allocated. I thought long and hard about whether this was
too expensive a change to make as it in a lot of cases requires an extra
copy. On balance, the flexbility is worth it. With any luck, the entry that
we build locally stays in a processor writeback cache (after all, it's only
64 bytes) so that the cost of actually flushing it to the memory area that is
the shared queue with the PCI device is not all that expensive. We may examine
this again and try to get clever in the future to try and avoid copies.

Another change that falls out of this is that MEMORYBARRIER should be taken
a lot more seriously. The macro ISP_ADD_REQUEST does a MEMORYBARRIER on the
entry being added. But there had been many other places this had been missing.
It's now very important that it be done.

For NetBSD, it does a ddi_dmamap_sync as appropriate. This gets us out of
the explicit ddi_dmamap_sync on the whole response queue that we did for SBus
cards at each interrupt.

Set things up so that platforms that cannot have an SBus don't get a lot of
the SBus code checks (dead coded out).

Additional changes:

Fix a longstanding buglet of sorts. When we get an entry via isp_getrqentry,
the iptr value that gets returned is the value we intend to eventually plug
into the ISP registers as the entry *one past* the last one we've written-
*not* the current entry we're updating. All along we've been calling sync
functions on the wrong index value. Argh. The 'fix' here is to rename all
'iptr' variables as 'nxti' to remember that this is the 'next' pointer-
not the current pointer.

Devote a single bit to mboxbsy- and set aside bits for output mbox registers
that we need to pick up- we can have at least one command which does not
have any defined output registers (MBOX_EXECUTE_FIRMWARE).

Explicitly decode GetAllNext SNS Response back *as* a GetAllNext response.
Otherwise, we won't unswizzle it correctly.

Nuke some additional __P macros.

diffstat:

 sys/dev/ic/isp.c         |   261 +++++-----
 sys/dev/ic/isp_inline.h  |  1105 +++++++++++++++++++++++++++++++++++++++++++++-
 sys/dev/ic/isp_netbsd.c  |    14 +-
 sys/dev/ic/isp_netbsd.h  |   304 +++--------
 sys/dev/ic/isp_target.c  |    80 +-
 sys/dev/ic/isp_target.h  |   217 +--------
 sys/dev/ic/isp_tpublic.h |     5 +-
 sys/dev/ic/ispmbox.h     |    39 +-
 sys/dev/ic/ispvar.h      |    54 +-
 sys/dev/pci/isp_pci.c    |   107 ++--
 sys/dev/sbus/isp_sbus.c  |   122 ++--
 11 files changed, 1566 insertions(+), 742 deletions(-)

diffs (truncated from 3407 to 300 lines):

diff -r d14946f28ee3 -r 88f7ff15a466 sys/dev/ic/isp.c
--- a/sys/dev/ic/isp.c  Fri Dec 14 00:07:55 2001 +0000
+++ b/sys/dev/ic/isp.c  Fri Dec 14 00:13:44 2001 +0000
@@ -1,4 +1,4 @@
-/* $NetBSD: isp.c,v 1.85 2001/11/13 13:14:39 lukem Exp $ */
+/* $NetBSD: isp.c,v 1.86 2001/12/14 00:13:44 mjacob Exp $ */
 /*
  * This driver, which is contained in NetBSD in the files:
  *
@@ -68,7 +68,7 @@
  */
 
 #include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: isp.c,v 1.85 2001/11/13 13:14:39 lukem Exp $");
+__KERNEL_RCSID(0, "$NetBSD: isp.c,v 1.86 2001/12/14 00:13:44 mjacob Exp $");
 
 #ifdef __NetBSD__
 #include <dev/ic/isp_netbsd.h>
@@ -140,37 +140,37 @@
 /*
  * Local function prototypes.
  */
-static int isp_parse_async __P((struct ispsoftc *, int));
-static int isp_handle_other_response
-__P((struct ispsoftc *, ispstatusreq_t *, u_int16_t *));
-static void isp_parse_status
-__P((struct ispsoftc *, ispstatusreq_t *, XS_T *));
-static void isp_fastpost_complete __P((struct ispsoftc *, u_int16_t));
-static void isp_scsi_init __P((struct ispsoftc *));
-static void isp_scsi_channel_init __P((struct ispsoftc *, int));
-static void isp_fibre_init __P((struct ispsoftc *));
-static void isp_mark_getpdb_all __P((struct ispsoftc *));
-static int isp_getmap __P((struct ispsoftc *, fcpos_map_t *));
-static int isp_getpdb __P((struct ispsoftc *, int, isp_pdb_t *));
-static u_int64_t isp_get_portname __P((struct ispsoftc *, int, int));
-static int isp_fclink_test __P((struct ispsoftc *, int));
-static char *isp2100_fw_statename __P((int));
-static int isp_pdb_sync __P((struct ispsoftc *));
-static int isp_scan_loop __P((struct ispsoftc *));
-static int isp_scan_fabric __P((struct ispsoftc *));
-static void isp_register_fc4_type __P((struct ispsoftc *));
-static void isp_fw_state __P((struct ispsoftc *));
-static void isp_mboxcmd __P((struct ispsoftc *, mbreg_t *, int));
-
-static void isp_update __P((struct ispsoftc *));
-static void isp_update_bus __P((struct ispsoftc *, int));
-static void isp_setdfltparm __P((struct ispsoftc *, int));
-static int isp_read_nvram __P((struct ispsoftc *));
-static void isp_rdnvram_word __P((struct ispsoftc *, int, u_int16_t *));
-static void isp_parse_nvram_1020 __P((struct ispsoftc *, u_int8_t *));
-static void isp_parse_nvram_1080 __P((struct ispsoftc *, int, u_int8_t *));
-static void isp_parse_nvram_12160 __P((struct ispsoftc *, int, u_int8_t *));
-static void isp_parse_nvram_2100 __P((struct ispsoftc *, u_int8_t *));
+static int isp_parse_async(struct ispsoftc *, int);
+static int isp_handle_other_response(struct ispsoftc *, int, isphdr_t *,
+    u_int16_t *);
+static void
+isp_parse_status(struct ispsoftc *, ispstatusreq_t *, XS_T *);
+static void isp_fastpost_complete(struct ispsoftc *, u_int16_t);
+static void isp_scsi_init(struct ispsoftc *);
+static void isp_scsi_channel_init(struct ispsoftc *, int);
+static void isp_fibre_init(struct ispsoftc *);
+static void isp_mark_getpdb_all(struct ispsoftc *);
+static int isp_getmap(struct ispsoftc *, fcpos_map_t *);
+static int isp_getpdb(struct ispsoftc *, int, isp_pdb_t *);
+static u_int64_t isp_get_portname(struct ispsoftc *, int, int);
+static int isp_fclink_test(struct ispsoftc *, int);
+static char *isp2100_fw_statename(int);
+static int isp_pdb_sync(struct ispsoftc *);
+static int isp_scan_loop(struct ispsoftc *);
+static int isp_scan_fabric(struct ispsoftc *);
+static void isp_register_fc4_type(struct ispsoftc *);
+static void isp_fw_state(struct ispsoftc *);
+static void isp_mboxcmd(struct ispsoftc *, mbreg_t *, int);
+
+static void isp_update(struct ispsoftc *);
+static void isp_update_bus(struct ispsoftc *, int);
+static void isp_setdfltparm(struct ispsoftc *, int);
+static int isp_read_nvram(struct ispsoftc *);
+static void isp_rdnvram_word(struct ispsoftc *, int, u_int16_t *);
+static void isp_parse_nvram_1020(struct ispsoftc *, u_int8_t *);
+static void isp_parse_nvram_1080(struct ispsoftc *, int, u_int8_t *);
+static void isp_parse_nvram_12160(struct ispsoftc *, int, u_int8_t *);
+static void isp_parse_nvram_2100(struct ispsoftc *, u_int8_t *);
 
 /*
  * Reset Hardware.
@@ -723,7 +723,7 @@
         * major, minor, micro revisions in the mailbox registers, which
         * is really, really, annoying.
         */
-       if (isp->isp_bustype == ISP_BT_SBUS) {
+       if (ISP_SBUS_SUPPORTED && isp->isp_bustype == ISP_BT_SBUS) {
                if (dodnld) {
 #ifdef ISP_TARGET_MODE
                        isp->isp_fwrev[0] = 7;
@@ -1138,7 +1138,7 @@
 isp_fibre_init(struct ispsoftc *isp)
 {
        fcparam *fcp;
-       isp_icb_t *icbp;
+       isp_icb_t local, *icbp = &local;
        mbreg_t mbs;
        int loopid;
        u_int64_t nwwn, pwwn;
@@ -1160,9 +1160,7 @@
        }
 
        loopid = DEFAULT_LOOPID(isp);
-       icbp = (isp_icb_t *) fcp->isp_scratch;
        MEMZERO(icbp, sizeof (*icbp));
-
        icbp->icb_version = ICB_VERSION1;
 
        /*
@@ -1320,8 +1318,7 @@
        icbp->icb_respaddr[RQRSP_ADDR4863] = DMA_WD3(isp->isp_result_dma);
        isp_prt(isp, ISP_LOGDEBUG1,
            "isp_fibre_init: fwoptions 0x%x", fcp->isp_fwoptions);
-       ISP_SWIZZLE_ICB(isp, icbp);
-
+       isp_put_icb(isp, icbp, (isp_icb_t *)fcp->isp_scratch);
 
        /*
         * Init the firmware
@@ -1414,7 +1411,7 @@
        mbs.param[7] = DMA_WD2(fcp->isp_scdma);
        isp_mboxcmd(isp, &mbs, MBLOGALL & ~MBOX_COMMAND_PARAM_ERROR);
        if (mbs.param[0] == MBOX_COMMAND_COMPLETE) {
-               ISP_UNSWIZZLE_AND_COPY_PDBP(isp, pdbp, fcp->isp_scratch);
+               isp_get_pdb(isp, (isp_pdb_t *)fcp->isp_scratch, pdbp);
                return (0);
        }
        return (-1);
@@ -2295,9 +2292,6 @@
 {
        fcparam *fcp = isp->isp_param;
        u_int32_t portid, first_portid, last_portid;
-       sns_screq_t *reqp;
-       sns_scrsp_t *resp;
-       mbreg_t mbs;
        int hicap, first_portid_seen, last_port_same;
 
        if (fcp->isp_onfabric == 0) {
@@ -2305,8 +2299,7 @@
                return (0);
        }
 
-       reqp = (sns_screq_t *) fcp->isp_scratch;
-       resp = (sns_scrsp_t *) (&((char *)fcp->isp_scratch)[0x100]);
+
        /*
         * Since Port IDs are 24 bits, we can check against having seen
         * anything yet with this value.
@@ -2316,22 +2309,26 @@
        first_portid = portid = fcp->isp_portid;
        fcp->isp_loopstate = LOOP_SCANNING_FABRIC;
 
+
        for (first_portid_seen = hicap = 0; hicap < 65535; hicap++) {
-               MEMZERO((void *) reqp, SNS_GAN_REQ_SIZE);
-               reqp->snscb_rblen = SNS_GAN_RESP_SIZE >> 1;
-               reqp->snscb_addr[RQRSP_ADDR0015] =
-                       DMA_WD0(fcp->isp_scdma + 0x100);
-               reqp->snscb_addr[RQRSP_ADDR1631] =
-                       DMA_WD1(fcp->isp_scdma + 0x100);
-               reqp->snscb_addr[RQRSP_ADDR3247] =
-                       DMA_WD2(fcp->isp_scdma + 0x100);
-               reqp->snscb_addr[RQRSP_ADDR4863] =
-                       DMA_WD3(fcp->isp_scdma + 0x100);
-               reqp->snscb_sblen = 6;
-               reqp->snscb_data[0] = SNS_GAN;
-               reqp->snscb_data[4] = portid & 0xffff;
-               reqp->snscb_data[5] = (portid >> 16) & 0xff;
-               ISP_SWIZZLE_SNS_REQ(isp, reqp);
+               mbreg_t mbs;
+               sns_screq_t *rq;
+               sns_ganrsp_t *rs0, *rs1;
+               u_int8_t sc[SNS_GAN_REQ_SIZE];
+
+               rq = (sns_screq_t *)sc;
+               MEMZERO((void *) rq, SNS_GAN_REQ_SIZE);
+               rq->snscb_rblen = SNS_GAN_RESP_SIZE >> 1;
+               rq->snscb_addr[RQRSP_ADDR0015] = DMA_WD0(fcp->isp_scdma+0x100);
+               rq->snscb_addr[RQRSP_ADDR1631] = DMA_WD1(fcp->isp_scdma+0x100);
+               rq->snscb_addr[RQRSP_ADDR3247] = DMA_WD2(fcp->isp_scdma+0x100);
+               rq->snscb_addr[RQRSP_ADDR4863] = DMA_WD3(fcp->isp_scdma+0x100);
+               rq->snscb_sblen = 6;
+               rq->snscb_data[0] = SNS_GAN;
+               rq->snscb_data[4] = portid & 0xffff;
+               rq->snscb_data[5] = (portid >> 16) & 0xff;
+               isp_put_sns_request(isp, rq, (sns_screq_t *) fcp->isp_scratch);
+               MEMORYBARRIER(isp, SYNC_SFORDEV, 0, SNS_GAN_REQ_SIZE);
                mbs.param[0] = MBOX_SEND_SNS;
                mbs.param[1] = SNS_GAN_REQ_SIZE >> 1;
                mbs.param[2] = DMA_WD1(fcp->isp_scdma);
@@ -2382,11 +2379,14 @@
                    fcp->isp_loopstate < LOOP_SCANNING_FABRIC) {
                        return (-1);
                }
-               ISP_UNSWIZZLE_SNS_RSP(isp, resp, SNS_GAN_RESP_SIZE >> 1);
-               portid = (((u_int32_t) resp->snscb_port_id[0]) << 16) |
-                   (((u_int32_t) resp->snscb_port_id[1]) << 8) |
-                   (((u_int32_t) resp->snscb_port_id[2]));
-               (void) isp_async(isp, ISPASYNC_FABRIC_DEV, resp);
+               MEMORYBARRIER(isp, SYNC_SFORCPU, 0x100, SNS_GAN_RESP_SIZE);
+               rs1 = (sns_ganrsp_t *) fcp->isp_scratch;
+               rs0 = (sns_ganrsp_t *) ((u_int8_t *)fcp->isp_scratch + 0x100);
+               isp_get_gan_response(isp, rs0, rs1);
+               portid = (((u_int32_t) rs1->snscb_port_id[0]) << 16) |
+                   (((u_int32_t) rs1->snscb_port_id[1]) << 8) |
+                   (((u_int32_t) rs1->snscb_port_id[2]));
+               (void) isp_async(isp, ISPASYNC_FABRIC_DEV, rs1);
                if (first_portid == portid) {
                        fcp->isp_loopstate = LOOP_FSCAN_DONE;
                        return (0);
@@ -2417,10 +2417,10 @@
 isp_register_fc4_type(struct ispsoftc *isp)
 {
        fcparam *fcp = isp->isp_param;
-       sns_screq_t *reqp;
+       u_int8_t local[SNS_RFT_REQ_SIZE];
+       sns_screq_t *reqp = (sns_screq_t *) local;
        mbreg_t mbs;
 
-       reqp = (sns_screq_t *) fcp->isp_scratch;
        MEMZERO((void *) reqp, SNS_RFT_REQ_SIZE);
        reqp->snscb_rblen = SNS_RFT_RESP_SIZE >> 1;
        reqp->snscb_addr[RQRSP_ADDR0015] = DMA_WD0(fcp->isp_scdma + 0x100);
@@ -2435,7 +2435,7 @@
 #if    0
        reqp->snscb_data[6] |= 20;      /* ISO/IEC 8802-2 LLC/SNAP */
 #endif
-       ISP_SWIZZLE_SNS_REQ(isp, reqp);
+       isp_put_sns_request(isp, reqp, (sns_screq_t *) fcp->isp_scratch);
        mbs.param[0] = MBOX_SEND_SNS;
        mbs.param[1] = SNS_RFT_REQ_SIZE >> 1;
        mbs.param[2] = DMA_WD1(fcp->isp_scdma);
@@ -2459,14 +2459,9 @@
 isp_start(XS_T *xs)
 {
        struct ispsoftc *isp;
-       u_int16_t iptr, optr, handle;
-       union {
-               ispreq_t *_reqp;
-               ispreqt2_t *_t2reqp;
-       } _u;
-#define        reqp    _u._reqp
-#define        t2reqp  _u._t2reqp
-#define        UZSIZE  max(sizeof (ispreq_t), sizeof (ispreqt2_t))
+       u_int16_t nxti, optr, handle;
+       u_int8_t local[QENTRY_LEN];
+       ispreq_t *reqp, *qep;
        int target, i;
 
        XS_INITERR(xs);
@@ -2698,7 +2693,7 @@
                isp_update(isp);
        }
 
-       if (isp_getrqentry(isp, &iptr, &optr, (void **) &reqp)) {
+       if (isp_getrqentry(isp, &nxti, &optr, (void **)&qep)) {
                isp_prt(isp, ISP_LOGDEBUG0, "Request Queue Overflow");
                XS_SETERR(xs, HBA_BOTCH);
                return (CMD_EAGAIN);
@@ -2709,6 +2704,7 @@
         * We do dual duty here (cough) for synchronizing for busses other
         * than which we got here to send a command to.
         */
+       reqp = (ispreq_t *) local;
        if (isp->isp_sendmarker) {
                u_int8_t n = (IS_DUALBUS(isp)? 2: 1);
                /*
@@ -2718,15 +2714,15 @@
                        if ((isp->isp_sendmarker & (1 << i)) == 0) {
                                continue;
                        }
-                       MEMZERO((void *) reqp, sizeof (*reqp));
+                       MEMZERO((void *) reqp, QENTRY_LEN);
                        reqp->req_header.rqs_entry_count = 1;
                        reqp->req_header.rqs_entry_type = RQSTYPE_MARKER;
                        reqp->req_modifier = SYNC_ALL;
                        reqp->req_target = i << 7;      /* insert bus number */
-                       ISP_SWIZZLE_REQUEST(isp, reqp);
-                       ISP_ADD_REQUEST(isp, iptr);
-
-                       if (isp_getrqentry(isp, &iptr, &optr, (void **)&reqp)) {
+                       isp_put_request(isp, reqp, qep);
+                       ISP_ADD_REQUEST(isp, nxti);
+                       isp->isp_sendmarker &= ~(1 << i);
+                       if (isp_getrqentry(isp, &nxti, &optr, (void **) &qep)) {
                                isp_prt(isp, ISP_LOGDEBUG0,
                                    "Request Queue Overflow+");
                                XS_SETERR(xs, HBA_BOTCH);
@@ -2735,7 +2731,7 @@
                }
        }
 
-       MEMZERO((void *) reqp, UZSIZE);
+       MEMZERO((void *)reqp, QENTRY_LEN);



Home | Main Index | Thread Index | Old Index