Subject: Re: what blocks splserial?
To: Manuel Bouyer <bouyer@antioche.lip6.fr>
From: Daniel Brewer <danielb@cat.co.za>
List: tech-kern
Date: 08/19/2003 17:42:24
>
> Ok, so there is only one interrupt for all the stuffs the PCI card can
> do, including the serial ports. From what I understand, this interrupt
> routine is registered to the kernel with IPL_TTY, right ?
>
> From what you describe, it's the pci driver for you card which then does
the
> interrupt dispatch to the various drivers attached to this driver.
> If it is registered with IPL_TTY, then all things at or above IPL_TTY
(which
> includes IPL_AUDIO and IPL_CLOCK) will block it. Note that IPL_TTY is also
> IPL_IMP (memory allocations), which I guess is the culprit here: memory
> management can sometimes take a long time.


Sorry, it seems I haven't been quite as clear as I thought. Heres some
segments from the PCI card's (from now on called PCIIO because it handles
all sorts of input triggers and output relays) attach:

...
if (pci_mapreg_map(pap, 0x14, PCI_MAPREG_TYPE_MEM,
     0,&memt, &memh, &mema, &memsize))
    {
      printf("  Can't map MEM!\n");
      return;
    }
...
if(pci_intr_map(pap->pa_pc, pap->pa_intrtag,
      pap->pa_intrpin,pap->pa_intrline,
    &intrhandle) != 0)
    {
      printf(": couldn't map PCI interrupt\n");
      return;
    }

  intrstr = pci_intr_string(pap->pa_pc, intrhandle);

  // choose TTY interrupt level to handle PCIIO interrupts
  sc->sc_ih = pci_intr_establish(pap->pa_pc, intrhandle,
     IPL_TTY,pciio_intr, sc);
...
//attach 4 comports which are on add-on board
for(i=0;i<4;i++) {
  bus_space_handle_t subregion_handle;
  paa.port = i;  paa.type = 0;
  paa.flags = COM_FREQ*4;
  paa.pc = pap->pa_pc;
  paa.intrhandle = intrhandle;
  paa.a = mema;   paa.t = memt;
  //get handles to subspace
  if(bus_space_subregion(memt, memh, 0x400+i*32, 32,
              &subregion_handle) != 0)
   {
      printf("%s: couldn't get subregion for port %d\n",
         sc->sc_dev.dv_xname, i);
      continue;
   }
  paa.h = subregion_handle;

   //attach com device
  sc->sc_ports[i].dev = config_found_sm(self, &paa,
         pciio_print, pciio_submatch);
 }
...

int
pciio_print(aux, pnp)
   void *aux;
   const char *pnp;
{
 struct pciio_attach_args *paa = aux;
 if(pnp)
  printf("%s at %s","",pnp);
 printf(" port %d", paa->port);
 return(UNCONF);
}

int pciio_submatch(parent, cf, aux)
   struct device *parent;
   struct cfdata *cf;
   void *aux;
{
 return ((*cf->cf_attach->ca_match)(parent, cf, aux));
}


Each comport on the add-on board (there are 4 com ports which will attach to
com2-com5) gets attached with the following function (modified slightly from
the puc device):

void
com_pciio_attach(parent, self, aux)
 struct device *parent, *self;
 void *aux;
{
 struct com_pciio_softc *psc = (void *)self;
 struct com_softc *sc = &psc->sc_com;
 struct pciio_attach_args *aa = aux;
 const char *intrstr;

 /*
  * XXX This driver assumes that 'com' ports attached to 'puc'
...
  */

 sc->sc_iobase = aa->a;
 sc->sc_iot = aa->t;
 sc->sc_ioh = aa->h;
 sc->sc_frequency = aa->flags; // & PUC_COM_CLOCKMASK;

 intrstr = pci_intr_string(aa->pc, aa->intrhandle);
 psc->sc_ih = pci_intr_establish(aa->pc, aa->intrhandle, IPL_SERIAL,
     comintr, sc);
...
 com_attach_subr(sc);
}

Now, the same interrupt pin should cause the comintr(...) isr to be called
at priority IPL_SERIAL and the pciio_intr(...) isr to be called at priority
IPL_TTY. The comintr(...) function is the generic comport interrupt handler
that works on all comports. In our case, it accesses the registers on the
add-on board through the bus_space_subregion tag & handle. As far as it's
concerned, it's talking directly to a 16550 USART. This interrupt handler
however, will not clear the pci-interrupt pin - that is controlled by the
PCIIO card & driver. The pciio_intr(...) function reads the interrupt status
register on the PCIIO. A bit in this status register will inform the driver
that a com-port on the add-on card has interrupted. The PCIIO driver takes
no action if this is the case - it only handles the other functions of the
card. The PCIIO driver will clear the interrupt on the PCIIO device once
nothing else needs to be dealt with on the PCIIO.

int
pciio_intr(arg)
     void *arg;
{
...
   isr = ReadBAR1(0x03); //see if we've got unhandled interrupts
   if (isr == 0)
     {  //nothing is unhandled, so clear interrupt
         ReadBAR0(0x03);
      }
...
   if (isr & 0x04) //com-port interrupt
   {
      //nothing happens here
   }
...
If only the com-port triggers an interrupt, the PCIIO driver won't do
anything (and will return 0). Note that the interrupt pin will only be
cleared when the status reads 0 (once all the com-port interrupts have been
handled, bit 3 of the status register will go low). This should cause the
pciio_intr(...) function to be re-entered until all interrupts have been
dealt with. This was a work-around we've had to implement and I'm wondering
if this is having nasty unforseen side-effects.

So, to summarise, once an interrupt occurs on the add-on card, the PCIIO
pulls its pci interrupt pin high. The com-port driver for com2-com5 should
respond at IPL_SERIAL and access the registers on the add-on card to handle
the interrupt. The PCIIO driver should respond at IPL_TTY but do nothing.
Once all the com interrupts have been dealt with, the PCIIO driver will
clear the interrupt on the PCIIO.

>
> I guess the way to solve it is to register your interrupt at IPL_SERIAL
> in the PCI driver. Then your PCI interrupt routine will then call the
> com interrupt routine from here, but for the others interrupts source
> of your PCI adapter you'll probably want to use a soft interrupt or a
> kernel thread, to not do all the work at SPL_SERIAL.