Subject: Re: PF_PACKET ???
To: Alfred Perlstein <bright@mu.org>
From: David Laight <David.Laight@btinternet.com>
List: tech-net
Date: 11/22/2001 11:03:25
> > > To 'shovel' 10Mb ethernet across the ISA bus means moving a byte
> > > > every 800ns.  I can't remember the exact figure, but a typical ISA
> > > > bus cycle is around 600ns (some of the cards I've seen actually add
> > > > wait states!).  So for an 8-bit card the data transfer takes 75% of
> > > > your cpu time -
> > > 
> > > Shouldn't that be "75% of your bus bandwidth"?
> > 
> > No, 100% of your ISA bus bandwidth, probably some thing like
> > 10% of your CPU<->memory bus bandwidth, and 75% of your CPU
> > time spent stalled in IO.

Acually the 'stalled in IO' and 'ISA utilisation' figures are much the
same as each other - unless you have a bus-mastering card (I don't think
just dma is enough!) but don't try that through the PCI-ISA bridge.....
> 
> That's almost certainly only true on an in-order processor with really
> dumb I/O hardware attached to it (which, I'll admit, the i486 is).

It doesn't matter whether you have a 4MHz 286, a 20MHz 386 or a 2GHz P4.
A 'rep insw' instruction will stall the cpu for EVERY transfer.
A 'rep outsw' will fill all the posted write buffers, then stall.
Since you stall mid-instruction the cpu can do nothing else.  The only
hope for getting any other work done is a multi-cpu system (or a cpu
that can execute 2 instuction streams).  But even then the system bus
is busy - probably with cycle-rerun requests.
> 
> However, it's not true anyway, because it demonstrably can't be.  I
> don't know what was wrong with the way the Unix the person who originally
> claimed this was doing its I/O, but in point of fact I've seen slow 486
> boxes running NetBSD do well more than 1MB/sec of programmed I/O on the
> ISA bus; with 16-bit cards (which you'd use, of course, were you not an
> idiot) I've seen them move well more than 3MB/sec.

Lets see, 3MBs is 1.5M cycles or 666ns - about what I said...
You can JUST move that amount of data about, but you have precious little
resource left to do anything with it.

This table - from the other mail - shows it all...

                 per packet       per byte     Packet rate    Throughput
                   delay           delay        (64bytes)     (1500 bytes)
-----------------------------------------------------------------------------
4.77 Mhz XT    | .51 msec       .00406 msec     1300 /sec    1.8 Mbits/sec
10 Mhz XT      | .237 msec      .00254 msec     2500 /sec    3.0 Mbits/sec
6 Mhz AT       | .169 msec      .00228 msec     3200 /sec    3.3 Mbits/sec
12 Mhz AT (est)| .100 msec      .00220 msec     4150 /sec    3.5 Mbits/sec
16 Mhz AT      | .050 msec      .00190 msec     5800 /sec    4.1 Mbits/sec

Now I think these times are 'through the bridge' - so involve 2 copies to
and from the card.
Note in paricular that changing from the 6M AT (286) to 16M AT has only a
small effect on the per-byte delay.  This is because the ISA bus cycles are
the limiting factor - and are probably about 800ns.

The readme also quotes that the WD card only has 6.5k of input buffer and that
it will underrun with NFS traffic....  The 16bit card stands a chance.

FYI the 286 will do 'out of order' execution, in particular it will execute
pre-fetched instructions while the result of an earlier write is waiting for
the system bus.  This means that the sequence:
        outb    interrupt_mask    /* 8259 interrupt controller */
        clri                      /* enable interrupts (or is it seti?) */
will break the cycle recovery time of the 8259 - trashing ALL of its internal
registers - because the first IACK cycle is too close to the write.
(Was fun tracking that one down..... didn't need to on the second project...)

    David