Subject: Re: Interrupt, interrupt threads, continuations, and kernel lwps
To: None <>
From: Bill Studenmund <>
List: tech-kern
Date: 02/22/2007 09:46:42
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Thu, Feb 22, 2007 at 09:13:36AM -0800, wrote:
> In message <>Andrew Doran writes
> >Hi Jonathan,
> >
> >On Wed, Feb 21, 2007 at 05:57:41PM -0800, wrot=
> >
> >My changes make this neither worse nor better.
> Hi Andrew,
> I truly don't know what to say to that.=20
> Here's my example case again:
>         ( IPsec'd packet comes in, asserts NIC interrupt)
> 1. switch from user to NIC interrupt thread
>         (...  NIC calls ether_input which demuxes packet, enqueues on
>               protocol input routine, requests softint processing, blocks)
> 2. switch from NIC hardware thread to softint thread
>         (... OCF submits job, blocks ...)
> 3. switch from softint to user =20
>         (... crypto hardware finishes, requests interrupt...)
> 4.  switch from user to crypto-interrupt thread
>         (... crypto driver calls OCF which wakes up softint processing...)
> 5. switch to softint thread, process cleartext packet
>         (... done with local  kernel packet processing, softint thread  .=
> 6. switch back to user.=20
> And here's an equivalent monolithic-kernel+biglock scenario,
> recognizable for 4.3BSD(-Tahoe) to NetBSD-3:
>         ( IPsec'd packet comes in, asserts NIC interrupt)
> 1. kernel takes interrupt, calls into NIC device interrupt handler
>    in currently active context.  Note no context switch.  [1]
>         (...  NIC calls ether_input which demuxes packet, enqueues on
>               protocol input routine, requests softint processing, return=
> 2. After return from hardware interrupt handler, but before returning
>    to the pre-interrupt state, the kernel checks for pending software
>    interrupts.  Here, we run softints (assuming they weren't active
>    at the time we took the interrupt).
>   =20
>         (... IP calls to FAST_IPSEC, to OCF, OCF submits job, returns ...)
> 3. continue returning from   softint to user.  Note no context switch.
>         (... crypto hardware finishes, requests interrupt...)
> 4.  Kernel takes hardware interrupt. Note, no context switch.
>     (... crypto driver calls OCF,  which calls FAST_IPSEC=20
>     continuation, which requests further softint processing via
>     schednetisr() ...)
> 5.  On return from hardware interrupt, the kernel checks for
>     pending softints. If softint processing was not already active,
>     the kernel does   software-interrupt callouts.
> 6.  Continue returning from interrupt back to the pre-interrupt
>     user code.
> The first scenario has several context switches. (It also has hardware
> interrupt traps, which we have to take and turn into scheduler events
> to wake up the corresponding thread; plus returns from those traps).

I think the problem is you've assumed an implementation, and specifically=
you've assumed one other than what Andy was suggesting.

My understanding is that Andy has figured out a way to have, at least on=20
x86, the interrupt handler borrow the context of the interrupted thread.=20
So the interrupt context switch is also the context switch to the thread.=
That's why he said it was the same as what we do now.

> Compare the TCP rates (multiple ttcp sessions on multiple NICs) that
> uniprocessor NetBSD-3.1 can sustain, to what FreeBSD-6 can sustain on
> the same hardware --- even *with* SMP and multiple CPUs, and even with
> handling the netisr processing in the context of the
> hardware-interrupt thread (as Jason mentioned earlier).

Let's try it and see! Rather than discuss hypotheticals, let's get #s.=20
Either the numbers will show little change, which will make ALL of us=20
happy, or the numbers will show a marked decrease, which will displease=20
ALL of us. :-)

Take care,


Content-Type: application/pgp-signature
Content-Disposition: inline

Version: GnuPG v1.4.3 (NetBSD)