Subject: Locking interrupts to CPUs ...
To: None <current-users@NetBSD.org>
From: Andrew K. Adams <email@example.com>
Date: 12/16/2003 10:25:59
A little less than a year ago, I built a firewall using a dual
processor i386 box, two PCI GigEs for the external & internal
interfaces, and RH Linux (8.0, then 9.0), as NetBSD did not yet have
multi-processor support. Yes, I would have preferred to use
ipf(ilter) to iptables ...
Anyway, immediately after bringing the firewall up under *real* load,
we noticed TCP re-ordering. I don't have a good understanding of the
buffering/queuing of a packet before and after an interrupt is issued
by the device driver, but it seemed likely that one CPU was forwarding
a higher sequenced packet through the box before the other CPU.
After learning far more about Linux than I wanted to, I was able to
use Linux's '/proc/irq' system to lock each GigE device driver
interrupt to a *specific* CPU (i.e., eth1 -> CPU1, eth2 -> CPU2). Sure
enough, re-ordering went away.
So, does anyone know A) if re-ordering *will* be an issue with NetBSD
2.0 on a multi-processor, dual-homed box, and B) if so, will we have
the hooks in place to lock specific device driver interrupts to
I apologize if this has already been addressed on the list, but I saw
no references in the archive.
Andrew K. Adams
Pittsburgh GigaPoP & Network Research Group
Pittsburgh Supercomputing Center Office: 306-A Mellon Institute
Carnegie Mellon University Phone: (412) 268-5142
4400 Fifth Ave. Fax: (412) 268-8200
Pittsburgh, PA 15213 WWW: http://www.psc.edu/~akadams/
D3 FA 7D 61 FD ED BD D9 0C DE 94 DB 0F 25 D0 2E