At 8:52 PM -0500 1/15/05, Tim Kelly wrote:
>while doing benchmarks I found that ping times on MP
>kernels are 30% or so slower than ping times on SP kernels.

As Allen Briggs indicated, this appears to be due to the kernel locking in
MP. I had a brief offline discussion with Nathan Williams prior to Allen's
response, and with Nathan's permission, I'm posting his answers to my
questions.

TK> Ok. Any suggestions as to why the MP kernel on this box (a 7300 w/ dual
TK> 180MHz 604e CPUs) would be almost 50% longer in ping times than the SP
TK> kernel built with the same -current sources?

NW> Sure. MP has overhead. Notably, in the current big-lock MP
NW> implementation, every interrupt and trap into the kernel has to
NW> acquire a lock, potentially needing to wait for the other CPU to
NW> relinquish the lock. MP is not a free lunch.

TK> OK. One last question: in extintr.c there are restrictions that limit
TK> interrupts to running only on cpu0. Do you know of any reason for this
TK> requirement, and would there be any significant performance gain to
TK> attempting to address this limitation?

NW >Given the big-lock MP structure, there's no advantage to letting both
NW> CPUs handle interrupts (since only one can be in kernel mode to
NW> process an interrupt at a time anyway), so permitting it it just adds
NW> complexity with no particular benefit.

NW> (Clock interrupts and IPIs are a different case from device
NW> interrupts, though)

Bill Studenmund some time ago made a comment (to the list) about true SMP
having no restrictions on which CPU handled interrupts, which is why I
asked Nathan about investigating removing this restriction that is built
into extintr.c

While there are significant time issues with ping, the disparity does not
carry to this extreme with ftp. I have only seen around a 5-10% drop off in
throughput from SP to MP kernels.

tim