Subject: Re: Multiprocessor with NetBSD ?
To: None <current-users@netbsd.org>
From: Greg A. Woods <woods@weird.com>
List: current-users
Date: 06/03/2001 17:07:21
[ On Sunday, June 3, 2001 at 15:42:55 (-0400), John Kohl wrote: ]
> Subject: Re: Multiprocessor with NetBSD ?
>
> I have understood it to mean something more.  Asymmetric multiprocessing
> is when there are multiple CPUs but they aren't all equal.  Typically
> that means that any of them can run user-level code, but kernel calls
> all get funneled to some master CPU.
> 
> Symmetric MP to me means that _all_ the kernel code can run on all
> processors (including interrupt handling, assuming that it's possible
> for the hardware to handle interrupts on all processors--my
> understanding is that some HW designs don't let you do this).

Yes, this is what I've always understood SMP to mean too.

These paragraphs from the October 1984 issue of the Bell Laboratories
Technical Journal, "Multiprocessor UNIX Operating Systems", by
M. J. Bach and S. J. Buroff might help:

    For the purposes of this paper, a multiprocessor hardware
    configuration is one that has two or more processors that share a
    common memory, corresponding to what is commonly called a tightly
    coupled system.  It is distinguished from a loosely coupled system,
    where each processor has private memory, and where the processors
    communicate using a networking facility instead of shared memory.
	
    Multiprocessor hardware configurations can be further classified by
    their symmetry with respect to input/output (I/O).  In an Associated
    Processor (AP) configuration, only one processor is capable of doing
    I/O operations, while in a true multiprocessor configuration either
    processor can do I/O.  Except as specified the ensuing discussion
    applies to MP and AP configurations.

    Another multiprocessor UNIX system[1] permits only one processor,
    the master, to execute the kernel of the operating system, avoiding
    the system data corruption problems described in Section 3.1
	[[ .... ]]
    The multiprocessor UNIX systems described in this paper permit all
    processors to execute kernel code simultaneously, yielding maximum
    efficiency from the hardware configuration.

    [1]G. H. Goble and M. H. Marsh, "A Dual Processor VAX 11/780,"
    Purdue University Technical Report, TR-EE 81-31, Sept. 1981.

Later on in the paper they go on to describe how device drivers are made
to work on MP systems and also say "whereas MP systems can do I/O from
all processors."

> SMP also implies relatively fine-grain locks, at least at a subsystem
> level (e.g. file system lock for all file system code), and more usually
> at the object level (e.g. locks on inodes/vnodes), so that multiple
> processes in the kernel simultaneously aren't stalled waiting for
> resources.  A single "Big Kernel Lock" doesn't count in my book.

Indeed.  Much of that 1984 paper talks about using semaphores to protect
various data structures and how one can go about determining the most
appropriate "graininess".  They suggest that an optimum design should
give about 1.7 times the throughput on a dual processor as compared to a
single processor system.  From the Bach & Buroff paper again:

    The use of semaphores must be carefully chosen to balance frequency
    of semaphore operations versus the "granularity" of semaphore
    protection, that is, how much data are protected by a single
    semaphore.

Bach and Buroff are also quick to point out that many algorithms used
inside traditional UNIX didn't scale well either and that they had to
re-design many of them in order to achieve the performance they did.  I
think 4.4BSD already includes some of these improvements, but I'm not
sure it includes them all, and unfortunately the paper I quoted from 

If I remember correctly though there's some kind of "glass ceiling" that
makes it much harder to continue to gain equally well from >4 CPUs and
indeed this was the reason for the commercial success of the likes of
Sequent and Unisys in building many-CPU boxes running commercial UNIX
(i.e. once they learned the techniques necessary to break through this
barrier).

-- 
							Greg A. Woods

+1 416 218-0098      VE3TCP      <gwoods@acm.org>     <woods@robohack.ca>
Planix, Inc. <woods@planix.com>;   Secrets of the Weird <woods@weird.com>