Subject: so why doesn't 'cc -pipe' create any noticable parallelism on SMP machines?
To: NetBSD Kernel Technical Discussion List <tech-kern@NetBSD.ORG>
From: Greg A. Woods <woods@weird.com>
List: tech-kern
Date: 04/11/2005 00:27:36
So as I sit here staring at my "build.sh -u" run with "CFLAGS+=-pipe" on
my dual-CPU alpha I'm wondering why I don't usually see any more than
one CPU fully utilized, and that's including with two xterms (one for
the scrolling build output) and "systat -w 1 vm" running on the machine
as well.  (this is with a 1.6.x kernel on the build machine)

In fact I rarely ever see more than one runnable process, and the times
I do can, I believe, easily be attributed to xterm and systat overhead.

Now I know that the compiler doesn't start spitting out object code
right away, and cpp isn't really that much of a CPU pig, but I really
did always assume that "-pipe" would be more than just window dressing,
and indeed it seemed to improve overall build performance on
uniprocessor machines, at last after the new pipe implementation was
first introduced.

Even more curiously:  during straight compiler runs over many files,
e.g. when building a kernel, I almost always see two processes waiting
for disk.  How can two processes in the compiler pipeline be both
waiting for disk almost all of the time?  Obviously cpp could be reading
from disk while the assembler writes object code out to a .o file, but
with a sample interval every second it's hard to believe that there's
almost always two processes waiting for disk and one runnable on every
sample while at the same time the total CPU utilization almost never
breaks 50%.

Are processes waiting for pipe I/O also being counted as "waiting for
disk"?  Is the pipe code still tied up in biglock too much to allow
pipeline processes to run in parallel?  Is GCC a very poor benchmark for
pipeline parallelism?

-- 
						Greg A. Woods

H:+1 416 218-0098  W:+1 416 489-5852 x122  VE3TCP  RoboHack <woods@robohack.ca>
Planix, Inc. <woods@planix.com>          Secrets of the Weird <woods@weird.com>