Subject: Re: mindless boredom, speed and compiling kernels
To: None <port-pmax@NetBSD.ORG>
From: Simon Burge <simonb@telstra.com.au>
List: port-pmax
Date: 05/19/1998 22:08:56
On Tue, 19 May 1998 13:26:21 +0900 (JST)  Toru Nishimura wrote:

> Simon Burge wrote;
> 
> > "make -j 3" is a winner, saving approx. a minute and a half over just
> > a normal "make" (or about 5%) based on the fastest timimgs.  It's sort
> > of interesting that there's a similar number of page faults for each -j
> > option (pretty much 3602 always), but the number of io's varies a bit
> > (between 9107 and 9686) as the number of concurrent jobs goes up.
> 
> Pretty interesting...  I'm also interested in how differently UVM
> performs on given the same hardware.  In another MIPS camp, UVM kernel
> is getting matured, but still a lot work is needed for field use.  The
> developer once stated he saw performance improvement (a little).

When UVM is stable on the pmax, I'll re-run the tests at that time
with both UVM and non-UVM kernels.  I'll also think about "make build"
timings in the future too...

> > This is on a 5000/260 running 1.3.1, with the source tree mounted of
> > another 5000/260 running Ultrix 4.5 with PrestoServe.
> 
> Ah, I experienced severe performace degration of kernel compile over
> NFS with IOASIC machine (Personal DECstation) compared to on local
> filesystem.  I guess the reason why is IOASIC serializes LANCE access
> and disk I/O.  (And, that's the reason that DEC3000 introduced TCDS
> asic to separate SCSI traffice from LANCE traffic)

I loaded a copy of /usr/src/sys onto a local disk on the same machine,
and re-run the tests (sans the -j 10 compile).  In all cases, the make
of the kernel was quicker, with the best times split between -j 2 and
-j 3.  The average saving by going to local disk was in the order of 30
to 40 seconds, or about 2% (if the mental math is correct).


make -j 1:
    src on NFS server
	1228.890u 253.673s 27:57.53 88.3% 0+0k 21+9107io 3601pf+0w
	1226.280u 254.449s 27:43.82 88.9% 0+0k 9+9110io 3601pf+0w
    src on local RZ26
	1221.925u 237.911s 27:30.43 88.4% 0+0k 1499+7346io 3561pf+0w
	1219.619u 236.399s 27:27.33 88.3% 0+0k 1406+7340io 3561pf+0w
make -j 2:
    src on NFS server
	1253.525u 278.465s 26:24.93 96.6% 0+0k 21+9425io 3602pf+0w
	1255.721u 278.033s 26:23.47 96.8% 0+0k 21+9431io 3602pf+0w
    src on local RZ26
	1247.834u 255.922s 25:32.67 98.1% 0+0k 2096+8523io 3602pf+0w
	1249.026u 254.736s 25:34.56 97.9% 0+0k 2341+8505io 3602pf+0w
make -j 3:
    src on NFS server
	1267.517u 279.625s 26:20.60 97.8% 0+0k 20+9476io 3602pf+0w
	1260.947u 279.276s 26:13.59 97.8% 0+0k 21+9470io 3602pf+0w
    src on local RZ26
	1256.111u 256.716s 25:35.39 98.5% 0+0k 2333+8529io 3602pf+0w
	1261.113u 256.652s 25:38.91 98.6% 0+0k 2318+8537io 3602pf+0w
make -j 4:
    src on NFS server
	1274.441u 280.777s 26:26.37 98.0% 0+0k 9+9507io 3606pf+0w 
	1271.042u 282.688s 26:26.64 97.9% 0+0k 21+9493io 3602pf+0w
    src on local RZ26
	1262.161u 258.988s 25:42.38 98.6% 0+0k 2333+8593io 3602pf+0w
	1260.652u 258.231s 25:40.24 98.6% 0+0k 2338+8561io 3602pf+0w
make -j 5: 
    src on NFS server
	1275.491u 281.621s 26:31.06 97.8% 0+0k 21+9523io 3602pf+0w
	1280.704u 282.510s 26:42.65 97.5% 0+0k 21+9534io 3602pf+0w
    src on local RZ26
	1265.650u 261.012s 25:48.86 98.5% 0+0k 2353+8631io 3602pf+0w
	1266.458u 260.314s 25:48.26 98.6% 0+0k 2343+8613io 3602pf+0w

Simon.