Re: Strange behaviour

[Thor Lancelot Simon <tls%panix.com@localhost>]

On Sun, May 29, 2011 at 06:24:06PM -0700, Paul Goyette wrote:
> The system: dual-12-core AMD Opteron-6172 (total 24 cores at
> 3.2GHz), and 32GB RAM.  (The motherboard is a ASUS KGPE-D16).
> 
> While running a 'build.sh -j48 release' I notice that there seems to
> be a rather excessive amount of "System Time".  Running systat(1) I
[...]
> Is this expected behavior?

Yes.  We do a little better with 16 build jobs than 12 and considerably
worse with 20 than 16.

On my 24-core 2Ghz Opteron, the only thing I've tried that really can
do an effective job building NetBSD with more than 12 CPUs in use is
Linux.  I should probably try Solaris.

Linux under Xen does even worse than NetBSD, probably because Xen has
no way to express to Linux the system's memory architecture.  Xen has
"NUMA support" in the sense that if you have, say, 6 cores assigned
to a virtual and those 6 can be mapped onto physical cores on the same
NUMA node, thus providing the virtual a uniform memory hierarchy, it
will do so; but if you have a 24-core machine built from two pairs
of dual-6-core chips, like the machines you and I have, it really doesn't
do a good job telling the guest what it needs to schedule effectively.

Thor