tech-kern archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: Strange behaviour



BTW, it was definitely a typo in the original - my system is 24 cores at 2.1GHz, not 3.2GHz.

On Sun, 29 May 2011, Thor Lancelot Simon wrote:

On Sun, May 29, 2011 at 06:24:06PM -0700, Paul Goyette wrote:
The system: dual-12-core AMD Opteron-6172 (total 24 cores at
3.2GHz), and 32GB RAM.  (The motherboard is a ASUS KGPE-D16).

While running a 'build.sh -j48 release' I notice that there seems to
be a rather excessive amount of "System Time".  Running systat(1) I
[...]
Is this expected behavior?

Yes.  We do a little better with 16 build jobs than 12 and considerably
worse with 20 than 16.

On my 24-core 2Ghz Opteron, the only thing I've tried that really can
do an effective job building NetBSD with more than 12 CPUs in use is
Linux.  I should probably try Solaris.

Despite the level of "System" time, I was able to run a complete "build.sh -j48 release" in less than 45 minutes. And afterwards, I was able to build a complete new kernel in 44 seconds. (I guess most all of the source directory was cached somewhere in that 32GB of RAM!)

Linux under Xen does even worse than NetBSD, probably because Xen has
no way to express to Linux the system's memory architecture.  Xen has
"NUMA support" in the sense that if you have, say, 6 cores assigned
to a virtual and those 6 can be mapped onto physical cores on the same
NUMA node, thus providing the virtual a uniform memory hierarchy, it
will do so; but if you have a 24-core machine built from two pairs
of dual-6-core chips, like the machines you and I have, it really doesn't
do a good job telling the guest what it needs to schedule effectively.





-------------------------------------------------------------------------
| Paul Goyette     | PGP Key fingerprint:     | E-mail addresses:       |
| Customer Service | FA29 0E3B 35AF E8AE 6651 | paul at whooppee.com    |
| Network Engineer | 0786 F758 55DE 53BA 7731 | pgoyette at juniper.net |
| Kernel Developer |                          | pgoyette at netbsd.org  |
-------------------------------------------------------------------------


Home | Main Index | Thread Index | Old Index