Re: Strange behaviour

[Paul Goyette <paul%whooppee.com@localhost>]

BTW, it was definitely a typo in the original - my system is 24 cores at 
2.1GHz, not 3.2GHz.

On Sun, 29 May 2011, Thor Lancelot Simon wrote:

> On Sun, May 29, 2011 at 06:24:06PM -0700, Paul Goyette wrote:
> > The system: dual-12-core AMD Opteron-6172 (total 24 cores at
> > 3.2GHz), and 32GB RAM.  (The motherboard is a ASUS KGPE-D16).
> >
> > While running a 'build.sh -j48 release' I notice that there seems to
> > be a rather excessive amount of "System Time".  Running systat(1) I
> [...]
> > Is this expected behavior?
>
> Yes.  We do a little better with 16 build jobs than 12 and considerably
> worse with 20 than 16.
>
> On my 24-core 2Ghz Opteron, the only thing I've tried that really can
> do an effective job building NetBSD with more than 12 CPUs in use is
> Linux.  I should probably try Solaris.

Despite the level of "System" time, I was able to run a complete 
"build.sh -j48 release" in less than 45 minutes.  And afterwards, I was 
able to build a complete new kernel in 44 seconds.  (I guess most all of 
the source directory was cached somewhere in that 32GB of RAM!)

> Linux under Xen does even worse than NetBSD, probably because Xen has
> no way to express to Linux the system's memory architecture.  Xen has
> "NUMA support" in the sense that if you have, say, 6 cores assigned
> to a virtual and those 6 can be mapped onto physical cores on the same
> NUMA node, thus providing the virtual a uniform memory hierarchy, it
> will do so; but if you have a 24-core machine built from two pairs
> of dual-6-core chips, like the machines you and I have, it really doesn't
> do a good job telling the guest what it needs to schedule effectively.





-------------------------------------------------------------------------
| Paul Goyette     | PGP Key fingerprint:     | E-mail addresses:       |
| Customer Service | FA29 0E3B 35AF E8AE 6651 | paul at whooppee.com    |
| Network Engineer | 0786 F758 55DE 53BA 7731 | pgoyette at juniper.net |
| Kernel Developer |                          | pgoyette at netbsd.org  |
-------------------------------------------------------------------------