tech-kern archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: lockstat from pathological builds Re: high sys time, very very slow builds on new 24-core system



On 4/1/2011 1:47 PM, Andrew Doran wrote:
On Fri, Apr 01, 2011 at 01:05:02PM -0400, Thor Lancelot Simon wrote:

On Thu, Mar 31, 2011 at 04:32:12PM -0400, Thor Lancelot Simon wrote:
On Thu, Mar 24, 2011 at 12:04:02PM +0000, Andrew Doran wrote:

Try lockstat as suggested to see if something pathological is going on.  In
addition to showing lock contention problems it can often highlight a code
path being hit too frequently for some reason.

I have attached build.sh and lockstat output from 24-way and 8-way builds
of the amd64 GENERIC kernel on this system.  Also dmesg and mount -v output
so you can see what's mounted how, etc.

Mindaugas spent quite a while looking at this with me.  It appears to
be largely due to contention on the page queue lock.

I did a few back-of-a-cigarette-packet calculations.

On the 8 CPU system we are burning about 1% of the total CPU time on lock
contention, so not all that significant.

On the 24 CPU system we're burning 27% of the available CPU time.  So
roughly 6.5 cores worth of your machine are doing nothing but spinning on
locks.  So lock contention in the VM system is a problem but it doesn't tell
the full story.

Is anyone working on lockless implementations within NetBSD? 24-core systems are becoming very common, even in two socket pizzabox rack servers.

RCU is one of the foundations of the Linux kernel but is patent encumbered with a GPL exception by IBM. There are may be valid alternatives using something like epoch observation. Check out this BSD licensed project http://concurrencykit.org/doc/safeMemoryReclamation.html for a rundown and good implementations.

I'm not sure how realistic widespread implementation would be as a GSoC project but I'd be willing to investigate some limited scope if there is a suitable mentor available.

Regards,
Kevin




Home | Main Index | Thread Index | Old Index