Subject: Re: one program, one machine, two wildly different execution times
To: Ben Harris <firstname.lastname@example.org>
From: Jim Bernard <email@example.com>
Date: 04/21/2003 07:20:34
On Mon, Apr 21, 2003 at 12:57:47PM +0100, Ben Harris wrote:
> In article <20030420180434.GA13151@zoo.bernard.org> you write:
> > I have tracked down a couple of tidbits that may relate to the problem:
> >This one, from a beowulf list in 1998:
> > http://www.beowulf.org/pipermail/beowulf/1998-June/000831.html
> >describes a similar observation with some C code compiled with egcs on a
> >PII. The poster eventually tracked it down to a dependence on the alignment
> >of the stack pointer on entry, which apparently was not always the same.
> That sounds entirely plausible. NetBSD puts the environment on the top of
> the user stack, and its size can be affected by all sorts of things. You
> could try running the program with a cleaned environment using "env -i" and
> see if that makes it behave consistently. Alternatively, you could try
> printing __builtin_frame_address(0) and seeing if that offers any clues.
Actually, this does turn out to be the part of the problem causing the
variability. Sverre Froyen, with a little help from me, figured this out
yesterday afternoon. The execve system call only rounds the stack up to
the nearest 4-byte boundary, so both argv and the environment can affect
the alignment of stack-resident double-precision variables in the program.
Changing that alignment to 8 bytes fixes the variability, but there's still
an apparently fixed 4-byte offset coming from somewhere else. When that is
taken into account, the program always runs fast.
I'll be filing a PR later today with details and a small C program that
clearly demonstrates the effect. I'll include a patch that fixes the problem,
but it is not yet the correct fix.
Thanks for the ideas.