On Mon, Apr 21, 2003 at 12:57:47PM +0100, Ben Harris wrote:
> In article <20030420180434.GA13151@zoo.bernard.org> you write:
> >  I have tracked down a couple of tidbits that may relate to the problem:
> >
> >This one, from a beowulf list in 1998:
> >
> >  http://www.beowulf.org/pipermail/beowulf/1998-June/000831.html
> >
> >describes a similar observation with some C code compiled with egcs on a
> >PII.  The poster eventually tracked it down to a dependence on the alignment
> >of the stack pointer on entry, which apparently was not always the same.
> 
> That sounds entirely plausible.  NetBSD puts the environment on the top of
> the user stack, and its size can be affected by all sorts of things.  You
> could try running the program with a cleaned environment using "env -i" and
> see if that makes it behave consistently.  Alternatively, you could try
> printing __builtin_frame_address(0) and seeing if that offers any clues.

  Actually, this does turn out to be the part of the problem causing the
variability.  Sverre Froyen, with a little help from me, figured this out
yesterday afternoon.  The execve system call only rounds the stack up to
the nearest 4-byte boundary, so both argv and the environment can affect
the alignment of stack-resident double-precision variables in the program.
Changing that alignment to 8 bytes fixes the variability, but there's still
an apparently fixed 4-byte offset coming from somewhere else.  When that is
taken into account, the program always runs fast.

  I'll be filing a PR later today with details and a small C program that
clearly demonstrates the effect.  I'll include a patch that fixes the problem,
but it is not yet the correct fix.

  Thanks for the ideas.

--Jim