Re: suitable shells for bootstrapping pkgsrc

[David Sainty <david.sainty%gmail.com@localhost>]

On 29 June 2015 at 07:24, David Holland <dholland-pkgtech%netbsd.org@localhost> wrote:

On Fri, Jun 26, 2015 at 02:59:57PM -0600, Brook Milligan wrote:
 > On Jun 26, 2015, at 2:06 PM, David Holland wrote:
 > > Both bash and dash ought to work. So the question is what the
 > > bootstrap script has been imbibing?
 >
 > Well, the test seems to be whether the shell's echo command is
 > BSD-compatible.  Apparently the dash echo is not.

Hmm... dunno. All I know is that there's been talk on here about
making dash the default pkgsrc shell for certain platforms, as bash is
horribly slow and despite whatever problems dash may have it still
beats pdksh. But it isn't something I've been keeping track of.

It would be glorious if some non-bash shell was automatically used on Linux.  Bash is far and away the worst performing shell I've ever come across.  But on Linux it's what you tend to bootstrap with.

Let's see how ludicrously atrocious it is [spoiler...].  I just did a time trial building "mail/evolution".

Environment (approximately):

CPU=i5

RAM=4GB

HARDDRIVE=SSD

MAKE_JOBS=4
WRKOBJDIR=/tmp/pkgsrc [4GB tmpfs]
USE_CWRAPPERS=yes

Method (approximately):

1. Build all dependencies of "mail/evolution"

2. Put into mk.conf: CONFIG_SHELL=${PREFIX}/bin/pdksh

3. pkg_delete libtool-base; cd libtool-base && make install

4. ( time bmake install ) >&| evolution-ksh.log

5. pkg_delete evolution

6. Put into mk.conf: CONFIG_SHELL=/bin/bash

7. pkg_delete libtool-base; cd libtool-base && make install

8. ( time bmake install ) >&| evolution-bash.log

wc -l evolution-*.log
   6003 evolution-bash.log
   6011 evolution-ksh.log

... verifies the work is in the same ballpark.  Minor differences in configure output, and MAKE_JOBS=4 is resulting in inconsistent task ordering.

Results:

% tail -1 evolution-bash.log
bmake install  3368.24s user 496.39s system 82% cpu 1:17:42.98 total
% tail -1 evolution-ksh.log
bmake install  1613.37s user 329.62s system 72% cpu 44:27.94 total

Well that's much worse than I imagined.  For a job that you'd expect to be mostly gcc and ld, Bash fully doubled the amount of user CPU burnt.  Elapsed time not quite double - but getting there.  No cwrappers would be still worse I expect.

But I'm not surprised Bash came out badly.  The number of times I've wondered why a build is dragging only to find a libtool/bash process that's been spinning for two or three minutes.

I chose Evolution for testing because it happened to be the package that finally made me crack today and look for a better way than using Bash.

I'm not sure exactly what that better way should be...  But ideally bootstrapping under Linux should "just work", and should arrange for Bash to be as ignored as is possible, because it's not really fit for purpose.