Seeking assistance with testing next sh feature

[Robert Elz <kre%munnari.OZ.AU@localhost>]

Hi all,

I sent a message to these lists about 30 hours or so ago (well over a day
anyway) and have had very little response - possibly because I made it
look very much like spam (it wasn't...) and might have been caught by
spam filters, or just deliberately overlooked.

Anyway, I think I have the next sh addition ready - it has actually been
ready for a while now, but has been sitting waiting for the LINENO changes
to stabilise first (as this is more useful, in some cases with a properly
working LINENO, -- with only the old LINENO hack some nice stuff would not
work.)

This change adds parameter ${var}, arithmetic $(( )), and (optionally - a
run time option, defaulting off) command substititon $( ) expansions of
PS1 PS2 and PS4 before they are printed.

(This is, or will soon be, required by posix - parameter expansions are
already required, arithmetic is very likely to be added, with command subs a
maybe.)   Note there are no other expansions performed by this shell (never
will be I hope, nor are any others needed) - no ! (even though posix says we
should) and no \h type noise that other shells support.

It is for PS4 that LINENO is really wanted, having

	PS4='[${LINENO}] + '

makes it much easier to follow, and use, "set -x" output.  Use a tab, before
or after the '+' to get the output lines lined up and looking nicer...

The problem (for me) with this set of changes is that prompts are only
printed, and so only evaluated, in interactive mode (aside from PS4, which
because of how and when it is used is much less of a problem - so much so
that while FreeBSD do expansions on PS4, they only do a few \ expansions
on PS1 and PS2)

This makes it hard to test.

So I am seeking people who are willing to use the new shell as their
interactive shell and push it hard, do all kinds of strange things to it,
and see if you can make it break (and generally whether it "feels right").

The advantage for this set of changes is that they do essentially nothing
to a shell running a script (certainly if -x is not set) so have almost
no chance of breaking anything - unlike the last set, which changed all
kinds of internal stuff.

Still, I would like to avoid committing these changes until there has been
at least some more testing than what I am capable of.

You can fetch the full sh source tarball from

	ftp://munnari.oz.au/kre/sh-expand_prompt.tgz

or you can just fetch the patches from what is in cvs HEAD (from any
time since 2017-06-20-00:00:00 UTC - there have been no commits there
since then, or even in the few hours before then) from

	ftp://munnari.oz.au/kre/sh-expand_prompt-PATCHES

If you pick the latter method, copy src/bin/sh (to avoid messing up those
sources) somewhere, and in the new somewhere, patch < ...-PATCHES
Alternatively unpack the tgz file (it will make a new directory called
sh-expand_prompt and put the sources in there), and then build it like

	make NETBSDSRCDIR=/usr/src USETOOLS=no

(it should just take a couple of minutes to compile, but don't bother even
trying on anything other than NetBSD - unless you want to include this in
your source tree, and use build.sh to do a normal cross-build to use on
NetBSD - that should work, attempting to make this run on some other system
would be more work than it is worth for just this test.  That is a longer
term goal.

This version (unless you change the Makefile) compiles with DEBUG mode
included, but disabled (so it will affect nothing - though the shell will
be bigger, and a bit slower, than normal) - but it might be useful if
some problem occurs and I need to get you to help me isolate what is happening.

Alternatively, if you have a NetBSD 6 amd64 system (or later version, up to
and including 8.99.1, and almost certainly 8.99.2 when it appears) then you
can instead (if you trust me enough) just fetch a compiled binary instead.

It is at
	ftp://munnari.oz.au/kre/sh-amd64-NetBSD-6.gz

Just gzip -d it, change its name to something suitable, and turn on execute
permission, and it should just work (on amd64 NetBSD >= 6.0)

I have tested that it works on systems from 6.0.1 to 8.99.1 (obviously
not every version in between, but enough to be confident it will work on
all of them.)   AMD64 only though (sorry other people!)

In addition to expanding the prompts before they are used (to use command
substitution you need to "set -o promptcmds" (there is deliberately no one
character name for this one), this also has one new variable, PSc which
contains just a one character string '$' or '#' (which it will be depends
on whether you are root or not.)   This is intended to make it easier to
construct a PS1 from nothing (no need to try and append the previous PS1,
which may have been imported from the environment, and contain anything,
just to get that one character).  This is just a normal variable, if you
change its value, well, you change its value...  (but its value will not be
inherited from the environ, the shell always sets it at startup.)

In a later version I will have (well already do have, but not included in
the version that is available) more new variables that obviate the need
for actually using command substitutions for most common usages (like
putting the time in the prompt).  I don't want that available yet, as it
is the command substitution code (and what happens when the commands get
interrupted, and similar) that is most likely to cause a problem, if anything
is, just doing variable expansions, and arithmetic, is fairly simple.

As a hint (probably of something you don't really want to do) if you
include something like $(sleep 5) in your prompt string, the shell will
help you slow down your pace of work - let you enjoy life more!   Do realise
however that the whole prompt string is expanded before any of it is
written, you cannot expect PS1='.$(sleep 1).$(sleep 1).$(sleep 1) $PSc '
to print three dots, with a one second sleep between each, and then end
with the $ or #.  Rather it will sleep 3 seconds (three 1 second sleeps,
one after the other) and then (quickly) write "... $ " as the prompt.

To offer a little more incentive for more testing, this version (the one for
FTP) also has improved -x output - it now shows redirects as well as
what it did before, and attempts (in a way I am not sure I like, so
suggestions welcome) to show redirections on compound commands as well
(aside from "for" variable assignments, compound commands themselves are
not shown in -x output, just the simple commands that are run.)

So, please, give it a try if you possibly can, see how you can break it,
and then let me know what you did and what happened when it does break (if
it does, it seems to just work for me, but I am not all that imaginative.)

Remember that when setting prompts, if you want expansions at prompt time,
the prompt needs to be set in single quotes PS1='....', alternatively
all the $ characters in there need to be escaped.  Otherwise the values
will be expanded when you set PS1 (with PS1="${PSc} " for example, which
in that case would be fine, and in fact the best way - provided that you
use care, "PS1="${PSc}_" would not do anything like what you want, though
that works fine as PS1='${PSc}_'   You can use all the variable expansion
modes though (including the pattern match and trim operators), and if
you can find a use for it, arithmetic.  One kind of cute (but currently too
difficult to actually use much) trick is to do

	L1=${LINENO}; PS2='$((LINENO - L1))> '

immediately (at least the L1= part, PS2 can be set any time earlier) before
you start typing a command that will be continued over multiple lines.
Then your PS2 prompt will be numbered, so you know how many lines you
are entering...


Also note that the prompt is expanded as a word (like an arg to echo would
be) it is not a command string, if you write "A=0; ..." then the prompt
will contain the characters A = 0 ; (and whatever follows).  The only thing
that is ever executed are command substitutions (if enabled) and as they
execute in sub-shell environments, they cannot change anything in the shell.

The prompt is also evaluated as if it occurs in a "" string, so there are no
pathname expansions, tilde expansions, or field splitting, or anything else
exotic like that.

Please, feedback is needed.

kre