Re: Future shell work - comments reqyuested

[Robert Elz <kre%munnari.OZ.AU@localhost>]

    Date:        Thu, 13 Jul 2017 09:01:26 -0400
    From:        Izaac <izaac%setec.org@localhost>
    Message-ID:  <20170713T124718Z@localhost>

  | Stop.
  | Leave /bin/sh alone.

Sorry, that is not going to happen, there are still far too many bugs,
and a few things missing that are needed.

  | If you want features for an interactive session, use a different shell.
  | If you want features for a scripting environment, use a different shell.

No.

I have no idea what you think is left after eliminating those, but it
is not anything I want, for any purpose.

One of the big advantages of early unix over other systems of the time, was
that its CLI (the shell) was also its scripting language.  True, back then it
was not much of a scripting language, but it was at least the same thing as
was used for interactive use.

I consider that to be of vital importance, and will continue to try to
make sure that /bin/sh offers both an environment that is functional, and
usable, for both interactive use and scripts.

I do however take your point, as I believe it too, that programs should be
written in the appropriate language, and I will certainly not be doing anything
to attempt to compete with other programming languages (not even bash/ksh93/
zsh.)

  | There is no good which can come from drifting away from the most basic
  | POSIX functionality.  The only purpose of adding features is for them to
  | be used.  And their use breaks compatibility.  Therefore, the addition
  | of features breaks compatibility.

Depends upon the features.   Until recently (and certainly in 2008)
the NetBSD sh did not have assignment operators in arithmetic - which
are mandated by posix, and may be expected by any portable script.
Should I have avoided adding them to not break compatibility?   (With what?)

Note that scripts that don't use assignment ops in arith can keep on
not using them without any problems.   Or you can make use of them, you
won't be sacrificing any portability, as I think that every other shell
has had this for years (and yes, every other shell, except a few ancient
commercial versions, which simply haven't been updated in decades.)

So, was adding that feature really the wrong thing to do?


I actually find it kind of fascinating that you commented this way in
response to a message where I was asking about deleting stuff that we have
(and have always had) but which is probably not necessary, rather than about
adding stuff (sure, there was the the prelude about the extra arith operators
that will be added, they are needed to keep us on par with other shells, so
scripts can avoid requiring a different shell be installed - they're also
very small changes, and easy to add - the ATF tests that will be needed for
them are way more bloat than the sh changes).

	[Aside: I have been told that the reason (the only reason) that the
	',' operator wasn't specified for sh by posix, is that awk does not
	implement that, and they only wanted one table of C expression
	operators, so ',' just got omitted - not because  ',' was not intended
	be implemented in sh - it is, and was, implemented in ksh, and except
	where it is absurdly stupid, and even sometimes where it is, ksh
	(the real one, and ksh88 in particular) formed the basis for what
	posix specifies about sh.]

If what you were trying to say is that the let and exp commands are useful
(or one of them) and should be left alone, given their ancient heritage)
then just say that - that is a position I can understand, even though I
have never seen anything use exp, anywhere, ever, and what uses of let I
have seen all expect the ksh version (which would be cheaper to implement
than our version.)   If that was your point, please make it more clearly.

I'm not sure how your comments could relate to the other question (whether
-i on the command line should work or not) - fixing that would mean deleting
two lines of code I believe (so the shell would be fractionally smaller,
and unmeasurably slightly faster at startup ... neither of which is sufficient
by itself to warrant a change, but arg processing consistency might be.
The -i option is not posix, so we get no guidance from that.]

I was also amazed to read (in a different message) that you believe that only
static languages survive - and citing C, Fortran, COBOL, and Ada for that,
one of which is essentially dead (and wasn't stable when it was alive)
another which (outside some niche environments) was more or less stillborn,
and the other two, which are still alive and in widespread use, are nothing
like stable, and are continually being changed.

If you think that C today is anything like it was in 1974, you should really
get a K&R C compiler, and try using it...   If you think Fortran today is
anything like it was in the 1960's or 70's, then again, do some research, and
see what has changed.

sh is no different, it is constantly being updated by POSIX - they way it
works is that some shell adds something, users use it, other shells copy it,
then posix standardises it (so any shells that haven't picked up the new
feature, need to, to remain conformant.)   Overall this improves the
environment for the users (and yes, shells  get bigger, and sometimes,
slightly slower - though the latter is often offset by scripts being able
to run faster.)   And while this seems like a simple process, it takes years.

There are a whole stack of that kind of changes already approved for the next
major version of POSIX, and more coming all the time (the minor revisions,
like the 2016 edition, which actually appeared earlier this year, contain
only "clarifications" - but even those can result in changes being required,
once it is discovered that apparently clear text (we thought) apparently all
this time has meant something different than we have believed...)

As an example of a change that is coming, and which will require (as yet
unmade) changes to sh, is one that was approved just a week or two ago (for
the next major revision, I think, though it might be another clarification,
and in the next minor revision, I will need to check sometime).

That will mean that the arg string of an eval, or the file that is the
subject of a '.' script, must be parsed completely, before executing any
of the commands in it - the point being that a syntax error late in the
string/script will cause execution of the whole thing to be avoided.

What is perhaps most amazing is that that is (at least in the opinion of
may of the posix standards writers) not the actual change - that is, they
view that as being the way it it was always meant to be, the actual change
is in relaxing that requirement in other environments - for example in
startup scripts (.profile, etc) and perhaps even in a command file like
"sh script" (where it might never have applied, but perhaps - exactly when
to execute, has never been clear.)

We don't do that, we parse commands, and as soon as we have something that
can be executed, we execute it, wherever it came from.

So, what should I do abut that?   Not that I am going to do anything in
the immediate future about this one, but perhaps eventually.   Should we
just ignore POSIX (and requirements of portable scripts) in the name of
stability, or should we implement what is required, and that scripts might
start (more than they do now) to expect (note that some shells do, and
always have, implemented eval and '.' this way, I suspect startup scripts
as well.)

When considering this, note that the only time there is a detectable
difference is when the eval string, or the . file script, contains an
error, correct commands will execute the same as they always have
(though the shell will probably grow bigger - dynamic malloc() type
memory, to hold the entire parsed script whereas now we process a command,
execute it, discard everything (free mem) and go onto the next.)


Some of the other future changes (particularly the easy ones...) we have
already, others are still to come.   Of course, we still have to complete
(at least most of, some I won't do, as it is trash) what is in the current
version of posix.

So, once again, sorry, but I will not simply leave sh alone as it is now.
If you want that (or some earlier version) just

	cd src/bin/sh
	cvs update -Pd -D2008-01-01

or whichever version you want, and you can stagnate for as long as you like.

Also apologies to Thor, sorry, but I am also not aiming at feature parity
with ksh, there's just too much in there that is unnecessary (eg: select,
read using REPLY when no var is given, ...).

kre