Re: X=1 : (was Re: /bin/sh redirect without command failure mode)

[Robert Elz <kre%munnari.OZ.AU@localhost>]

    Date:        Sun, 25 Nov 2018 23:00:08 +0100
    From:        Rhialto <rhialto%falu.nl@localhost>
    Message-ID:  <20181125220008.GK20600%falu.nl@localhost>

  | This may not be documented in sh(1) but here I find some description:

There is a limit to just how muct we can put in sh(1) - the posix spec is
60 pages (without any of the built-ins being included).   So a lot of the
fine points are going to be missing there.   We should probably find the
USD doc for sh and update that, and include it, if the legalities allow.

  | So what we do in sh conforms, and what bash does doesn't.

In posix mode, bash conforms as well.

  | However this still leaves me unsatisfied. "Special built-ins" seem to be
  | those commands that cannot possibly be implemented as external commands:

That looks to be the characterisation, but it really more relates to how things
were implemented in the original Bourne shell.  The concept of a "special"
built-in was invented to allow the way those things behaved to be documented
as a group, rather than as a lot of special cases.

  | And even for the special built-ins, I find it not obvious that the
  | assignment should work differently.

I agree, and if you had made this argument 40 years ago, things might
have ended up different than they are.    But it is too late to do anything
about that now.

It used to be that this applied to functions as well, that is with

	unset X
	X=1 f

whether X was set or not after that sequence depended upon whether
'f' was defined as a function or not (not existing at all is the same as
being a command in the filesys).    This was because of how the early
implementation of functions was done (in simple terms, if the shell did
not fork, then variable assignments persisted).

That conflicted with one of the goals of functions, that they should be
indistinguishable from external commands (other than performance, etc).

So, now posix just says this is unspecified ...   which means useless for
everyone,

And from the later message ....

  | This is an interesting unexpected case:
  |     X=/ cd $X
  | cd's to $HOME, not to / ...

  | This really violates the POLA, I'd say... 

Perhaps, though this one is easier to explain.   The command and args
are expanded first, they need to be, as the command could have been

	X=/ $CMD $X

(or something) and we don't know what to do with the assignment until
we know what $CMD is (or if it is empty, and vanishes, what X is (or was
more correctly)

The var assignments, and redirects can always be
detected without any expansions, as something like

	$X $Y $Z

can never have a var assign, nor a redirect, even if

	X='foo=bar'
	Y='>'
	Z=/dev/null

instead that is the "foo=bar" command (or function) with 2
args, ">" and "/dev/null".

Processing a shell simple command is always done by first removing
any var assigns, and any redirects, and then the rest of the words are
the command and args - those are expanded (~, param, arith, cmd-sub,
field splitting, glob, and quote removal) and made into an argv[] for the
command to be run - then the var assigns are done and added to the
environ for the command, and finally redirects are processed.
(For special builtins, just to be different, posix allows the redirects
to be processed before the var-assigns, if the shell prefers... we don't).

Note that in a command like

	X=file ls >"$X"

it is unspecified whether the "$X" that is used will be the one on the
var assign, or whatever value X had previously (so if you're wanting
to do something like that, just write
	X=file ls >file
instead if that is what you mean...    Or
	OldX=$X ; X=file ls >"$OldX"
if that was the intent.   It is also unspecified whether effects of one
var assign, or side effects from any of the expansions (like ${X=foo})
are visible to any of the others, or to redirects.   For much of
this it really is the case that different shells do it all differently.

And just to add some more weirdness, note that in this one case ...

	PATH=/bin:/somewhere:/foo cmd

the new value of PATH is used (by the shell) for locating "cmd" as
well as being placed in its environment.

That's the one (the only I believe) case where a value in a var-assign
is actually used by the shell proper for anything at all (not counting when
it adds it to the command's environment of course.) 

Some of the builtin utilities use values from the environment, so

	HOME=/ cd

should do "cd /" (though it is a kind of verbose way or writing it)
as in cd's environment the value of HOME should be what was
exported to it (not that there is really any exportig happening).
but what the utilities (even the built in ones in sh) do is not really
the shell - and few of them use the environment at all.

kre