Re-establishing the "magic" of the special vars in sh(1)

[Robert Elz <kre%munnari.OZ.AU@localhost>]

The shell (/bin/sh) has a number of "special" or "magic" variables.

One of those (LINENO) is required by POSIX (and autoconf) so the
general concept is not going away (so please don't bother replying
to this message along the lines of "the whole thing is a bad idea
and should be done another way" regardless of how true that might be.)

A couple of them (including LINENO, obviously) but also RANDOM are
very special, the magic cannot be removed, though it can temporarily
be suspended by unsetting them.   That is, those variables cannot be
used as an ordinary variable name in any script.   This is not generally
a problem, as they are well known and have the same properties in many
shells.

When I added a few more magic vars to sh(1) (mostly so I would not need
to add the horrid csh style \x escape char handling to prompt string
expansions) I made all of them act like LINENO and RANDOM in this regard
- they could be unset, but if set again they would revert to being magic
so none of the new ones could be used as ordinary variable names either.

That was a mistake, and particualrly affected HOSTNAME, which while a
common name (as in, used the same way in other shells) for its purpose,
is also a name that is quite reasonable for scripts to want to use simply
as an ordinary variable name (and we had some script breakage because of that.

Consequently, that was changed, and now (like most, and perhaps all,
other shells with these kinds of variables) if a magic var (except the
very special couple) is ever assigned to, it loses its magic for the
life of the shell.

That's better.   But it is not perfect.   One consqeuence of this is that
if such a variable name is set in the environment, then it is assigned in
the shell, and so loses its mojo, even if the script does not want the
value from the environment, and would prefer the magic values to be available.

I believe that some other shells process the environment first, and then
set up the magic - but that's wrong, there's no reason that a set of
coordinated scripts should not pass data between them in the environment
and if they happen to choose a variable name that happens to be one of
the special ones, then that should still work, and the desires of the
shell to make things like HOSTNAME behave in a different way should not
usurp that.   That is, the way the current NetBSD sh processes the
environment and uses assignments there to override the specialness of
the magic vars is correct.

So, we have a problem....   One fact we do know however, is that if a
script wants to make use of the special properties of a magic variable
- then it knows not to use the variable internally in some other way,
defeating its intent.   It also knows the name of the special var it
wants to use (obviously).

So, one solution to this might be to add a new builtin command
something like this:

    specialvar variable ...
            For each variable name given, if the variable named is one which,
            in this sh, could be treated as a special variable, then cause
            that variable to be made special, undoing any effects of an
            earlier unset or assignment to the variable.  If all variables
            given are recognized special variables in this sh the specialvar
            command will exit with status 0, otherwise 1.  Invalid usage will
            result in an exit status of 2.

            Note that all variables capable of being special are created that
            way, this command is not required to cause that to happen.
            However should such a variable be imported from the environment,
            that will cause (for those special variables so designated) the
            special effects for that variable to be lost.  Consequently, as
            the contents of the environment cannot be controlled, any script
            which desires to make use of the properties of most of the special
            variables should use this command, naming the variables required,
            to ensure that their special properties are available.

Which I have done in uncommitted code (the above, aside from the lack
of markup caused by my cut&paste from an xterm into this e-mail, is from
the uncommitted sh.1 update that goes with it.)

It all seems to work, and is very small.   There is no check that the
"variable" named in the arg list is syntatically correct, anything is
OK (provided that if it looks like an option, it is either preceded by
another name, or the "--" special option ender ... just like in most other
commands).   Even though this costs little, it will not be added to
SMALL shells (there's no need, there are no affected variables in those.)

It is safe to use as

	if specialvar FOOBAR 2>/dev/null
	then
		# I can use the magic FOOBAR variable
	else
		# I cannot, and have to cope some other way
	fi

should work (in all correct shells) without generating noise
(though other shells do not have this command - yet - even those
that have special variables, and might have one called FOOBAR).

I am suggesting this now, as it ought to be pulled up for 8.1
(if added) as the relevant issues all occur (will occur) in that.
That is 8.0 release has the "once special always special" initial
implementation, but the change to undo that has already been pulled
up, so 8.1 will have the "it's trivial to break my script using the
environment" variation instead.

Any comments or suggestions (including to the wording of the man
page extract included above, or the proposed command name) ?

If not, I will commit this sometime this coming week, and request
a pullup to -8 soon after.

kre

ps: making any of the special vars "local" works (or will work with
this addition) in all variations you can imagine, as it logically should.
That's already implemented (nothing extra needs doing) though currently
there is no way to make a var which has lost its magic outside behave in
a magic way inside a function, even when declared local.  This command
(or something functionally equivalent) would allow that.

pps: one option I have (very briefly) considered, but not bothered with,
as I am not sure there is any need, would be to extend the command to
allow "specialvar newname=magicvar" which would not assign anything, but
would instead make "newname" a magic variable name, with the same
properties as "magicvar" whether or not "magicvar" has been overrun
by mediocrity and cannot be made magic again for logistal reasons.
That would not be hard to add if enough people believe it might be useful
(in that case, "newname" would be syntax checked, and it would be an
error if it was not in the correct form to be a variable name.)  If
"magicvar" does not name a magic var (regardless of whether its mojo has
already been lost), "newname" would not be changed (typically would just
remain unset and the only error report would be the non-zero exit status.