Re: killing subshells in /bin/sh scripts

[Robert Elz <kre%munnari.OZ.AU@localhost>]

    Date:        Sat, 24 Jun 2017 13:51:52 +0000
    From:        "Johnny C. Lam" <jlam%NetBSD.org@localhost>
    Message-ID:  <20170624135152.GA7059%homeworld.netbsd.org@localhost>

  | Mr. Elz:

Please, forget the formality, we're all friends here, right, just "kre"
is fine .. I know I'm ancient, but ...

  | I'm curious what is the best way for a script to kill off all of its
  | children and grandchildren (the process tree that was spawned by it)
  | in a portable manner.

Well the simple answer is that you put the processes in a process group,
and then use killpg(3) to send a signal to the entire group.

Any other way (chasing processes using ps, and ppid values, or even process
group ids, to make a tree, whether done in the sh script, or in an external
command is doomed to fail in many cases, processes can be created much more
quickly than it is ever possible to observe, and then kill them in separate
steps.)


But you want to know how to do it from sh script, which makes it a little
messier.

Putting processes in a separate process group is easy, that happens if you
have job control turned on.   That's on by default in interactive shells,
and not otherwise, but you can always enable it (it is even a posix defined
option.)   That is, "set -m" (or "set -o monitor" if you prefer.)
(Technically, -m is only required with the "User Portability Utilities"
option of posix - but that really means everywhere except embedded 
environments.)

But see below for an important side effect to be aware of if you take this
approach.

With "set -m" when you start the background process, from the shell that is
running the script directly (not when a process is started in any sub-shell
inside it) it will be placed in a separate process group (actually, all 
commands that need a separate process to run, foreground and background,
when run from the top level shell directly, will go in a separate process
group - which is also important later - and is also necessary to make
normal, interactive, job control work - so you can run a process, like say
an editor, in the foreground, and then later suspend it and (though probably
not for an editor) resume it as a background process.)

Next is how to find and kill it - in an interactive shell, you just use the
job number reported when it is started (with a shell that does that, or use
the "jobs" command to find out), and kill %n.   As in ...

$ sleep 10&
$ 
$ jobs
[1] + Running                 sleep 10
$ kill %1
$ 
[1]   Terminated              sleep 10

The "kill [-s signal] %n" works just fine in a script as well, provided that
you can somehow work out what "n" to use.   That is not necessarily easy,
though in some cases you might be able to just count, and use the knowledge
that the shell (well, /bin/sh anyway, I cannot promise all shells work like
this) will always assign a new job a job number one bigger than the highest
currently existing job (ie; not yet terminated and reported).  Not easy...

That's something I had been planning on fixing, provide a mechanism something
like $! which would provide the job number instead of the process number, so
you would simply know which 'n' to use.   But even if I do that, it would not
be portable, no other shell I know of (though I have not read all of zsh's
documentation ... there is just so much, not all of ksh93's or even bash's
for recent versions, either for that matter) provides a mechanism like that.

/bin/sh also has a "jobid" command (which is not standard) which will tell
you the ID (as in process group) of the "current" job (which will be the
last one started).   Unfortunately, for use in scripts, other limitations
in our shell make this almost useless JOB=$(jobid) which would be the
rational way to use it fails, as in the sub-shell created to run the $( )
there is no current job...  You can redirect output to a temp file and
then read that, but that's just too ugly.   (Fixing this, which is needed
for more important things than jobid, is fairly high on the todo list though,
and this problem is much less likely to affect other shells - though the
lack of a jobid command in them might be an impediment...)

However, there's another way which comes pretty close - the process group id
assigned is (in practice, and perhaps required) always the process id of the
process group leader - that is the common ancestor of all the processes in
the process group.   In our case that is what $! tells us, so we know the
process group number created - it is the same as the process id of the top
level process.

So now all we need to know is how to do the killpg(3) from the shell, given
that we have a process group id, and not the job id

One way would be to use the process id / process group id., and search the
output of jobs -l for that pid, and match that to a job number, but I am not
convinced that the output of jobs -l, even though its format is specified
by posix, is necessarily going to be the same everywhere, and anyway that
is messy.

But if you do that you will have a 'n' you can use with "kill [-s sig] %n',
to send the signal to the process group.   (And yes, that is standard posix
notation, though they don't talk about process groups, much as they don't
really even mention processes in the shell standard - the idea is to be able
to, somehow, implement the shell, and all that goes with it on systems that
have no concept of separate processes ... not likely to work, but...)

We could write a new command to do it (a killpg command), but that would not
be portable.  Nor would adding a "-g" option to kill(1).

Instead (and here we go beyond posix, but into fairly widely supported
territory I believe, in that it has been like this a very long time), if
you send a signal to a negative process ID, it sends the signal to the
process group that is the absolute value of that ID.

What all this means, is that after starting a background process with
set -m, you can do "kill -s sig -$!" to kill it and its entire group.)

In this the signal number (either using the more standardised form, "-s sig"
or in the more traditional "-SIG") must be given, or the '-' before the
process id will be taken as if you were specifying a signal,
ie:
	kill -1234
is sending signal 1234 to - well, no-one.

There are a couple of issues to be aware of.

First, some shells, including /bin/sh (currently, must put this on
the list to fix) and bash, simply do not believe that when a job
finishes (for any reason) and it was running in a separate process
group, than we really don't need to be told about its passing - that
is, there's an assumption that the shell just must be interactive
if that happens.   Better shells are more discrete, and only blather
about completed jobs when we really are interactive.

That means that we have to muzzle the shell when we kill the processes
to avoid needless noise.

Given all of that (but be aware, another issue follows this) this script
seems to work with most shells I have tried (which includes our /bin/sh
and none of the recent modifications are needed for this, a shell from years
ago works just as well as a recent one, and also bash, zsh, yash, dash,
mksh, the FreeBSD shell, and ksh93) but not our /bin/ksh (which is just
too ancient) and not Joerg Schilling's "bosh" (and hence, probably not SysVR4
vintage shells.)   I don't have the posh shell to test.

So, here is the script that shows the method:

set -m

( 
	(
		(sleep 5; echo hello) && echo 5 done &
		(sleep 6; echo goodbye) && echo 6 done &
		wait
	) && echo all done
) & PID=$!

sleep 1; ps T
exec 5>&1 6>&2 2>/dev/null 1>&2
kill -s TERM -${PID}
wait ${PID}
exec 1>&5 2>&6 5>&- 6>&-
ps T


None of the "hello/goodbye/done" messages get printed (with a shell
which works for this) as we kill everything while the sleeps are
still running.

Also note that the background processes created inside the innermost ()
subshell, do not get separate process groups here, even though -m is still
enabled when they are created (attempting to turn it on, again, there
would make no difference at all).  Only the top level shell will create
a new group, so the process that is backgrounded has be be started by the
base level of the script (inside an "if" or "while" or even a "{ }" group,
or a function, is fine, but not inside anything that makes any kind of
sub-shell.)

The first exec line is just the shell muzzle, and just as with dogs,
if it is well behaved, the muzzle is not needed...   The second exec
line is taking the muzzle off again after the danger has passed.

The "ps T" commands are just so it is possible to observe what is happening.
You can add "-O pgid" to those commands to see the process groups if you like.
You should see a similar number of shell processes for any shell
implementation here (maybe one more for /bin/sh) as the "&& echo" stuff
prevents the shell from skipping a sub-shell (except perhaps the first
one, in the "( ) &".)

OK, now you know how to do it, there is the one remaining important issue
that was mentioned earlier, which might make you decide that this simply
isn't worth the bother.

When processes are put into a separate process group, they are, naturally
no longer in the process group they would have been in.  What that means
is that external job control attempts (like typing ^Z while the script
is running) have no effect on the separate process group that has been
created.  Nor would an attempt to kill the script with "kill %1" or
something from your interactive shell that started the script (which
here I am assuming is job 1) kill that separate process group.   You
would need to go find it and kill it separately.

You can observe this if you (quickly) type a ^Z while that script is
running.   As given there, the ^Z will (if you time it correctly)
actually cause the "sleep 1" to become suspended.  -m applies to all
processes created, not only background ones, that sleep will have its
own, and will be the running foreground process, so that will be the
process group signalled by the tty driver when you type ^Z - the shell
running the script will only see the effect of the "sleep 1" suspendind.

Then the shell simply goes on, kills the background job immediately, and
finishes - warning you when it exits that there is a suspended job left
behind (or at least our /bin/sh does).  You can mitigate this behaviour
by turning -m off again (set +m) when it is not needed.  Make that change,
insert "set +m" before the "sleep 1" and now the shell, and the sleep 1, will
all be in the same process group.  The ^Z will suspend all of them,
but the other process group will keep on running, and if you wait the
needed 6 seconds before resuming the suspended shell, you will see the
output. as in ...

$ sh /tmp/tss
^Z[1] + Suspended               sh /tmp/tss
$ hello
5 done
goodbye
6 done
all done

(that version has the "set +m" in it).

Then later ...

$ fg
sh /tmp/tss
  PID TTY    STAT    TIME COMMAND
  617 pts/45 S+   0:00.01 sh /tmp/tss 
 9270 pts/45 O+   0:00.00 ps -T 
17569 pts/45 Ss   0:00.25 sh 
  PID TTY    STAT    TIME COMMAND
  617 pts/45 S+   0:00.01 sh /tmp/tss 
 2093 pts/45 O+   0:00.00 ps -T 
17569 pts/45 Ss   0:00.25 sh 

There's no error from the kill failing because the muzzle is still there.

This is why non-interactive shells (those created to run scripts, etc) do
not have -m normally enabled - we want the whole script to run in a
single process group, so we can control it as a unit.

kre