NetBSD-Bugs archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
Re: bin/55979 (sh single quotes removes nul characters)
The following reply was made to PR bin/55979; it has been noted by GNATS.
From: Justine Tunney <jtunney%gmail.com@localhost>
To: gnats-bugs%netbsd.org@localhost
Cc: kre%netbsd.org@localhost, gnats-admin%netbsd.org@localhost, netbsd-bugs%netbsd.org@localhost
Subject: Re: bin/55979 (sh single quotes removes nul characters)
Date: Mon, 15 Feb 2021 13:18:53 -0800
--00000000000079a54205bb668aaf
Content-Type: text/plain; charset="UTF-8"
I think the fact that people are still discovering novel use cases for the
design of the Thompson Shell after all these years, just goes to show how
gracefully the technology has aged, and how empowering it continues to be
as a technology. Thanks for being flexible. I'm looking forward to seeing
the fixes!
On Fri, Feb 12, 2021 at 9:45 AM Robert Elz <kre%munnari.oz.au@localhost> wrote:
> The following reply was made to PR bin/55979; it has been noted by GNATS.
>
> From: Robert Elz <kre%munnari.OZ.AU@localhost>
> To: Justine Tunney <jtunney%gmail.com@localhost>
> Cc: gnats-bugs%netbsd.org@localhost, netbsd-bugs%netbsd.org@localhost
> Subject: Re: bin/55979 (sh single quotes removes nul characters)
> Date: Sat, 13 Feb 2021 00:42:42 +0700
>
> Date: Wed, 10 Feb 2021 10:39:13 -0800
> From: Justine Tunney <jtunney%gmail.com@localhost>
> Message-ID: <
> CANtdasRbJQCznmCStFQFsYuOrb8BjQdoQiv8XLfAxOEYgD2RAw%mail.gmail.com@localhost>
>
> | Try downloading https://justine.lol/hello.com again.
> | That file can be your test case.
>
> I did that, and used it, though:
>
> | I can create a more minimal one too if you need it.
>
> I did that for myself as well, once I worked out just what was
> going on, and what was really needed to perform the test (small
> tests that test nothing more than the bug are ideal ... of course
> to make one of those you need to first know exactly what the
> bug really is).
>
> Turns out that back in 1995 a fix for another problem broke the
> way that \0's were being dropped, and caused corrupted input (more
> corrupted than just dropping the nul chars) to be handed to the
> shell parser.
>
> That we have had this bug for more than 25 years says something
> about just how rare it is for shell scripts to actually contain
> nul characters. Of course, any that do are non-conforming, so
> we could simply claim our current behaviour as correct (though
> it clearly is an obvious bug ... now it has been found).
>
> I am currently building an updated system (HEAD) with the fix
> (and the MSAN detected bug fix as well, not that that one really
> matters), after which (tomorrow, or later today depending upon
> timezones) I'll run the ATF tests to check that nothing new appears
> to have broken (as this fix should change nothing for a script without
> nul characters in it, and the MSAN fix change is truly trivial,
> that should not be a problem).
>
> Expect to see the fix(es) in HEAD early next week.
>
> After it has caused no problems there for a while, I'll request
> a pullup to -9. But as neither of these bugs bother almost
> anyone, ever, and because its remaining lifetime isn't expected
> to be all that long, I don't think I'll bother with pullups to -8.
> (Of course, any other developer who wants to do the work to
> incorporate the fixes there, and test them properly, could do that.)
>
> If you want to test the updated sh on a -9 system, just copy the
> sources from HEAD (after the fix appears) and build it on a -9
> system (cd src/bin/sh; <update sources to HEAD>; make) (or as part
> of a build of the -9 system on any host). There should be no problems
> doing that (but going to -8, while not all that hard, is slightly more
> complex than just copy and build). Note that building sh requires
> access to the sources of its built-in commands (test, printf, kill, ...)
> so the rest of the (userland at least) sources should be present.
>
> kre
>
> ps: no changes are needed to the way we detect binary files so as not to
> interpret them as scripts - that turned out to all be misdirection, the
> actual problem is closer to what was originally reported, though single
> quoted strings have nothing specific to do with it, and nuls certainly
> do not survive sh processing.
>
>
>
--00000000000079a54205bb668aaf
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div>I think the fact that people are still discovering no=
vel use cases for the design of the Thompson Shell after all these years, j=
ust goes to show how gracefully the technology has aged, and how empowering=
it continues to be as a technology. Thanks for being flexible. I'm loo=
king forward to seeing the fixes!</div></div><br><div class=3D"gmail_quote"=
><div dir=3D"ltr" class=3D"gmail_attr">On Fri, Feb 12, 2021 at 9:45 AM Robe=
rt Elz <<a href=3D"mailto:kre%munnari.oz.au@localhost";>kre%munnari.oz.au@localhost</a>> w=
rote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0p=
x 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">The follow=
ing reply was made to PR bin/55979; it has been noted by GNATS.<br>
<br>
From: Robert Elz <<a href=3D"mailto:kre%munnari.OZ.AU@localhost"; target=3D"_blank"=
>kre%munnari.OZ.AU@localhost</a>><br>
To: Justine Tunney <<a href=3D"mailto:jtunney%gmail.com@localhost"; target=3D"_blan=
k">jtunney%gmail.com@localhost</a>><br>
Cc: <a href=3D"mailto:gnats-bugs%netbsd.org@localhost"; target=3D"_blank">gnats-bugs@n=
etbsd.org</a>, <a href=3D"mailto:netbsd-bugs%netbsd.org@localhost"; target=3D"_blank">=
netbsd-bugs%netbsd.org@localhost</a><br>
Subject: Re: bin/55979 (sh single quotes removes nul characters)<br>
Date: Sat, 13 Feb 2021 00:42:42 +0700<br>
<br>
=C2=A0 =C2=A0 =C2=A0Date:=C2=A0 =C2=A0 =C2=A0 =C2=A0 Wed, 10 Feb 2021 10:39=
:13 -0800<br>
=C2=A0 =C2=A0 =C2=A0From:=C2=A0 =C2=A0 =C2=A0 =C2=A0 Justine Tunney <<a =
href=3D"mailto:jtunney%gmail.com@localhost"; target=3D"_blank">jtunney%gmail.com@localhost</a>&g=
t;<br>
=C2=A0 =C2=A0 =C2=A0Message-ID:=C2=A0 <<a href=3D"mailto:CANtdasRbJQCznm=
CStFQFsYuOrb8BjQdoQiv8XLfAxOEYgD2RAw%mail.gmail.com@localhost" target=3D"_blank">CANt=
dasRbJQCznmCStFQFsYuOrb8BjQdoQiv8XLfAxOEYgD2RAw%mail.gmail.com@localhost</a>><br>
<br>
=C2=A0 =C2=A0| Try downloading <a href=3D"https://justine.lol/hello.com"; re=
l=3D"noreferrer" target=3D"_blank">https://justine.lol/hello.com</a> again.=
<br>
=C2=A0 =C2=A0| That file can be your test case.<br>
<br>
=C2=A0I did that, and used it, though:<br>
<br>
=C2=A0 =C2=A0| I can create a more minimal one too if you need it.<br>
<br>
=C2=A0I did that for myself as well, once I worked out just what was<br>
=C2=A0going on, and what was really needed to perform the test (small<br>
=C2=A0tests that test nothing more than the bug are ideal ... of course<br>
=C2=A0to make one of those you need to first know exactly what the<br>
=C2=A0bug really is).<br>
<br>
=C2=A0Turns out that back in 1995 a fix for another problem broke the<br>
=C2=A0way that \0's were being dropped, and caused corrupted input (mor=
e<br>
=C2=A0corrupted than just dropping the nul chars) to be handed to the<br>
=C2=A0shell parser.<br>
<br>
=C2=A0That we have had this bug for more than 25 years says something<br>
=C2=A0about just how rare it is for shell scripts to actually contain<br>
=C2=A0nul characters.=C2=A0 =C2=A0Of course, any that do are non-conforming=
, so<br>
=C2=A0we could simply claim our current behaviour as correct (though<br>
=C2=A0it clearly is an obvious bug ... now it has been found).<br>
<br>
=C2=A0I am currently building an updated system (HEAD) with the fix<br>
=C2=A0(and the MSAN detected bug fix as well, not that that one really<br>
=C2=A0matters), after which (tomorrow, or later today depending upon<br>
=C2=A0timezones) I'll run the ATF tests to check that nothing new appea=
rs<br>
=C2=A0to have broken (as this fix should change nothing for a script withou=
t<br>
=C2=A0nul characters in it, and the MSAN fix change is truly trivial,<br>
=C2=A0that should not be a problem).<br>
<br>
=C2=A0Expect to see the fix(es) in HEAD early next week.<br>
<br>
=C2=A0After it has caused no problems there for a while, I'll request<b=
r>
=C2=A0a pullup to -9.=C2=A0 =C2=A0But as neither of these bugs bother almos=
t<br>
=C2=A0anyone, ever, and because its remaining lifetime isn't expected<b=
r>
=C2=A0to be all that long, I don't think I'll bother with pullups t=
o -8.<br>
=C2=A0(Of course, any other developer who wants to do the work to<br>
=C2=A0incorporate the fixes there, and test them properly, could do that.)<=
br>
<br>
=C2=A0If you want to test the updated sh on a -9 system, just copy the<br>
=C2=A0sources from HEAD (after the fix appears) and build it on a -9<br>
=C2=A0system (cd src/bin/sh; <update sources to HEAD>; make) (or as p=
art<br>
=C2=A0of a build of the -9 system on any host).=C2=A0 There should be no pr=
oblems<br>
=C2=A0doing that (but going to -8, while not all that hard, is slightly mor=
e<br>
=C2=A0complex than just copy and build).=C2=A0 =C2=A0Note that building sh =
requires<br>
=C2=A0access to the sources of its built-in commands (test, printf, kill, .=
..)<br>
=C2=A0so the rest of the (userland at least) sources should be present.<br>
<br>
=C2=A0kre<br>
<br>
=C2=A0ps: no changes are needed to the way we detect binary files so as not=
to<br>
=C2=A0interpret them as scripts - that turned out to all be misdirection, t=
he<br>
=C2=A0actual problem is closer to what was originally reported, though sing=
le<br>
=C2=A0quoted strings have nothing specific to do with it, and nuls certainl=
y<br>
=C2=A0do not survive sh processing.<br>
<br>
<br>
</blockquote></div>
--00000000000079a54205bb668aaf--
Home |
Main Index |
Thread Index |
Old Index