Re: bin/55979 (sh single quotes removes nul characters)

[Robert Elz <kre%munnari.OZ.AU@localhost>]

The following reply was made to PR bin/55979; it has been noted by GNATS.

From: Robert Elz <kre%munnari.OZ.AU@localhost>
To: Justine Tunney <jtunney%gmail.com@localhost>
Cc: gnats-bugs%netbsd.org@localhost, gnats-admin%netbsd.org@localhost, netbsd-bugs%netbsd.org@localhost
Subject: Re: bin/55979 (sh single quotes removes nul characters)
Date: Wed, 10 Feb 2021 20:08:53 +0700

     Date:        Tue, 9 Feb 2021 20:08:50 -0800
     From:        Justine Tunney <jtunney%gmail.com@localhost>
     Message-ID:  <CANtdasQJxExVw_fpBHGX=qPWMs56PC-6RH6nufTYM-X25CCORQ%mail.gmail.com@localhost>
 
   | This could be a memory corruption issue. /bin/sh behaves unpredictably when
   | it encounters nul characters inside single quotes.
 
 That's unlikely, as it simply ignores nul chars when it reads them,
 what you're seeing is probably something different.
 
   | When they don't work it'll usually prints garbled data:
 
 Can you find a simple (short) test case (doesn't matter if it
 does, or should do, anything meaningful) that you believe behaves
 incorrectly, and send it to me?   Then I can take a look.
 
 Actually, by inventing my own test case, I see that while we have
 ancient code that deletes nul chars when it sees them, the way that's
 done is (and has been for decades) broken, so we only ignore some of
 them, not all.   Since \0 chars anywhere in shell scripts make a
 non-conforming script, actually seeing a \0 char in a script is very rare,
 so no-one has ever noticed.   I will fix the way we do that (make nul
 chars be truly ignored, so that they're just not there), but I doubt that
 it will fix your problem, as the effect seems to be different than you
 described (but without seeing an actual failing test case I cannot be
 certain).
 
   | I misdiagnosed the issue earlier.
 
 Yes, I had worked that out.
 
   | What I need
   | is for the shell to safely ignore binary data inside single quotes.
 
 Assuming that you don't try and use it (which is what I believe is
 your intent) that should work, just provided, of course, the binary
 data doesn't happen to contain a ' character.   Aside from \0, the
 shell doesn't (shouldn't) really care what binary values form any of
 the parts of the script which doesn't have syntax constraints.
 
 Of course, bugs can always exist (and have in the past).
 
   | APE binaries always start with "MZqFpD=\n" so it won't impact this use
   | case.
 
 If that was true, you wouldn't have a problem, but at least the hello.com you
 provided a link to earlier started \177ELF which is where the issue arises.
 
 kre