Re: bin/55979 (sh single quotes removes nul characters)

[Robert Elz <kre%munnari.OZ.AU@localhost>]

    Date:        Tue, 9 Feb 2021 20:08:50 -0800
    From:        Justine Tunney <jtunney%gmail.com@localhost>
    Message-ID:  <CANtdasQJxExVw_fpBHGX=qPWMs56PC-6RH6nufTYM-X25CCORQ%mail.gmail.com@localhost>

  | This could be a memory corruption issue. /bin/sh behaves unpredictably when
  | it encounters nul characters inside single quotes.

That's unlikely, as it simply ignores nul chars when it reads them,
what you're seeing is probably something different.

  | When they don't work it'll usually prints garbled data:

Can you find a simple (short) test case (doesn't matter if it
does, or should do, anything meaningful) that you believe behaves
incorrectly, and send it to me?   Then I can take a look.

Actually, by inventing my own test case, I see that while we have
ancient code that deletes nul chars when it sees them, the way that's
done is (and has been for decades) broken, so we only ignore some of
them, not all.   Since \0 chars anywhere in shell scripts make a
non-conforming script, actually seeing a \0 char in a script is very rare,
so no-one has ever noticed.   I will fix the way we do that (make nul
chars be truly ignored, so that they're just not there), but I doubt that
it will fix your problem, as the effect seems to be different than you
described (but without seeing an actual failing test case I cannot be
certain).

  | I misdiagnosed the issue earlier.

Yes, I had worked that out.

  | What I need
  | is for the shell to safely ignore binary data inside single quotes.

Assuming that you don't try and use it (which is what I believe is
your intent) that should work, just provided, of course, the binary
data doesn't happen to contain a ' character.   Aside from \0, the
shell doesn't (shouldn't) really care what binary values form any of
the parts of the script which doesn't have syntax constraints.

Of course, bugs can always exist (and have in the past).

  | APE binaries always start with "MZqFpD=\n" so it won't impact this use
  | case.

If that was true, you wouldn't have a problem, but at least the hello.com you
provided a link to earlier started \177ELF which is where the issue arises.

kre