OK, it turns out there is an obvious bug in the libtool included with
gettext which triggers this problem, fix that bug (patch appended below,
right at the end of this e-mail - how to make it work in pkgsrc I leave
to the pkgsrc experts) and whatever issue the new sh is having is
irrelevant. (Someone might want to check whether pkgsrc libtool has
the same issue, and perhaps other applications which embed their own
libtool copies).
There is clearly a bug in the current sh - but as I said in the HEADS-UP
message (on current-users) anything wrong in the changes that were made
really should affect no-one, as no-one ever does what would be needed to
trigger a problem. Unfortunately, the libtool issue here causes it to do
what "no-one ever does" and seems to have triggered what looks like it
is an off by one memory management problem in sh (which I will now find
and fix, since I now know exactly what input triggers it, sometimes anyway).
At this point, if you don't want to know all the gory details, and believe
me, you don't, you can skip to the patch at the end (if someone else hasn't
already arranged to apply it to pkgsrc) and forget all the rest of this e-mail.
(Nb: none of this is about whatever the sh bug is, that I haven't even started
looking for yet).
STOP READING, you have been warned!
OK, for however many masochists we have still reading, it turns out that
there's this huge long echo command in the libtool script (generated in
gettext from build-aux/ltmain.sh which is what gets patched) which ends
up installed in ${OBJDIR}/gettext-tools/libtool and is then used to generate
the wrapper scripts which were failing. I never managed to find out how
generating gettext-tools/libtool happens (I wasted too much time looking) -
and now I no longer care.
That echo command is:
$ECHO "\
# A function that is used when there is no print builtin or printf.
func_fallback_echo ()
{
eval 'cat <<_LTECHO_EOF
\$1
_LTECHO_EOF'
}
ECHO=$qECHO
fi
# Very basic option parsing. These options are (a) specific to
# the libtool wrapper, (b) are identical between the wrapper
# /script/ and the wrapper /executable/ that is used only on
# windows platforms, and (c) all begin with the string "--lt-"
# (application programs are unlikely to have options that match
# this pattern).
#
# There are only two supported options: --lt-debug and
# --lt-dump-script. There is, deliberately, no --lt-help.
#
# The first argument to this parsing function should be the
# script's $0 value, followed by "$@".
lt_option_debug=
func_parse_lt_options ()
{
lt_script_arg0=\$0
shift
for lt_opt
do
case \"\$lt_opt\" in
--lt-debug) lt_option_debug=1 ;;
--lt-dump-script)
lt_dump_D=\`\$ECHO \"X\$lt_script_arg0\" | $SED -e 's/^X//' -e 's%/[^/]*$%%'\`
test \"X\$lt_dump_D\" = \"X\$lt_script_arg0\" && lt_dump_D=.
lt_dump_F=\`\$ECHO \"X\$lt_script_arg0\" | $SED -e 's/^X//' -e 's%^.*/%%'\`
cat \"\$lt_dump_D/\$lt_dump_F\"
exit 0
;;
--lt-*)
\$ECHO \"Unrecognized --lt- option: '\$lt_opt'\" 1>&2
exit 1
;;
esac
done
# Print the debug banner immediately:
if test -n \"\$lt_option_debug\"; then
echo \"$outputname:$output:\$LINENO: libtool wrapper (GNU $PACKAGE) $VERSION\" 1>&2
fi
}
# Used when --lt-debug. Prints its arguments to stdout
# (redirection is the responsibility of the caller)
func_lt_dump_args ()
{
lt_dump_args_N=1;
for lt_arg
do
\$ECHO \"$outputname:$output:\$LINENO: newargv[\$lt_dump_args_N]: \$lt_arg\"
lt_dump_args_N=\`expr \$lt_dump_args_N + 1\`
done
}
# Core function for launching the target application
func_exec_program_core ()
{
"
And yes, that is all one long double quoted string, a single arg passed
to $ECHO (which is just a way the script uses to get an echo command which
works as desired - that's fine).
In that, the line:
test \"X\$lt_dump_D\" = \"X\$lt_script_arg0\" && lt_dump_D=.
is what was being corrupted in the places the problem was observed,
but I suspect that's pure chance, it just happens to be the right
number of bytes into that long string to trigger whatever the problem is.
Now pay particular attention to these lines, which form part of that string:
# Very basic option parsing. These options are (a) specific to
# the libtool wrapper, (b) are identical between the wrapper
# /script/ and the wrapper /executable/ that is used only on
# windows platforms, and (c) all begin with the string "--lt-"
# (application programs are unlikely to have options that match
# this pattern).
#
# There are only two supported options: --lt-debug and
# --lt-dump-script. There is, deliberately, no --lt-help.
#
# The first argument to this parsing function should be the
# script's $0 value, followed by "$@".
and the problem is obvious, isn't it?
In there there are two pairs of "" and two uses of '$' which aren't
escaped, which means in a double quoted string, the first " of each "" pair
ends the double quoting, and the second starts it again, and the unescaped
$'s cause expansion to happen as part of generating the args to echo, rather
than generating a literal '$' which is clearly intended there.
I can imagine what the author was thinking - these are all comments in
the eventual script, it doesn't really matter what they say, no-one will
ever look at it anyway.
In one of the scripts generated using an old(ish) version of sh (before
the recent changes), the trailing 2 lines of that leading comment end up
being:
66 # The first argument to this parsing function should be the
67 # script's ../libtool value, followed by no.
(and there I quote from prlw1's message to pkgsrc-users (and me)).
That 2nd line should be saying
67 # script's $0 value, followed by "$@".
As it is, the (closing) " before $@ (an opening one follows it) caused
unquoted $@ expansion in a field splitting context, which is exactly the
area of the recent change to sh ($@ and $* are the same thing in that usage,
they're interchangeable).
That unquoted $@ expansion in the middle of this long word (which could
potentially be generating multiple words - echo wouldn't care) seems to
be triggering whatever memory management bug I managed to add into sh,
or possibly was there all the time, just hidden, I don't know yet.
That's what (I believe) explains the stray '(' which was getting inserted
further down in the string (it will probably be at the point where the shell's
string buffer space runs out, and it needs to allocate more for the long
string).
While I'm here, I will also point out (in case anyone wants to forward this
to gettext or libtool maintainers) that the comment in question:
# The first argument to this parsing function should be the
# script's $0 value, followed by "$@".
is nonsense, the first arg to the function can be anything, it isn't
used at all. The generated function starts:
69 func_parse_lt_options ()
70 {
71 lt_script_arg0=$0
72 shift
(again from prlw1's e-mail, just because that's easier for me).
You can see there the "shift" is performed before $1 is accessed.
Whatever was the first arg to to function is simply thrown away, unused.
My guess (and it is just that) is that perhaps whoever designed this
function believed that
lt_script_arg0=$0
was accessing one of the positional params, probably the first, and
believes that the shift which follows is removing that one from further
consideration. It isn't, $0 is a special param, fixed for the duration
of the shell from when it is first invoked, it is global, constant (in the
script) and not a fuunction parameter. Probably uses of the
func_parse_lt_options pass "$0" as the first arg. That's fine, harmless,
useless, it simply is not used.
kre
Patch follows:
--- build-aux/ltmain.sh.ORIG 2022-09-11 19:52:56.000000000 +0700
+++ build-aux/ltmain.sh 2026-03-25 17:50:57.948867219 +0700
@@ -5531,7 +5531,7 @@
# Very basic option parsing. These options are (a) specific to
# the libtool wrapper, (b) are identical between the wrapper
# /script/ and the wrapper /executable/ that is used only on
-# windows platforms, and (c) all begin with the string "--lt-"
+# windows platforms, and (c) all begin with the string \"--lt-\"
# (application programs are unlikely to have options that match
# this pattern).
#
@@ -5539,7 +5539,7 @@
# --lt-dump-script. There is, deliberately, no --lt-help.
#
# The first argument to this parsing function should be the
-# script's $0 value, followed by "$@".
+# script's \$0 value, followed by \"\$@\".
lt_option_debug=
func_parse_lt_options ()
{