Re: bin/52458 (\n in prompt breaks history mechanism in /bin/sh)

[kre%NetBSD.org@localhost]

Synopsis: \n in prompt breaks history mechanism in /bin/sh

State-Changed-From-To: open->analyzed
State-Changed-By: kre%NetBSD.org@localhost
State-Changed-When: Sat, 05 Aug 2017 10:44:46 +0000
State-Changed-Why:
This is understood now ... the problem occurred with the PS1 expansion
code (etc.)   That works by treating the prompt (PS1 etc) as an input
string, "reading" it (as if we were inside double quotes, or a here
document that needs expanding) parsing the "line" read, expanding it
(much as if it were an arg on a command line) - then the expanded prompt
is eventually output.   Normally (no \n in the prompt) there's no issue
with any of that, but when there is a \n, the shell needs to make sure
no prompt is emitted, so it just turned the prompting off (this is
a variable with 3 states, "off" "use PS1" and "use PS2").

That by itself would all be fine, but it turns out that the shell
decides after it has read each line, whether to add that line to
the history list as a new entry, or whether to append it to the last
one, by looking to see which prompt it just used, if it was PS1,
then this is a new command line, if it was PS2, then we are continuing
the previous entry (a continuation of the command).   (This only
occurs when reading from the terminal, so prompting would generally
be on - note that PS1='' does not turn off prompting, it simply causes
there to be no characters to output in the prompt when it is issued...)

Where it got complicated was where when the prompt was being re-read
to be expanded, and contained a \n (which normally would be the time
for a new (PS2) prompt to be issued) prompting was turned off.  So,
after the prompt had been issued, instead of the variable saying "PS1
needed (or really, here, just used)" (which is what it had said) it now
says "no prompt" - then when the shell comes to add the line just read
to the history, the prompt did not say "PS1" so it assumed it must have
been "PS2", and appended the line to the previous event instead of making
a new one.

This can be fixed in either of two ways (I tested both, separately).

First, rather than testing "if PS1, new entry, else, append" change it to,
"If not PS2, new entry, else, append" (or if you like, "If PS2, append,
else new entry" which is the same thing with more code churn.)

Alternatively, when the prompt is being expanded, save, and then
restore, the "which prompt" variable, so after the expansion is
done, the state is unchanged.

I will do both ... (better safe!)

While here, I noticed a (semi-related) other bug though, empty
lines are not added to history.   That's fine (and a good thing)
if you're sitting at PS1, and just banging the return key to
avoid the screen saver (or something similar), it is OK, though
probably not really correct, in a command like

	ls |


	wc

where the empty lines did not get added to the history event,
but which does not really matter, as what did get added:

	ls |
	wc

means the same thing, but is completely broken when the command is

	echo 'abc


	def'

where the history would say

	echo 'abc
	def'

which is not the same thing at all.   This is an old bug, which has
been around for a long time, but which can be fixed in a very similar
way to the above ... if the prompt used is PS2, then we will enter
empty lines into the history event (append them) just the same way as
lines that contain something (nb: here, by "empty" I mean a line containing
nothing but the terminating \n, and that \n will be appended to the
history event.)