Re: shell quoting

[Robert Elz <kre%munnari.OZ.AU@localhost>]

    Date:        Wed, 9 Sep 2026 17:28:56 +0200
    From:        Edgar Fu�<ef%math.uni-bonn.de@localhost>
    Message-ID:  <aqF7OGfLoEy+r9Nk%trav.math.uni-bonn.de@localhost>

  | This is getting interesting.

Really?

  | So for the $1="a b" case, this makes
  | 	foo=$1
  | behave diffently from
  | 	local foo=$1
  | and
  | 	export foo=$1

Absolutely, it always has been.

  | Of course one could argue (and I would expect KRE to argue that way) that 
  | yes, that's the way sh works, and everyone In The Know would expect this,
  | and
  | 	assign foo=$1
  | would also behave that way no matter how "assign" is defined

For one second, just imagine that the "assign" you invented was "dd" instead,
and think of how you'd expect

	dd if=file ...

to behave.   Are you expecting that anything related to being able (anywhere)
expand ${if} (or ${of} or ${conv} if the similarity of "if" to the sh reserved
word is causing any confusion) and get "file" as the result).   If not, what
about the name (dd vs assign or export) makes one of them special, compared
to the others?   Do you really want special cases like that?

  | (well, I would't bet KRE couldn't make up a definition that makes
  | it work the assignment way, but I myself can't).

You should have, you'd have won, as apart from re-inventing POSIX's
declaration utilities, which I simply would not do, there is no way.

  | Another point is that That GNU Monstrosity behaves the way I would (sorry) 
  | have expected.

That GNU monstrosity has declaration utilities, which were invented precisely
to make all this happen the way people who refuse to learn the rules & idioms
of the language expect it to just work "because it does in this other 
language".

  | > and probably never will, there is no reason to do it
  | Depending on your mindset, it would follow POLA.

Yes - the point is that convoluting mindsets, by trying to make everything
the same, removes the reason for everything to exist in the first place.
If there was one perfect programming language, that was ideal for every
possible purpose one could imagine, then that one would be all that would
exist.   So far, no-one has found that one.

ef%math.uni-bonn.de@localhost said:
  | P.S. I'm inclined to say the root cause for all this mess (as I regard it)
  | is assignments being magic because someone decided to save four keystrokes
  | (s, e, t, SP). 

You're right, they are magic, to an extent, but not because anyone was
trying to save keystrokes - that you'd think that suggests that you believe
that
	foo=bar
is a command in sh.    It isn't, and never has been.

The shell "simple command" (simple as to distinguish it from all the
more complicated variants, like "if" and subshells, and ...) is of the
form
	name arg arg ...

In other systems (before the Thompson shell changed it all) that was the
common format for everything.   The revolution was to move a lot of the
processing out of every "name" command, and into the shell, so an arg
like *.c is expanded by the shell, rather than having every different "name"
to it in its own peculiar way (which comes with the disadvantage that no
special "name" can do it in its own peculiar way, even when that would seem
to be desirable - other systems can do (more or less)
	mv *.c old-*.c
for example, to rename a set of files in a common way).

Similarly, a construction like "> filename" in the arg list (or even before
the name) is handled by the shell to allow output to be directed to filename,
rather than every "name" command having its own peculiar way to allow that
kind of thing (which also has its drawbacks, what happens with "cat * > file"
depends just where "file" appears in the expansion of '*' - but filling the
local filesystem is the most common result).

And third, sh (though this one waited for the Bourne shell to appear,
unlike the previous two), a "name" command can be given local config
environment variables before the "name" that is the command - not after,
as then there would be no way to distinguish them from args (as in the "dd"
case) where such args just happen to look like variable assignments, but
aren't.   That is to write

	NAME=whatever name arg ...

(where there can be as many NAME=whatever prefixes to the command as required).

This is a standard way to supply little used (as in varied) data to a command
rather than ever command needing to invent its own way (cf make, and awk).

And of course, like everything else, this has its drawbacks, the obvious
one is that now, commands with names of the form name=anything no longer
simply work by just writing their names (so foo=bar cannot be a command,
not that anyone really cares about that issue) but more than that, it alters
the way that "whatever" is processed compared the same "NAME=whatever" if
it were placed after the command name rather than before it.

Now, once having altered the world in this way, something needed to be
done to define what would happen if one of these standardised constructions
was used without a command "name" at all.

The filename expansion variant is simple, if one writes just

	foo*

instead of something like

	cat foo*

then the foo* is expanded, and the first result becomes the command name,
and the remaining expansions are its args.   While not a common usage (of
the 3, this is probably the least often used), it can occasionally be useful.

In a directory one can have a bunch of executable files (scripts, or compiled
code, makes no difference) foo bar baz (no, they don't all need to be three
letters) and a bunch of data files for each foo-1 foo-2 foo-abc ... (and
similarly for bar, and baz, and anything else), and then just

	foo*
or	bar*

to run:
	foo foo-1 foo-2 foo-abc ...

and similarly for the others.   Yes, seeing a practical use for this is
kind of difficult, but it is the natural definition of what should happen,
and isn't completely useless.

For redirects, the Thompson shell simply made them be a syntax error, if
no accompanying command existed.   The Bourne shell changed that, and
allowed them, they are simply evaluated for their side effects.   So,
a
	<file

(or "< file", the white space is irrelevant) opens "file" for reading,
generating an error if "file" cannot be opened, then simply closes it again).

Similarly
	>file
opens "file" for writing, checking that can be done (no permission problems,
etc) but also, truncates "file" to 0 size, the more common use for that
construction, and then closes it again.

For var-assigns (which didn't exist in the Thompson shell, which didn't
have variables to assign to at all) the same kind of thing could have been
done

	foo=bar

could simply have tested that it was possible to do that assignment, but given
the nature of sh assignments, the only way it can fail is if "foo" is readonly,
so that's an unlikely thing to want to achieve (unlike the redirect case).
So, for that one, the assignment in this case was made permanent in the shell
environment (how one assigns to a variable for the script) and is so common,
that it is often believed to be a command - but it just isn't (and given
that one can write

	a=1 b=2 c=3

if "a=1" were a command, then the remainder of the line would need to be
its args, and ...   no, let's just not go there.

How much of that was deliberate design in the original Bourne shell, and
how much was just an accident of the implementation is not clear to me.

That is, in

	var=value cmd arg1 arg2 >file

the way the shell works, is to expand the cmd and args, as needed, then
fork (ready to exec the cmd) and then process the redirect and var-assigns,
placing the var's assigned to into the environment, and altering the state
of the file descriptors, and then perform the exec).

The parent shell would see none of this (other than the arg expansion,
which is simply thrown away as soon as the fork() has happened) the
redirects would not affect it, and nor would the var-assigns.

But when the shell does not ever fork, things are different - then the
var-assigns affect the current shell environment, as do the redirects.
It was obvious that the redirect case needed special handling, to  not
have a lasting effect, so in the non-fork case, after the open, there
would be a close of the result, rather than "close(1); dup(fd); close(fd)"
which happens for >file assuming that's implemented as
	fd=creat("file", 0666)
which it would have been in 7th edition (before O_CREAT, O_TRUNC, etc
were all invented).

For var-assigns, just having the assignment remain in the shell environment
in this case seemed sane, so that's what was done, just not marked for
export into the environment (unless the -a option is enabled).

Now note that there are 2 cases where no fork() happens - the first when
there is no command name at all, which is largely what was assumed just
above, and the second case is when the command is a built-in command.
Initially, no distinction was made at all, if a redirect was used with
a built in command it behaved just the same way (open the file, run the
command, close the file) - which assuming things are done in a sane way,
turns out to be just what is wanted.   But for var-assigns, the default
action (keep the variable in the current shell) turned out to not be
such a good idea ... but for a long time, that was how it worked.  No-one
cared, as none of the original built in commands would use anything from
the environment (think "true" "false" "exit" ...) so placing variables
in the environment for one of those just was not something anyone ever
did.

When was the last time you ever wrote

	foo=bar true

for example?   What would be the point.

That all changed when CDPATH was invented, and it actually became reasonable
to do something like

	CDPATH=/a:/b cd foo

to cd to either /a/foo or /b/foo whichever exists.   If the assignment
simply persisted in the environment of the shell after this, which it did
initially, that would be no different than

	CDPATH=/a:/b
	cd foo

and affect all following cd commands, until CDPATH was altered again.
Not having that happen (it doesn't for fork()'d commands) was the whole
point of the prefix var-assigns, so that needed to be fixed.

The fixing added a whole extra set of complication to the way the shell
needed to work, it now needed the ability to temporarily set a variable,
and then revert it to the previous state after the built-in was finished.
And if by this stage you aren't seeing the parallel to "local" in functions
then you're not thinking properly, as that is the exact same thing, and is
often implemented using the same mechanism (the original Bourne shell had
no functions, hence no need for "local", and hence no mechanism anything
like this).

But fixing this for "cd" (and others similar) also corrected another anomaly
that had existed - previously the behaviour of

	foo=bar true

would vary depending upon whether "true" was built in or not.  That one,
and several other commonly built in commands are built in just for
efficiency, there is no other reason - that is, purely to avoid the
fork() exec() costs for such a simple operation.  Having the results
differ depending whether the shell implemented some command as a built-in
(to be faster) or not (to be smaller) really could not be considered as
anything but a bug, regardless of how rare it was for code to ever encounter
that problem (just because there is no point providing var-assigns for
any of those commands).   So fixing it was a good thing (even if messy).

But even now, the fix only applies to the "regular" built-in commands, for
the "special" ones (which include things like "break" "return" ...) a
var-assign before the command might persist in the shell environment.
For a time the standard actually required that it did, but that made no
sense, so now it is just allowed, rather than required.   But because it
used to be required, and still is permitted, our shell still implements
it that way - just no-one notices or cares, as no-one ever bothers to
write something like
	foo=bar break
(but in our sh, try it sometime, it really does leave "foo" set to "bar"
in the shell after the break is executed).

(Note, "." is a special built-in, and is the one of them, the only one,
where using a var-assign makes any sense at all).

So, all of this, to get into the state we're in now, is just a result of
sane development history, until:

uwe%stderr.spb.ru@localhost said:
  | Yes, that's a very old footgun.  IIRC, old shell didn't do `export
  | VAR=value`, only `export VAR`.

You do remember correctly.   The change to allow "export VAR=value"
rather than just "export VAR" (and yes, also local and readonly, and in
other shells, usually several more similar commands) was the start of
the problems.

There was never any reason to allow that change, it was simply pandering
to the demands of lazy programmers/users, who didn't want to have to
write the variable name twice, as in

	VAR=value; export VAR

which was the original way (same for readonly).   local is a little
different, as it (would have) required

	local VAR; VAR=value

which can't work for "readonly", which must be the other sequence, though
"export" works either order).

Once implementers start pandering to the demands of users for an "easier way"
without properly considering all of the ramifications, that's when problems
start to arise - and then when they do, rather than just reversing the original
mistake, which would be to admit making it, more invention (declaration 
utilities) is needed to patch things back up to be "reasonable" again.

More and more mess.

I'm not the kind to do much of that pandering (if you hadn't noticed), and
generally just ignore requests for fixes/changes to make our shell "easier"
to use or "more like that other one" - the one exception I can think of was
adding the "suspend" built-in, which was totally unnecessary, and probably
should just be removed again (though has no actual drawbacks I have seen
yet - it would override a filesystem "suspend" command which might do something
quite different, but as far as I'm aware, there isn't one).

On the other hand, I'm not a minimalist either, adding functionality which
can't be done other reasonable ways, and is useful, is always a possibility.

Reversing the "export VAR=value" thing is impossible (practically) now,
though the world would be better if we did - but it has been around for
so long that it would cause huge script breakage to undo it.   However,
we don't need the additional mess that declaration utilities add as an
attempt to clean up the mess that change causes - we just need to be aware
of it, and live with it.

kre

ps: the original Bourne shell had a "-k" option.  Look it up if you feel
you must.  That one was such a disaster (and useless) that is the one
original Bourne shell "feature" that has been abandoned completely.
I mention it here, as it is related to all of the above.