Re: standards/42828: Almquist shell always evaluates the contents of ${ENV} even if non-interactive

[Robert Elz <kre%munnari.OZ.AU@localhost>]

    Date:        Mon, 22 Feb 2010 18:07:46 -0500
    From:        Richard Hansen <rhansen%bbn.com@localhost>
    Message-ID:  <4B830E42.9060504%bbn.com@localhost>

  | I would like to reach consensus on whether the benefits of fixing the 
  | annoyance outweigh the costs.

Yes, that would be good - and that's never going to happen as long as it is
just the two of us having this discussion, so after this message I'm going to
leave it for others to give an opinion, and will (probably) say no more.

  | This criticism brings up a broader question:  How much should NetBSD 
  | care about POSIX compliance?

I suspect my opinion on that one is clear, so I'll dodge that bullet...

  | I like to distinguish "design flaw" from "implementation flaw".

That's reasonable, but I'm not sure it is material.

  | I don't think there is a requirement to evaluate ${ENV} for 
  | non-interactive shells.

My take is that if ${ENV} isn't wanted for non-interactive shells, it can
make itself a no-op.   On the other hand, if it is, other than adding some
other way to do the same thing to the shell, there is no remedy.

  | Let me rephrase by using an example:  Suppose a user needed to change 
  | the behavior of one particular shell script.  In NetBSD, there are three 
  | ways this might be done:
  | 
  |    1.  modify the script
  |    2.  via ${ENV}
  |    3.  change the invoker's environment (e.g., export environment 
  | variables, open/close file descriptors, etc.) before running the script
  | 
  | Option #1 may not be available.  Maybe the file is not writable, or 
  | maybe modifications are too impractical (e.g., upgrading the package 
  | providing the script will revert any changes).

Yes, and in cases that sometimes matter, it isn't clear which script needs
to be modified in any case - the whole point of using ${ENV} is so the
modifications apply to every script, which might then allow the actual
change that's needed to be detected.

  | The big question:  Do options #1 and #3 provide enough flexibility in 
  | practice?

No, or not without considerable work (one can always replace /bin/sh
with something else, that does whatever is required, and then exec's the
real /bin/sh from its new location - but that's rarely convenient...)

  | Certainly options #1 and #3 are insufficient in come cases, but I 
  | would argue that those cases are rare and the result of an exceptional 
  | flaw that should be dealt with in another way.

What other way?

  | This is an example of where NetBSD's behavior is handy but not needed. 
  | If this bug was fixed, you could still make your scripts work.

Yes, but ...

  | For this particular example, current behavior vs. fixed is a tradeoff 
  | between having to remember to add the following at the top of the ${ENV} 
  | file:
  | 
  |    case "$-" in *i*);; *) return;; esac
  | 
  | versus having the remember to add the following to the scripts:
  | 
  |    [ -r "${ENV}" ] && . "${ENV}"
  | 
  | Either one is annoying, but fixing it is standards compliant and won't 
  | cause your login shell to lock up if you forget the magic line.

My login shell (nor any other shell) has never locked up, so that isn't
a problem I'm particularly worried about - perhaps just because I'm a little
more conservative about what I have the shell do in ${ENV}

But for those two remedies, the former (modifying ${ENV}) you do exactly
once (once in all time) - the other (adding a line to the script) needs to
be repeated for every script that is created, which can be several times
a day.

That is, this is hardly a "add a line here vs add a line there, which is
better?" debate (if it were, that's exactly the kind of place where following
the standards - ie: doing what others have already decided is best - is the
better way).   The alternatives you're suggesting are not comparable that
way however, there's a big difference between a one off, and a repeat forever. 

  | This is another example of where NetBSD's behavior is handy but not 
  | needed.  The same results can be achieved by prepending a special 
  | directory to PATH and populating that directory with custom versions of 
  | standard commands.

No, that doesn't work, built in commands are executed before commands
found in $PATH, so that doesn't work to override stuff like test, let
alone the built in commands that can't go in $PATH (like cd) which happens
to be one of the commands I most often want to override.   That can be
done by defining a cd() function, and that works in ${ENV}.

  |    * working directory:  Initially the current directory matches the 
  | working directory of the invoker.  If the user adds 'cd "${HOME}"' to 
  | the ${ENV} file, then every shell script will assume it was invoked from 
  | ${HOME}.

Of course, and it would be, and if that's what the user wanted to happen
(which we can assume they must have when they put that in ${ENV}) then
what's the problem?   If that causes things to break, the user who added
the cd can remove it (or make it conditional).

  |    * positional parameters:  Initially the positional parameters match 
  | the command line arguments passed by the invoker.  If the user adds 
  | 'shift' somewhere in the ${ENV} file, all shell scripts will ignore the 
  | first command line argument.

But those parameters all depend upon how the script is invoked anyway?
I can just invoke any script without its first parameter, can't I?
The script needs to handle that possibility in some rational way, and if
it does that, then I don't see that it matters that ${ENV} could do the
same thing (though in normal circumstances, I don't really see that
as a problem I'd typically worry about - that is, I don't think it's
very likely that a normal ${ENV} will just randomly decide that a shift
or two might be nice...)

On the other hand, the ability to rearrange parameters allows for version
shift variances to be handled: if you have two packages X and Y. where X
needs stuff from Y, upgrades are available for both, and the new X wants
the new Y, but while you need the new X (bug fixes, or new features) you
don't want the new Y (platform not supported, or has new bugs that break
your work, or just cost) then you might (OK, it is a stretch) be able to
massage the exec from X into a script from Y to rearrange the args to
be what the older Y expects rather than what the newer X is handing it.

Perhaps not very plausible, but it is the kind of thing that only works when
it is possible to alter the environment of every script that runs.

  |    * shell options:  Initially they're all turned off.  If the user adds 
  | 'set -C' to ${ENV}, '>' will no longer work as expected for every shell 
  | script.

And if I run the script as "sh -C script" ???
Or for that matter, if I run the script as "sh -i script" ?

Options are only all off initially if the script is invoked as "./script"
(or whatever) and even that's only true if the #! line doesn't set any

        #! /bin/sh -C

should work (but that's not relevant to our discussion).

  | Shell scripts could use $- to test for sane shell options, but none of 
  | them do.  (Why would they?  The POSIX spec says they're all turned off 
  | by default.)

By default, yes, but the script is not required to be run "by default".
And while I totally understand that there's no particularly good reason
that a script author needs to necessarily anticipate some idiot running
a script with "sh -C script" when the script doesn't expect noclobber mode,
there's also no real reason to anticipate that someone will do set -C
in ${ENV} (outside a test for an interactive shell).

  | Shell scripts can't tell if the current working directory 
  | or positional parameters were modified so there's nothing to test or set.

They cannot tell that they were modified, but nor can they control the
environment from which they were invoked.   If a script needs to be in
a particular directory, or to have some options set, or reset, it needs
to make those things happen itself.   It always needs to test its parameters
for sanity (correct number, types, ...)   If a script just wants to operate
upon whatever happens to be ${PWD} when it starts, it should just do that,
and whether that was altered as a result of
        (cd /some/where && ./script)
or as a result of
        case "$0" in script) cd /some/where;; esac
in $ENV shouldn't matter very much, should it?

  | If a user's ${ENV} file follows the POSIX spec,

Local system documentation trumps the POSIX spec every time.   Remember
here you're asking for NetBSD to alter its documented behaviour, not to
alter the way some undocumented event is handled, nor to actually comply
with what it documents about itself.

  | I do not mean for respect to entail capitulation, only that the tone be 
  | professional and not condescending or dismissive.

I apologise if I ever sounded condescending, that wasn't the intent.
However, I don't think your proposal should be implemented, so dismissive,
yes - but that's just me, and I have nothing more than an opinion here,
nothing I can do can do more than perhaps influence others (I don't get to
make the change, or prevent it.)

kre

ps: If there's anyone else bothering to read all of this stuff, sometime
around now would be a good time to offer opinions.