Re: standards/42828: Almquist shell always evaluates the contents of ${ENV} even if non-interactive

[Robert Elz <kre%munnari.OZ.AU@localhost>]

The following reply was made to PR standards/42828; it has been noted by GNATS.

From: Robert Elz <kre%munnari.OZ.AU@localhost>
To: Richard Hansen <rhansen%bbn.com@localhost>
Cc: gnats-bugs%NetBSD.org@localhost, standards-manager%NetBSD.org@localhost,
        netbsd-bugs%NetBSD.org@localhost
Subject: Re: standards/42828: Almquist shell always evaluates the contents of 
${ENV} even if non-interactive
Date: Tue, 23 Feb 2010 21:30:38 +0700

     Date:        Mon, 22 Feb 2010 18:07:46 -0500
     From:        Richard Hansen <rhansen%bbn.com@localhost>
     Message-ID:  <4B830E42.9060504%bbn.com@localhost>
 
   | I would like to reach consensus on whether the benefits of fixing the 
   | annoyance outweigh the costs.
 
 Yes, that would be good - and that's never going to happen as long as it is
 just the two of us having this discussion, so after this message I'm going to
 leave it for others to give an opinion, and will (probably) say no more.
 
   | This criticism brings up a broader question:  How much should NetBSD 
   | care about POSIX compliance?
 
 I suspect my opinion on that one is clear, so I'll dodge that bullet...
 
   | I like to distinguish "design flaw" from "implementation flaw".
 
 That's reasonable, but I'm not sure it is material.
 
   | I don't think there is a requirement to evaluate ${ENV} for 
   | non-interactive shells.
 
 My take is that if ${ENV} isn't wanted for non-interactive shells, it can
 make itself a no-op.   On the other hand, if it is, other than adding some
 other way to do the same thing to the shell, there is no remedy.
 
   | Let me rephrase by using an example:  Suppose a user needed to change 
   | the behavior of one particular shell script.  In NetBSD, there are three 
   | ways this might be done:
   | 
   |    1.  modify the script
   |    2.  via ${ENV}
   |    3.  change the invoker's environment (e.g., export environment 
   | variables, open/close file descriptors, etc.) before running the script
   | 
   | Option #1 may not be available.  Maybe the file is not writable, or 
   | maybe modifications are too impractical (e.g., upgrading the package 
   | providing the script will revert any changes).
 
 Yes, and in cases that sometimes matter, it isn't clear which script needs
 to be modified in any case - the whole point of using ${ENV} is so the
 modifications apply to every script, which might then allow the actual
 change that's needed to be detected.
 
   | The big question:  Do options #1 and #3 provide enough flexibility in 
   | practice?
 
 No, or not without considerable work (one can always replace /bin/sh
 with something else, that does whatever is required, and then exec's the
 real /bin/sh from its new location - but that's rarely convenient...)
 
   | Certainly options #1 and #3 are insufficient in come cases, but I 
   | would argue that those cases are rare and the result of an exceptional 
   | flaw that should be dealt with in another way.
 
 What other way?
 
   | This is an example of where NetBSD's behavior is handy but not needed. 
   | If this bug was fixed, you could still make your scripts work.
 
 Yes, but ...
 
   | For this particular example, current behavior vs. fixed is a tradeoff 
   | between having to remember to add the following at the top of the ${ENV} 
   | file:
   | 
   |    case "$-" in *i*);; *) return;; esac
   | 
   | versus having the remember to add the following to the scripts:
   | 
   |    [ -r "${ENV}" ] && . "${ENV}"
   | 
   | Either one is annoying, but fixing it is standards compliant and won't 
   | cause your login shell to lock up if you forget the magic line.
 
 My login shell (nor any other shell) has never locked up, so that isn't
 a problem I'm particularly worried about - perhaps just because I'm a little
 more conservative about what I have the shell do in ${ENV}
 
 But for those two remedies, the former (modifying ${ENV}) you do exactly
 once (once in all time) - the other (adding a line to the script) needs to
 be repeated for every script that is created, which can be several times
 a day.
 
 That is, this is hardly a "add a line here vs add a line there, which is
 better?" debate (if it were, that's exactly the kind of place where following
 the standards - ie: doing what others have already decided is best - is the
 better way).   The alternatives you're suggesting are not comparable that
 way however, there's a big difference between a one off, and a repeat forever. 
 
   | This is another example of where NetBSD's behavior is handy but not 
   | needed.  The same results can be achieved by prepending a special 
   | directory to PATH and populating that directory with custom versions of 
   | standard commands.
 
 No, that doesn't work, built in commands are executed before commands
 found in $PATH, so that doesn't work to override stuff like test, let
 alone the built in commands that can't go in $PATH (like cd) which happens
 to be one of the commands I most often want to override.   That can be
 done by defining a cd() function, and that works in ${ENV}.
 
   |    * working directory:  Initially the current directory matches the 
   | working directory of the invoker.  If the user adds 'cd "${HOME}"' to 
   | the ${ENV} file, then every shell script will assume it was invoked from 
   | ${HOME}.
 
 Of course, and it would be, and if that's what the user wanted to happen
 (which we can assume they must have when they put that in ${ENV}) then
 what's the problem?   If that causes things to break, the user who added
 the cd can remove it (or make it conditional).
 
   |    * positional parameters:  Initially the positional parameters match 
   | the command line arguments passed by the invoker.  If the user adds 
   | 'shift' somewhere in the ${ENV} file, all shell scripts will ignore the 
   | first command line argument.
 
 But those parameters all depend upon how the script is invoked anyway?
 I can just invoke any script without its first parameter, can't I?
 The script needs to handle that possibility in some rational way, and if
 it does that, then I don't see that it matters that ${ENV} could do the
 same thing (though in normal circumstances, I don't really see that
 as a problem I'd typically worry about - that is, I don't think it's
 very likely that a normal ${ENV} will just randomly decide that a shift
 or two might be nice...)
 
 On the other hand, the ability to rearrange parameters allows for version
 shift variances to be handled: if you have two packages X and Y. where X
 needs stuff from Y, upgrades are available for both, and the new X wants
 the new Y, but while you need the new X (bug fixes, or new features) you
 don't want the new Y (platform not supported, or has new bugs that break
 your work, or just cost) then you might (OK, it is a stretch) be able to
 massage the exec from X into a script from Y to rearrange the args to
 be what the older Y expects rather than what the newer X is handing it.
 
 Perhaps not very plausible, but it is the kind of thing that only works when
 it is possible to alter the environment of every script that runs.
 
   |    * shell options:  Initially they're all turned off.  If the user adds 
   | 'set -C' to ${ENV}, '>' will no longer work as expected for every shell 
   | script.
 
 And if I run the script as "sh -C script" ???
 Or for that matter, if I run the script as "sh -i script" ?
 
 Options are only all off initially if the script is invoked as "./script"
 (or whatever) and even that's only true if the #! line doesn't set any
 
        #! /bin/sh -C
 
 should work (but that's not relevant to our discussion).
 
   | Shell scripts could use $- to test for sane shell options, but none of 
   | them do.  (Why would they?  The POSIX spec says they're all turned off 
   | by default.)
 
 By default, yes, but the script is not required to be run "by default".
 And while I totally understand that there's no particularly good reason
 that a script author needs to necessarily anticipate some idiot running
 a script with "sh -C script" when the script doesn't expect noclobber mode,
 there's also no real reason to anticipate that someone will do set -C
 in ${ENV} (outside a test for an interactive shell).
 
   | Shell scripts can't tell if the current working directory 
   | or positional parameters were modified so there's nothing to test or set.
 
 They cannot tell that they were modified, but nor can they control the
 environment from which they were invoked.   If a script needs to be in
 a particular directory, or to have some options set, or reset, it needs
 to make those things happen itself.   It always needs to test its parameters
 for sanity (correct number, types, ...)   If a script just wants to operate
 upon whatever happens to be ${PWD} when it starts, it should just do that,
 and whether that was altered as a result of
        (cd /some/where && ./script)
 or as a result of
        case "$0" in script) cd /some/where;; esac
 in $ENV shouldn't matter very much, should it?
 
   | If a user's ${ENV} file follows the POSIX spec,
 
 Local system documentation trumps the POSIX spec every time.   Remember
 here you're asking for NetBSD to alter its documented behaviour, not to
 alter the way some undocumented event is handled, nor to actually comply
 with what it documents about itself.
 
   | I do not mean for respect to entail capitulation, only that the tone be 
   | professional and not condescending or dismissive.
 
 I apologise if I ever sounded condescending, that wasn't the intent.
 However, I don't think your proposal should be implemented, so dismissive,
 yes - but that's just me, and I have nothing more than an opinion here,
 nothing I can do can do more than perhaps influence others (I don't get to
 make the change, or prevent it.)
 
 kre
 
 ps: If there's anyone else bothering to read all of this stuff, sometime
 around now would be a good time to offer opinions.