Re: the zen of SIGPIPE

["Roland C. Dowdeswell" <elric%imrryr.org@localhost>]

On Sun, Jan 06, 2013 at 10:14:47PM -0500, James K. Lowden wrote:
>

FWIW, I agree with jkl that outputting ``xargs: <foo> terminated
by SIGPIPE'' is not terribly helpful.  I would make the argument,
however, in a slightly different way which is: what are we trying
to achieve by outputting that message to stderr and how does it
affect our ability to use xargs interactively and in scripts?

At first glance, it would seem that xargs will output the message
if the command after the pipe does not consume all of output
generated by the commands that it runs, however, this is incorrect.

E.g.:

$ jot 1 | xargs /usr/bin/printf "%s\n" | head -1   
1
$ jot 2 | xargs /usr/bin/printf "%s\n" | head -1
1
$ jot 3 | xargs /usr/bin/printf "%s\n" | head -1
1
.
.
.
$ jot 3643 | xargs /usr/bin/printf "%s\n" | head -1
1
$ jot 3644 | xargs /usr/bin/printf "%s\n" | head -1
1
$ jot 3645 | xargs /usr/bin/printf "%s\n" | head -1
1
xargs: /usr/bin/printf terminated by SIGPIPE

So, what is going on here?  Looks like we're only getting a SIGPIPE
if the buffer associated with stdout fills up blocking printf(1)
until head(1) exits after which time printf(1) tries to write again
leading to a SIGPIPE.

Another interesting example is:

$ jot 2 | xargs -n1 /usr/bin/printf "%s\n" | head -1
1
xargs: /usr/bin/printf terminated by SIGPIPE

I'd guess that in this case the fact that we're executing two printf(1)s
gives the head(1) time to exit before the second printf(1) writes to
stdout thus leading to a SIGPIPE.  If my guess is correct then it's
a race.

But then, if I do:

$ jot 2 | xargs -n1 /usr/bin/printf "%s\n" | ( head -1; sleep 1 )
1

I get no message as a different end of the pipe wins the race.

In short, from where I sit, it appears as if the diagnostic message
will be output indeterministically from the point of view of someone
writing scripts that may be run on many different systems as there
is a dependency on the sizes of buffers and a race condition.  This
limits the usefulness of the diagnostic significantly, given that
the absence of the message doesn't actually give you any information.
The message is sent to stderr which clutters the errors of the
underlying command and/or pipeline as well.

We should also recognise that xargs terminating as soon as a single
child receives SIGPIPE is quite useful behaviour in many situations
as it allows a level of lazy evaluation, that is only producing
output until it is no longer necesary.  But cluttering stderr with
messages if we use xargs in this way makes this option, let's say,
less attractive.  We could 2>/dev/null but that would hide real
errors.  Or we could consume all of the output like:

$ some_command | xargs another_command | ( head -1; cat >/dev/null )

but that would defeat the lazy evaluation.

I think that it would be more useful for xargs to raise(2) certain
signals which terminated its child rather than write to stderr.
This would leave the caller more flexibility to deal with it.

This is coincidentally how some shells deal with receiving SIGINT
to ensure that it propagates to parent shells so that they can,
e.g.  terminate loops.  Here's another example of xargs having
[unnecessarily?] different behaviour:

$ cat /tmp/foo
#!/bin/sh

echo "$@"
kill -2 $$
sleep 1
$ cat /tmp/adsf
foo bar baz
$ for i in $(jot 3); do /tmp/foo foo bar baz; done             
foo bar baz

$ for i in $(jot 3); do xargs /tmp/foo < /tmp/adsf; done
foo bar baz
xargs: /tmp/foo terminated by SIGINT
foo bar baz
xargs: /tmp/foo terminated by SIGINT
foo bar baz
xargs: /tmp/foo terminated by SIGINT

If xargs used raise(2) to propagate the SIGINT back to the parent,
both of those commands would behave in the same way.

All that said, it appears that the standards suggest that we keep
our current behaviour and so I am a bit divided.  I think that the
current behaviour is not optimal, however, we should be careful
about diverging from the standard as this may have unintended
consequences.

--
    Roland Dowdeswell                      http://Imrryr.ORG/~elric/