audio/eawpatches distfile wreaking havoc

[Hans Rosenfeld <hans%netbsd.org@localhost>]

Hi,

I ran into an odd problem with audio/eawpatches the other day when I was
building packages on my OpenIndiana system. I originally wanted to file
a bug for this, but after digging in I'm not so sure anymore whose fault
all of this is. But let's start at the beginning.

While building a binary package of audio/eawpatches worked fine, the
resulting package could not be installed:

# pkg_add /usr/pkgsrc/packages/All/eawpatches-12nb3.tgz
pkg_add: eawpatches-12nb3: failed to read from archive: (null)
pkg_add: Couldn't remove /usr/pkg/share/eawpats/charang.pat: No such file or directory
...
pkg_add: 1 package addition failed

As the generated package archive can be read just fine and can also be
untarred with bsdtar with no issue whatsoever, I really wanted to know
what pkg_add was up to. It turns out that share/eawpats/charang.pat
comes with a mtime=-1 header in the package archive, which isn't invalid
strictly speaking but libarchive.so`pax_attribute_read_time() returns
ARCHIVE_WARN for it anyway (without setting a descriptive error message,
which IMHO is a bug).

Consequently, pkg_add stops there as it really expects ARCHIVE_OK all
the time. (And then it tries to rollback what it has done so far, and
more, hence the boatload of error messages about being unable to remove
files it didn't even extract yet.)

Trying to figure out where this came from, I took a closer look at the
package build output again, and I spotted this:

$ bmake package
...
===> Extracting for eawpatches-12nb3
/usr/bin/gtar: eawpats/charang.pat: implausibly old time stamp 1969-12-31 23:59:59

Ok, so apparently I'm using GNU tar to extract distfiles because this is
on illumos. And it seems GNU tar doesn't like the time stamp on that
file either, so the problem originated in the distfile.

So I tried extracting the distfile using different methods:

GNU tar 1.35:
$ gtar xpf eawpats12_full.tar.gz
gtar: eawpats/charang.pat: implausibly old time stamp 1969-12-31 23:59:59
$ stat -c %y eawpats/charang.pat
1969-12-31 23:59:59.000000000 +0000

bsdtar 3.4.3: (libarchive 3.4.3 zlib/1.2.13 liblzma/5.4.0 bz2lib/1.0.8)
$ bsdtar xpf eawpats12_full.tar.gz
$ stat -c %y eawpats/charang.pat
1969-12-31 23:59:59.000000000 +0000

illumos pax:
$ gzcat eawpats12_full.tar.gz | pax -r
$ stat -c %y eawpats/charang.pat
1969-12-31 23:59:59.000000000 +0000

illumos tar:
$ tar xpf eawpats12_full.tar.gz
$ stat -c %y eawpats/charang.pat
1995-05-31 20:58:50.000000000 +0000

So, GNU tar is the only one that warns about the timestamp. And illumos
tar gets a completely different timestamp!

That's when I looked closer at the original distfile. Here's a hexdump
of the tar header for eawpats/charang.pat:

00e62a00  65 61 77 70 61 74 73 2f  63 68 61 72 61 6e 67 2e  |eawpats/charang.|
00e62a10  70 61 74 00 00 00 00 00  00 00 00 00 00 00 00 00  |pat.............|
00e62a20  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00e62a60  00 00 00 00 30 31 30 30  36 34 34 00 30 30 30 30  |....0100644.0000|
00e62a70  30 30 30 00 30 30 30 31  37 35 30 00 30 30 30 30  |000.0001750.0000|
00e62a80  33 33 31 32 30 34 35 00  ff ff ff ff ff ff ff ff  |3312045.........|
00e62a90  ff ff ff ff 30 32 30 32  34 33 00 20 30 00 00 00  |....020243. 0...|
00e62aa0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00e62b00  00 75 73 74 61 72 20 20  00 72 6f 6f 74 00 00 00  |.ustar  .root...|
00e62b10  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00e62b20  00 00 00 00 00 00 00 00  00 73 69 6c 6b 00 00 00  |.........silk...|
00e62b30  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

So this looks like it was created in the GNU format, using "ustar  "
magic in the place of the magic and version fields specified by POSIX.
The mtime field certainly looks odd, containing only 12 0xff bytes.
libarchive tries to parse this as a base-256 integer and gets -1 for the
mtime.

All other files in the distfile have proper mtimes, so this was probably
done on purpose. I guess I could go hunt old GNU tar source and see if
it ever wrote -1 into the mtime field, and for which reason, but I'm
not sure that's worth the hassle. The distfile is what it is, and there
doesn't seem to be any sort of proper "upstream" for this package that
could generate a new distfile to get rid fo this issue.

So, I'd like to suggest putting a workaround for the corrupted distfile
into audio/eawpatches/Makefile, touching charang.pat at post-extract to
set its mtime explicitly using the changelog.txt file asa reference.
Would anyone object to that?

Also, I wonder whether pkg_add should be changed to be more relaxed
about ARCHIVE_WARN, based on the assumptions that warnings aren't errors
for a reason.

On the other hand, it's probably not unreasonable for pkg_add to insist
that binary packages created by pkgsrc should never cause warnings when
being extracted, but in that case pkg_add should be fixed to be a bit
more helpful about it. Which, in turn means fixing libarchive to provide
an explanation for returning ARCHIVE_WARN here. I think there are a few
more similar situations in libarchive when a warning or error is
returned, but no message is given.

Any opinions?


Hans


-- 
%SYSTEM-F-ANARCHISM, The operating system has been overthrown