EXTRACT_USING improvements: a concrete proposal

[Greg Troxel <gdt%lexort.com@localhost>]

Looking at the code that sets and uses EXTRACT_USING, I am finding a few
problems.

1) nbtar is said to be "NetBSD's pax-as-tar", but nbtar causes TOOLS.tar
to be used (see extract/extract.mk line 201).

That is set to a variety of tar implementations, ranging from what's in
/bin/tar, to pathnames with gtar in them to ${TOOLS_PLATFORM.gtar}.
(grep for that in platform)

So this is kind of a mess, and nbtar seems to mean "some OS default",
not reliably finding a NetBSD pax-as-tar implementation like the name
and doc implies.

2) defaults/mk.conf sets EXTRACT_USING?= nbtar, but extract/extract.mk
does too, so while it's good to document, setting it seems unhelpful.

defaults/mk.conf is read before platform/ so this means setting
EXTRACT_USING in a platform file doesn't work.

So I propose:

  comment out the setting of EXTRACT_USING?=
  say in the comment that the default is platform specific

  set EXTRACT_USING?= bsdtar in platform/NetBSD.mk
  (Note that this gets set to /bin/tar via tool.NetBSD.mk on NetBSD
  systems that have bsdtar configured by /usr/share/mk+mk.conf, so
  people on 9/current should see no difference other than a tool name
  being printed.)

  fix comment about what nbtar means

  encourage people to set EXTRACT_USING?= in platform/* to something
  appropriate for that platform, e.g. native gtar on Linux and Solaris.

  start removing EXTRACT_USING from packages, at least if they are ok
  with bsdtar (assuming that nobody distributes archives that fail with
  GNU tar).

  (eventually, either remove the nbtar notion, or rename it to
  arbitrarytar so people don't expect anything in particular)

with the first three in the patch at end.

I did a bootstrap with the patch on NetBSD 8.  I then built devel/m4 (as
an arbitrary package that doesn't have that many dependencies) with the
bootstrap, which built digest and bsdtar, and all was ok.  So with only
the bootstrap packages installed, the process to force bsdtar worked
fine.

I'm pretty sure other-than-NetBSD is either 100% ok with this patch,
because if I leave out the platform/NetBSD.mk change, I get the old
behavior of nbtar.

I think this is ready to commit, and that if there are problems they
will be easy to spot, not mysterious.

Comments?  Objections?



Index: defaults/mk.conf
===================================================================
RCS file: /cvsroot/pkgsrc/mk/defaults/mk.conf,v
retrieving revision 1.306
diff -u -p -r1.306 mk.conf
--- defaults/mk.conf	12 Jan 2020 23:38:42 -0000	1.306
+++ defaults/mk.conf	11 Mar 2020 16:55:30 -0000
@@ -571,10 +571,10 @@ PATCH_FUZZ_FACTOR?= -F0
 # Possible: any combination of packages
 # Default: not set
 
-EXTRACT_USING?= nbtar
+#EXTRACT_USING?= nbtar
 # Use the specified tool to extract tar/ustar archives.
 # Possible: bsdtar, gtar, nbtar, pax
-# Default: nbtar
+# Default: platform specific
 
 #FAILOVER_FETCH=
 # Perform a checksum at "make fetch" time.  If the checksum doesn't
Index: extract/extract.mk
===================================================================
RCS file: /cvsroot/pkgsrc/mk/extract/extract.mk,v
retrieving revision 1.38
diff -u -p -r1.38 extract.mk
--- extract/extract.mk	7 May 2019 19:36:44 -0000	1.38
+++ extract/extract.mk	11 Mar 2020 16:55:30 -0000
@@ -31,7 +31,9 @@
 #    EXTRACT_USING specifies the tool used to extract tar/ustar-format
 #	archives when using EXTRACT_CMD_DEFAULT.  The possible values are
 #	"bsdtar", "gtar", "nbtar", and "pax".
-#	By default, we use the "nbtar" tool (pkgsrc's pax-as-tar).
+#	By default, we use the "nbtar", which means the value of
+#	${TOOL_PLATFORM.tar}, which is typically an arbitrary
+#	implementation already found on the platform.
 #
 #    EXTRACT_ELEMENTS is a list of files within the distfile to extract
 #	when using EXTRACT_CMD_DEFAULT.  By default, this is empty, which
Index: platform/NetBSD.mk
===================================================================
RCS file: /cvsroot/pkgsrc/mk/platform/NetBSD.mk,v
retrieving revision 1.58
diff -u -p -r1.58 NetBSD.mk
--- platform/NetBSD.mk	9 Jan 2019 13:19:03 -0000	1.58
+++ platform/NetBSD.mk	11 Mar 2020 16:55:30 -0000
@@ -13,6 +13,9 @@ PS?=		/bin/ps
 SU?=		/usr/bin/su
 TYPE?=		type				# Shell builtin
 
+# pax-as-tar, found on <=8, and optionally later, fails on many archives
+EXTRACT_USING?=	bsdtar
+
 USERADD?=	/usr/sbin/useradd
 GROUPADD?=	/usr/sbin/groupadd