Re: www/serf install permissions fix

["J. Lewis Muir" <jlmuir%imca-cat.org@localhost>]

On 04/03, Greg Troxel wrote:
> Generally I would expect to see OS-specific variables in the per-OS
> config file, in mk/platform/Darwin.mk.

If that would be better, I can do that.  I initially put the change in
mk/defaults/mk.conf because that's where EXTRACT_USING was defined and
documented.

> But, I'm feeling that I don't quite grasp enough details.
> 
> You've shown that the native Mac /usr/bin/tar creates writable files.
> We more or less agree that this is wrong, but tar is no longer specified
> by POSIX.  (POSIX does specify pax, which I see you note below.)

I just want a fix that will be accepted so that I don't have to carry a
patch.

It does seem strange to me that macOS's /usr/bin/tar, when run as
root, preserves the file modes on extraction without the "-p" option.
However, maybe that's actually POSIX behavior, strange as it may be to
me.  tar doesn't seem to be in the latest POSIX spec, and I can never
find a good archive of POSIX specs, but Google knew about:

  http://pubs.opengroup.org/onlinepubs/7908799/xcu/tar.html

That's from The Single UNIX Specification, Version 2, with a copyright
year of 1997.  In that, tar has no "-p" option.  It has an "x" key
operand about which it says, "If a named file in the archive does not
exist on the system, the file is created with the same mode as the one
in the archive, except that the set-user-ID and set-group-ID modes are
not set unless the user has appropriate privileges."  So, that makes it
sound like macOS's /usr/bin/tar is behaving according to this spec when
run as root.

Anyway, here's what I know:

* The problem is that when I run "bmake extract" as root on macOS
  Sierra (10.12.3), the files in the distfile are extracted with
  /usr/bin/tar with their modes preserved, and tar's "-p" option was
  not specified.  The modes of the files in the distfile are all group-
  and world-writable.  The install logic of the package's source
  distribution preserves the file mode of some of the extracted files
  that are not built (e.g., header files) when it installs them.  With
  PKG_DEVELOPER=yes, the permissions check on the installed files fails
  because some of the files are group- and world-writable (because they
  were that way in the distfile, the mode was preserved on extraction,
  and the mode was preserved when installed).

* I submitted a patch to correct the file permissions post install.

* It was noted that preserving group- and world-writable file mode bits
  on extraction is a security issue as things are right now in pkgsrc
  because an unprivileged user could modify those files.

* macOS Sierra's /usr/bin/tar is bsdtar 2.8.3, and it preserves modes on
  extraction when run as root without the "-p" option.

* pkgsrc-current's archivers/bsdtar is version 3.3.1, and it has the
  same behavior as macOS Sierra's /usr/bin/tar; that is, it preserves
  modes on extraction when run as root without the "-p" option.

* NetBSD 6 stable's /bin/tar does not have the same behavior as macOS
  Sierra's /usr/bin/tar nor pkgsrc-current's archivers/bsdtar; that is,
  it does not preserve modes on extraction when run as root without the
  "-p" option.

* If I set EXTRACT_USING=pax, file modes are not preserved on extraction
  when run as root, and the package installs successfully with
  PKG_DEVELOPER=yes.

* I submitted a patch to change the EXTRACT_USING default on Darwin from
  nbtar to pax.

> > where it says in the extract phase section about the EXTRACT_USING
> > variable:
> >
> >   This variable can be set to bsdtar, gtar, nbtar (which is the default
> >   value), pax, or an absolute pathname pointing to the command with
> >   which tar archives should be extracted.  It is preferred to choose
> >   bsdtar over gtar if NetBSD's pax-as-tar is not good enough.
> 
> pkgsrc distinguishes between a user-settable variable, a
> package-settable variable, and more or less per-OS infrastructure.
> EXTRACT_USING seems to be for pkgsrc makefiles.
> 
> I think the real problem is that the Mac tar is broken and we need to
> avoid using it.
> 
> You could try commenting out the TOOLS_PLATFORM.tar line in
> mk/tools/tools.Darwin.mk, and see what happens.

I'd be willing to do that if that's what would be accepted, but I feel
like I need to know what would be accepted so I don't keep spending
effort doing things that are ultimately not accepted.

> > (If pax could be the default on all operating systems, then that would
> > be a simpler change and a simpler change to document. pax is part of
> > POSIX.1-2008 [1], whereas tar is not.  But I didn't want to change the
> > default on all operating systems since I didn't know what the fallout
> > might be from that.)
> >
> > [1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/pax.html
> 
> That seems preferable.
> 
> One suggestion is that if you wrote a test for writable files as a
> PKG_DEVELOPER test, we could fine out how widespread this is.
> On NetBSD 7, www/serf unpacks without writable files, both as a regular
> user and as root.  That's using /bin/tar, which is the same binary as
> pax an cpio.

I can see value in such a test.  What would it check for?  Just group-
or world-writable files?  Other things (e.g., setuid or setgid files)?

Thanks,

Lewis