Re: pkgin can't process URL

[Jason Bacon <outpaddling%yahoo.com@localhost>]

On 2019-07-17 14:14, Jonathan Perkin wrote:
> * On 2019-07-17 at 17:13 BST, Jason Bacon wrote:
>
> > A handful of dependencies fail to install when using pkgin.  The error
> > always shows a wildcard for the package name, e.g. 'libLLVM*'.  I'm not sure
> > if that's anything special, or if it's just failing on these packages for
> > some other reason.
> >
> > In every case, the package can be installed manually, but fails to install
> > as a dependency.
> >
> > Any ideas what might be causing this?  pkgin bug?  Might it have to do with
> > the web server config?
> There are a few things odd about this:
>
>   - Why is the log showing "pkg_add.bin"?  pkgin hardcodes the path to
>     pkg_add so there is clearly something non-standard going on here -
>     it would help to include anything like that in the initial bug
>     report, otherwise we will assume you are using the defaults.
>
>   - After downloading all the required packages, pkgin calls pkg_add
>     with a full path to the downloaded package.  The fact it's falling
>     back to the glob method shows something already went wrong earlier.
>
>   - As the log shows your pkg_install is built without SSL support, so
>     trying to install from https:// is not going to work.
>
> I would first debug why the packages are not being installed directly
> from the pkgin cache and go from there, the rest (SSL support and "no
> pkg found") are essentially false flags as you shouldn't be falling
> back to that method in the first place.  This will likely be caused by
> whatever you are doing to pkg_add.bin.
>
> Cheers,
That little overview of how pkgin interacts with pkg_add was just the 
clue I needed.

FYI, the pkg_add.bin you see is due to a simple wrapper we use:

shell: cat /sharedapps/pkg-2019Q2/sbin/pkg_add
#!/bin/sh -e

# Make sure software installed is world-readable on systems with tighter
# default umask.
umask 022
pkg_add.bin "$@"

We have a default umask of 027 on our HPC clusters, which would leave 
some installed files unreadable.  I verified that unwrapping pkg_add 
does not make any difference.

I can, however, alleviate the issue by changing pkg_install.conf from

PKG_PATH=https://mirror2.hpc.uwm.edu/pkgsrc/packages/sharedapps/pkg-2019Q2/RHEL7/All;

to

PKG_PATH=http://mirror2.hpc.uwm.edu/pkgsrc/packages/sharedapps/pkg-2019Q2/RHEL7/All;

pkgin/repositories.conf is

https://mirror2.hpc.uwm.edu/pkgsrc/packages/sharedapps/pkg-2019Q2/RHEL7/All

I built pkgin from source with SSL support (PKG_OPTIONS.libfetch=inet6 
openssl) but left the original pkg_install built during bootstrap.

The question that remains is why does https work in PKG_PATH for 99% of 
packages, but not for a handful of dependencies?  It seems that PKG_PATH 
must be configured correctly in very rare cases, even though pkg_add is 
not downloading anything.

I confirmed that all packages are present in 
/sharedapps/pkg-2019Q2/var/db/pkgin/cache/.

I think the real solution on our end is to rebuild pkg_install with SSL 
support.  This will ensure security in case someone bypasses pkgin.  ( 
Or I may have another go at building with SSL support during bootstrap, 
although that was not practical before for reasons I can't remember. )

I'd still like to understand what's going on here, though.

Thanks,

    JB