Re: pkgsrc vs. https-only master sites

[Thomas Klausner <wiz%NetBSD.org@localhost>]

On Mon, Feb 23, 2015 at 03:18:24PM +0100, Tobias Nygren wrote:
> The problem of master sites redirecting to https or only providing
> https is growing and I think we need to make pkgsrc deal with this
> better than it does now. As you are probably aware base ftp(1) on
> NetBSD < 7 does not support https URLs.
> 
> Having only MASTER_SITE_BACKUP is both fragile and annoying. When a
> package has recently updated and no mirrored distfile is available yet
> the user has a bad experience.

Well, that's only relevant for pkgsrc-current (since for stable,
ftp.netbsd.org has lots of time to get the files) and if you don't set
FETCH_USING. But let's continue... :)

> One way to fix this would be to remove the SSL option from
> pkgsrc/net/tnftp and always build it with SSL. Then we could depend
> on it if the download URL has https.
> This does not solve the problem of http->https 301 redirects though.
> So maybe we need to unconditionally pull in an SSL aware fetch tool
> from pkgsrc if base does not provide one.
> 
> Thoughts on this?

Don't forget bootstrap and non-NetBSD.

Since on platforms without ssl, the bootstrap needs to fetch openssl,
a two-step approach would be needed:

* build tnftp(1) without ssl support to get a basic tool that can be
  used to (at least) fetch openssl

* build openssl (if necessary) and build a second tnftp with ssl
  support

It's mostly SMOP, but it involves the bootstrap process and no-one was
motivated enough yet to tackle it.

The next question is how useful ssl support is without a certificate
chain... but we can postpone that discussion.
 Thomas