Re: haskell: use buildlink3 files?

[PHO <pho%cielonegro.org@localhost>]

Sorry for my late response.

On 2020-04-03 17:21, Thomas Klausner wrote:
> Hi!
> 
> While fixing some problems with non-building hs-* packages I noted
> that I can add DEPENDS to get the dependencies installed and usable by
> the Haskell compiler.
> 
> However, most hs-* packages provide a buildlink3.mk file.
> 
> For Python, we decided that since the compiler is not involved, we
> want to avoid using buildlink3.mk and instead use plain DEPENDS. I
> think this also gives a parsing speedup because less files are opened.
> 
> Do we want to decide the same for Haskell, or can buildlink be
> extended to work as intended for Haskell packages as well? (I.e., that
> hs-* packages that are not buildlinked will not be found by the
> Haskell compiler.)
> 
> If yes, is anyone willing to work on this?

I prefer to keep using buildlink3 for Haskell packages because in theory
it's better than DEPENDS if it's working correctly. Here are reasons why
it isn't working as intended right now:

1) The compiler (GHC) builds Haskell packages as native .a (and
optionally .so as well), and pkgsrc installs them.

2) While building package A which depends on another package B,
buildlink3 creates symlinks to those libraries under BUILDLINK_DIR. And
GHC invokes ld(1) with appropriate -L and -l flags.

3) However, GHC uses @file syntax for those flags. This causes our
wrapper to not rewrite them to redirect to BUILDLINK_DIR, so those
symlinks are not actually used. The reason it uses @file is apparently
that the limit on the length of argv is unacceptably short on a certain
platform, namely Windows.

4) I tried patching GHC to avoid using @file. It mostly worked except
for one specific case: building GHC itself. The build system for GHC
uses $ORIGIN for linking .so that are used by GHC. This only works
because actual ld flags mentioning $ORIGIN are hidden in @file. They'd
be removed if they were exposed to our wrapper.

5) Patching lang/ghc* to not using $ORIGIN might sound like the best
option, but it isn't necessarily so. It's in an unfortunate situation
like openjdk and rust: It needs itself to bootstrap. If we were to
disable the use of $ORIGIN, our binary bootstrap kit would no longer be
position-independent. I mean, a bootkit built for /usr/pkgsrc will no
longer work anywhere else. And also the build system is terribly
complicated and patching it is going to be a great deal of work.

6) One might argue that we should use LD_LIBRARY_PATH instead of using
$ORIGIN. That's usually the best way to go, but in this case it's not
that simple. Haskell libraries are installed under separate directory
for each library as opposed to a uniform one such as /usr/pkg/lib. To
make matters worse, those directories contain a package hash in their
name so generating a list of LD_LIBRARY_PATH is not an easy task.

7) Patching our wrapper to not removing flags that mention $ORIGIN, and
also rewriting flags in @file, comes with its own troubles. I personally
think this isn't an unacceptable change, but others don't necessarily agree.

So the situation is rather complicated. I *really* love to fix it so
buildlink does what it's intended for, but I haven't managed to do it
yet. Suggestions are welcomed.