On Sat, Apr 13, 2002 at 10:51:19PM -0400, Amitai Schlair wrote:
> graphics/gphoto2 gratuitously adds -I and -L paths in ${LOCALBASE}. Since
> these added paths are the only thing preventing gphoto2 from being strongly
> buildlinked, I started wading through and patching them out.
> 
> I know gphoto2 is far from the only package that has had this problem, and
> my train of thought led me along: what if setting USE_BUILDLINK_ONLY also
> dropped a cc wrapper in ${BUILDLINK_DIR}/bin? The wrapper would remove bad
> -I and -L paths before passing its arguments to the real ${CC}.

This is exactly what I'm experimenting with for changing pkgsrc internals
after the 1.6 pkgsrc branch point.  Results have been very good thus far.

> A quick example:
> 
> -----
> 
> #!/bin/sh
> 
> remove_bad_paths()
> {
>         badpaths="$@"; sedcmd="sed"
> 
>         for i in $badpaths; do
>                 sedcmd="$sedcmd -e 's|$i ||g'"
>                 sedcmd="$sedcmd -e 's|$i\$||g'"
>                 sedcmd="$sedcmd -e 's|$i/[^ ]*||g'"
>         done
> 
>         eval $sedcmd
> }
> 
> exec cc `echo "$@" | remove_bad_paths "-I/usr/pkg/include -L/usr/pkg/lib"`
> 
> -----
> 
> (Of course, this would be auto-generated by a make target somewhere in
> bsd.buildlink.mk, with appropriately substituted values for ${LOCALBASE},
> ${CC}, ${SED}, etc.)

There will also have to be wrappers for c++, gcc, g++, and ld, but that's
pretty simple to arrange.  You're a better shell script writer than I am
and your logic is more efficient than mine, so I'll steal this for the
scripts I'm currently generating, though instead of just removing the bad
paths, I convert them to point into ${BUILDLINK_DIR}.

> Benefits:
> 
> 1) No need to grep package build logs for buildlink-dirty paths, then grovel
> third-party makefiles and patch. Dirty paths automatically won't get passed
> to the real compiler.

Yes, wading through the build log to check that everything is building without
reliance on things in ${LOCALBASE} or ${X11BASE} is a real time-sink.  This is
the primary motivation for why I was experimenting with the compiler wrapper
script.

> 2) A package that sets USE_BUILDLINK_ONLY but isn't actually strongly
> buildlinked will often fail to build. (Not always -- for example, a package
> that uses curses but doesn't include devel/ncurses/buildlink.mk will still
> find curses because it's in the default compiler path. But often.)
> 
> 3) Making strongly buildlinked packages will be easier, so package
> developers will be more inclined to do it.

I don't think most pkgsrc developers understand what buildlink does yet.  Most
just use buildlink.mk files instead of writing dependencies and are happy that
files are found in ${BUILDLINK_DIR}.  This does make the packages weakly
buildlinked, and as such is a good thing in itself.  I agree that using the
compiler wrapper script will make it easier to make packages strongly
buildlinked for everyone.

> Drawbacks:
> 
> 1) Someone who doesn't know about this wrapper might be confused when build
> output in strongly buildlinked packages occasionally doesn't correspond
> exactly to the arguments passed to the compiler. The differences are small
> and predictable, though, and we can document them in Packages.txt.

This is a minor concern.  Developers will simply get used to the fact that
references to directories in either ${LOCALBASE} or ${X11BASE} are removed
as more packages become strongly buildlinked.

> 2) In strongly buildlinked packages, every call to ${CC} will be a bit
> slower. However, pkgsrc already imposes a minor speed penalty on the process
> of building from source, which itself isn't the fastest way to install
> software. Which is fine by me -- our goal is to make it easy to install
> software well, and measured in user time, we're speed demons. :-p

This is my main worry with the compiler wrapper script.  Most of our platforms
are slow, and pkgsrc isn't designed to be cross-built, so the penalties for
using the wrapper scripts can be substantial.  I'll do some experiments on a
mac68k at some point to get some real statistics on the penalties resulting
from using the wrapper scripts.

> 3) We have to make sure all callers of ${CC} find the wrapper instead of the
> system compiler. ${BUILDLINK_DIR}/bin is prepended to $PATH in sub-makes. Is
> this sufficient?

I have this addition to bsd.buildlink.mk:

.if defined(USE_BUILDLINK_ONLY)
PATH:=	${BUILDLINK_DIR}/bin:${PATH}
.endif

and it's already simplified finding config script wrappers (*-config) and the
usual GNU configure script logic for finding the compiler.  The only packages
for which I'm worried are packages that use Imakefiles, but I think something
like xpkgwedge can solve that problem.  It's something I'll worry about later,
as packages that use Imakefiles can't be strongly buildlinkified yet anyway.

> My opinion (naturally) is that the benefits are pretty sizable, and that the
> drawbacks are more than worth the trouble (though I am worried about #3).
> Thoughts?

I agree with you.  Making packages strongly buildlinked is an important goal
and using compiler wrapper scripts definitely takes us in that direction.
Using the wrapper scripts also greatly simplifies the logic in
bsd.buildlink.mk, which is also a good thing as the amount of stuff that file
does has grown by much more than I originally anticipated and has made the
packaging process "funky", as one developer has described it.  If I can be
sure that the penalties (drawback #2) are not too bad after measuring on the
mac68k, then I'm going to make sure this change goes in.

	Cheers,

	-- Johnny Lam <jlam@jgrind.org>