the pixman problem

[David Holland <dholland-pkgtech%netbsd.org@localhost>]

People using -5 have become all too familiar with this:

.../work/.x11-buildlink/lib/libcairo.so: undefined reference to 
`pixman_image_set_destroy_function'
.../work/.x11-buildlink/lib/libcairo.so: undefined reference to 
`pixman_image_composite32'
.../work/.x11-buildlink/lib/libcairo.so: undefined reference to 
`pixman_image_fill_boxes'
*** Error 1

I finally got direct access to it while failing last night, and here's
what happens at least in the case I found:

(1) Something in the victim package's config logic adds
-L/usr/X11R7/lib -R/usr/X11R7/lib to LDFLAGS or LIBS.

(2) This gets added *before* the -Wl,-R/usr/pkg/lib -Wl,-R/usr/X11R7/lib
that comes from pkg-config.

(3) The wrappers remove -L/usr/X11R7/lib but leave the
-R/usr/X11R7/lib alone and print:
   WARNING: [transform-gcc] passing unknown option -R/usr/X11R7/lib

(4) Thus the link line ends up with an rpath that has /usr/X11R7/lib
before /usr/pkg/lib. This is wrong, and the rest follows on a
predictable downhill path.

(5) Because the builtin pixman in -5 is Olde, cairo quite properly
installs and links against a newer one from pkgsrc. This means that
libpixman-1.so.0 is present in both /usr/X11R7/lib and /usr/pkg/lib.

(6) Because cairo links privately to pixman, pkg-config cairo --libs
quite properly does not include -lpixman-1.

(7) Therefore, the link line for the victim package does not include
-lpixman-1 either, so when the linker brings in cairo it searches the
rpath for pixman-1.so; because /usr/X11R7/lib comes first on that path
it finds the old one instead of the new one, which doesn't include
symbols cairo uses, and the link fails.

The problem can be worked around by editing cairo's pkgconfig file
(/usr/pkg/lib/pkgconfig/cairo.pc) and adding -lpixman-1 after
-lcairo. However, that just invites the problem to pop up again; point
(3) is what really needs to be fixed. I have no idea how to go about
this correctly.

Relatedly, one of the things I tried as a workaround was
   BUILDLINK_TRANSFORM+=   rm:-R/usr/X11R7/lib
and this did not work -- it was ignored and the -R option was not
removed. I would appreciate if someone could explain or look into why
this doesn't do what it's allegedly supposed to.

Other than "if ELF weren't so stupid we wouldn't have problems like
this", which while true isn't a useful line of reasoning in this
context, is there anything we can do to solve this problem in general?
Just as there are people who believe that /usr/bin should come before
/usr/pkg/bin in $PATH, I'm sure there are developers who believe that
/usr/lib and /usr/X11R7/lib should come before /usr/pkg/lib in linker
paths; manually hunting down and patching their mistakes when weird
behavior shows up doesn't seem like a good long-range plan.

-- 
David A. Holland
dholland%netbsd.org@localhost