pkg/35012: pkgsrc binary package dependancy system is broken

[kre%munnari.OZ.AU@localhost]

>Number:         35012
>Category:       pkg
>Synopsis:       pkgsrc binary package dependancy system is broken
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    pkg-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Nov 08 02:30:00 +0000 2006
>Originator:     Robert Elz
>Release:        NetBSD 3.99.15   (pkgsrc current as of last few hours)
>Organization:
        Prince of Songkla University
>Environment:
        
        
System: NetBSD jade.coe.psu.ac.th 3.99.15 NetBSD 3.99.15 
(GENERIC-1.696-20060125) #8: Wed Jan 25 04:59:39 ICT 2006 
kre%jade.coe.psu.ac.th@localhost:/usr/obj/current/kernels/JADE_ASUS i386
Architecture: i386
Machine: i386
>Description:
        Binary packages are built with a copy of the source package
        dependencies as their dependencies.   That's totally broken.

        A binary package *must* depend upon at least the version of
        another package that was installed at the time it was built, not
        the minimum version that might have been installed in order to
        build the source package.

        To see this, just consider the NetBSD C library (I know this is
        not a pkgsrc binary, but it illustrates the problem).   The vast
        majority of NetBSD programs will compile, link and run, against
        any version of the NetBSD C library (that is, the source dependency
        is simply that a C library exists).  Those programs that need at
        least a particular version - eg: maybe they need statvfs or something,
        show the same issue, but we don't need to consider those.

        That is, for example, /usr/src/bin/cat/* will happily compile
        and link (I guess, if I happened to pick a bad example, just
        pick another) against any version of the C library.

        However, a version that has been linked with the NetBSD 3 version
        of the C library isn't really expected to work on NetBSD 1.6, or
        NetBSD 2 (it might, sometimes, if it is lucky, but it isn't really
        expected).   On the other hand, a version linked against the
        NetBSD 3 C library is expected to work on a NetBSD 4 system.

        pkgsrc dependency system simply ignores this, and claims that
        any version will do, when it builds a binary package.   That's bogus.

>How-To-Repeat:
        The way I (am fairly sure) I saw this just recently was by
        installing firefox2, to see what it was like.   First, I was most
        pleased to see I could install it without having to delete  and
        upgrade half the universe first, I hadn't really expected that,
        even better, that it installed wthout even needing to remove
        firefox1 (or its gtk1 version) first.   Credit to the firefox
        package developers for that.

        However, it really didn't work for me at all once installed,
        some functionality was there, but core dumps, ...   I haven't
        bothered to debug this, but I think the problem is that I always
        install from binary packages (and did this time).   I build
        them myself, but I always build the binary package first, and
        then install it.

        The binary package I built for firefox2 claims that it needs...

jpeg>=6bnb2
libIDL>=0.8.6nb1
gtk2+>=2.8.17nb1
cairo>=1.0.4nb1
png>=1.2.9nb2
Xft2>=2.1.7nb2

        And on the system I installed it, what I have installed is ...

jpeg-6bnb3          IJG's jpeg compression utilities
libIDL-0.8.7        CORBA Interface Definition Language parser
gtk2+-2.8.20nb1     GIMP Toolkit v2 - libraries for building X11 user interfaces
cairo-1.2.0nb2      Vector graphics library with cross-device output support
png-1.2.12          Library for manipulating PNG images
Xft2-2.1.7nb2       Library for configuring and customizing font access

        which is ever dependency satisfied, nothing needed upgrading
        to install firefox2.

        However, now let's look at the packages that were installed
        when the firefox2 binary package was built - first, this is
        it (and its build date)

-rw-r--r--  1 root  wheel  14789895 Nov  1 01:38 firefox2-2.0.tgz

        And the dependency binary packages (which are all what were
        installed in my pkg_comp arena when firefox2 was compiled)

-rw-r--r--  1 root  wheel    305432 Jul 19 16:39 jpeg-6bnb3.tgz
-rw-r--r--  1 root  wheel    154330 Aug  6 00:15 libIDL-0.8.7.tgz
-rw-r--r--  1 root  wheel  11155929 Oct 15 22:56 gtk2+-2.10.6.tgz
-rw-r--r--  1 root  wheel    529259 Sep 30 19:42 cairo-1.2.4nb3.tgz
-rw-r--r--  1 root  wheel    307696 Jul 19 16:32 png-1.2.12.tgz
-rw-r--r--  1 root  wheel     78732 Sep 18 22:36 Xft2-2.1.7nb2.tgz

        Some of those are fine (ones that have not been upgraded in
        a while, like jpeg libIDL png and Xft2)

        But, firefox2 was compiled against gtk2+ version 2.10.6
        and is not attempting to use version 2.8.20 to run
        (which pkg_add doesn't object to, as the package only
        claims that it needs 2.8.17 or better).   Similarly,
        cario version 1.2.4 (the nb3 most likely doesn't matter)
        was used to build firefix2, but version 1.2.0 is being
        used to run it (again, pkg_add has no idea, as all it is
        being told is that anything 1.0.4nb1 or later is OK).

        I'm not going to go and attempt to figure out why firefox2
        was behaving strangely with a setup like this - I wouldn't
        expect it to work properly, and I am not surprised in the
        least that it doesn't.   Nor is the problem with pkg_add,
        as fas as it could tell, everytyhing was fine.   The problem
        is in pkgsrc, with what dependency information it is putting
        in the binary packages when it makes them.

        Note that this is what *any* user of binary packages will
        experience - note that all of the available binary packages are
        consistent, if I removed and reinstalled all of my /usr/pkg
        (or a substantial subset of it) this problem wouldn't arise.
        That is, it isn't a problem of building one binary package
        that's out of step with everything else.   There's no way that
        you can rationally expect anyone to do that however - rmeove and
        reinstall everything, every time they add a new binary package.
        And *everything* is th eonly choice most people are likely to
        have, as there's no current way to tell them which of their
        installed packages do need upgrading (I guess they could
        upgrade everything that is out of date - for me, that would
        have meant upgrading bind9 and bash and ... even though none of
        those has anything to do with firefox at all - I *think*).
        And even that assumes that there's a rational way to figure out
        what packages I have installed are out of date, if I have no
        pkgsrc on my system (relying entirely on binary packages) and
        the binary packages are fetched over the net, what technique would
        someone use to work out which of their installed packages is
        out of date (even if this was the right way forward - which it
        is not).

>Fix:
        When a binary package is built, the dependencies (for at least
        libraries, and anything else that influences the compilation
        process - anything that provides C .g files, or their equivalents
        in other languages) MUST be (at least) the versions that were
        installed when the binary package was built - later versions
        are generally OK (most library developers attempt to keep backwards
        binary compatability - not all of them, but most - even more than
        backwards source compatability).

        In this case, that would have told me that I needed
                gtk2+>=2.10.6
        (etc) and then i would have known that I needed to upgrade
        gtk2+ before firefox2 could be installed (which most likely
        would have meant upgrading a bunch of other stuff as well).

        Note: this cannot (should not, or even must not) be fixed with
        the dependencies in the source versions of the packages.
        If firefox2 says that gtk2+ version 2.8.17nb1 or better is
        OK to use to compile against, then that's all that should be
        required - that is, if I compiled firefox2 on the system
        I installed it on, upgrading nothing, I'd expect it to work.
        (If that's not true, then there's a bug in the firefox2
        dependency list, but that would, if it happened to be true,
        be a comparatively trivial issue - and I have no reason at all
        to assume that anything like that is a problem).