>Number:         35012
>Category:       pkg
>Synopsis:       pkgsrc binary package dependancy system is broken
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    pkg-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Nov 08 02:30:00 +0000 2006
>Originator:     Robert Elz
>Release:        NetBSD 3.99.15   (pkgsrc current as of last few hours)
>Organization:
	Prince of Songkla University
>Environment:
	
	
System: NetBSD jade.coe.psu.ac.th 3.99.15 NetBSD 3.99.15 (GENERIC-1.696-20060125) #8: Wed Jan 25 04:59:39 ICT 2006 kre@jade.coe.psu.ac.th:/usr/obj/current/kernels/JADE_ASUS i386
Architecture: i386
Machine: i386
>Description:
	Binary packages are built with a copy of the source package
	dependencies as their dependencies.   That's totally broken.

	A binary package *must* depend upon at least the version of
	another package that was installed at the time it was built, not
	the minimum version that might have been installed in order to
	build the source package.

	To see this, just consider the NetBSD C library (I know this is
	not a pkgsrc binary, but it illustrates the problem).   The vast
	majority of NetBSD programs will compile, link and run, against
	any version of the NetBSD C library (that is, the source dependency
	is simply that a C library exists).  Those programs that need at
	least a particular version - eg: maybe they need statvfs or something,
	show the same issue, but we don't need to consider those.

	That is, for example, /usr/src/bin/cat/* will happily compile
	and link (I guess, if I happened to pick a bad example, just
	pick another) against any version of the C library.

	However, a version that has been linked with the NetBSD 3 version
	of the C library isn't really expected to work on NetBSD 1.6, or
	NetBSD 2 (it might, sometimes, if it is lucky, but it isn't really
	expected).   On the other hand, a version linked against the
	NetBSD 3 C library is expected to work on a NetBSD 4 system.

	pkgsrc dependency system simply ignores this, and claims that
	any version will do, when it builds a binary package.   That's bogus.

>How-To-Repeat:
	The way I (am fairly sure) I saw this just recently was by
	installing firefox2, to see what it was like.   First, I was most
	pleased to see I could install it without having to delete  and
	upgrade half the universe first, I hadn't really expected that,
	even better, that it installed wthout even needing to remove
	firefox1 (or its gtk1 version) first.   Credit to the firefox
	package developers for that.

	However, it really didn't work for me at all once installed,
	some functionality was there, but core dumps, ...   I haven't
	bothered to debug this, but I think the problem is that I always
	install from binary packages (and did this time).   I build
	them myself, but I always build the binary package first, and
	then install it.

	The binary package I built for firefox2 claims that it needs...

jpeg>=6bnb2
libIDL>=0.8.6nb1
gtk2+>=2.8.17nb1
cairo>=1.0.4nb1
png>=1.2.9nb2
Xft2>=2.1.7nb2

	And on the system I installed it, what I have installed is ...

jpeg-6bnb3          IJG's jpeg compression utilities
libIDL-0.8.7        CORBA Interface Definition Language parser
gtk2+-2.8.20nb1     GIMP Toolkit v2 - libraries for building X11 user interfaces
cairo-1.2.0nb2      Vector graphics library with cross-device output support
png-1.2.12          Library for manipulating PNG images
Xft2-2.1.7nb2       Library for configuring and customizing font access

	which is ever dependency satisfied, nothing needed upgrading
	to install firefox2.

	However, now let's look at the packages that were installed
	when the firefox2 binary package was built - first, this is
	it (and its build date)

-rw-r--r--  1 root  wheel  14789895 Nov  1 01:38 firefox2-2.0.tgz

	And the dependency binary packages (which are all what were
	installed in my pkg_comp arena when firefox2 was compiled)

-rw-r--r--  1 root  wheel    305432 Jul 19 16:39 jpeg-6bnb3.tgz
-rw-r--r--  1 root  wheel    154330 Aug  6 00:15 libIDL-0.8.7.tgz
-rw-r--r--  1 root  wheel  11155929 Oct 15 22:56 gtk2+-2.10.6.tgz
-rw-r--r--  1 root  wheel    529259 Sep 30 19:42 cairo-1.2.4nb3.tgz
-rw-r--r--  1 root  wheel    307696 Jul 19 16:32 png-1.2.12.tgz
-rw-r--r--  1 root  wheel     78732 Sep 18 22:36 Xft2-2.1.7nb2.tgz

	Some of those are fine (ones that have not been upgraded in
	a while, like jpeg libIDL png and Xft2)

	But, firefox2 was compiled against gtk2+ version 2.10.6
	and is not attempting to use version 2.8.20 to run
	(which pkg_add doesn't object to, as the package only
	claims that it needs 2.8.17 or better).   Similarly,
	cario version 1.2.4 (the nb3 most likely doesn't matter)
	was used to build firefix2, but version 1.2.0 is being
	used to run it (again, pkg_add has no idea, as all it is
	being told is that anything 1.0.4nb1 or later is OK).

	I'm not going to go and attempt to figure out why firefox2
	was behaving strangely with a setup like this - I wouldn't
	expect it to work properly, and I am not surprised in the
	least that it doesn't.   Nor is the problem with pkg_add,
	as fas as it could tell, everytyhing was fine.   The problem
	is in pkgsrc, with what dependency information it is putting
	in the binary packages when it makes them.

	Note that this is what *any* user of binary packages will
	experience - note that all of the available binary packages are
	consistent, if I removed and reinstalled all of my /usr/pkg
	(or a substantial subset of it) this problem wouldn't arise.
	That is, it isn't a problem of building one binary package
	that's out of step with everything else.   There's no way that
	you can rationally expect anyone to do that however - rmeove and
	reinstall everything, every time they add a new binary package.
	And *everything* is th eonly choice most people are likely to
	have, as there's no current way to tell them which of their
	installed packages do need upgrading (I guess they could
	upgrade everything that is out of date - for me, that would
	have meant upgrading bind9 and bash and ... even though none of
	those has anything to do with firefox at all - I *think*).
	And even that assumes that there's a rational way to figure out
	what packages I have installed are out of date, if I have no
	pkgsrc on my system (relying entirely on binary packages) and
	the binary packages are fetched over the net, what technique would
	someone use to work out which of their installed packages is
	out of date (even if this was the right way forward - which it
	is not).

>Fix:
	When a binary package is built, the dependencies (for at least
	libraries, and anything else that influences the compilation
	process - anything that provides C .g files, or their equivalents
	in other languages) MUST be (at least) the versions that were
	installed when the binary package was built - later versions
	are generally OK (most library developers attempt to keep backwards
	binary compatability - not all of them, but most - even more than
	backwards source compatability).

	In this case, that would have told me that I needed
		gtk2+>=2.10.6
	(etc) and then i would have known that I needed to upgrade
	gtk2+ before firefox2 could be installed (which most likely
	would have meant upgrading a bunch of other stuff as well).

	Note: this cannot (should not, or even must not) be fixed with
	the dependencies in the source versions of the packages.
	If firefox2 says that gtk2+ version 2.8.17nb1 or better is
	OK to use to compile against, then that's all that should be
	required - that is, if I compiled firefox2 on the system
	I installed it on, upgrading nothing, I'd expect it to work.
	(If that's not true, then there's a bug in the firefox2
	dependency list, but that would, if it happened to be true,
	be a comparatively trivial issue - and I have no reason at all
	to assume that anything like that is a problem).