Subject: Re: Same shared libraries in base and pkgsrc, was Re: lang/gcc3/buildlink2.mk
To: Todd Vierling <tv@pobox.com>
From: Frederick Bruckman <fredb@immanent.net>
List: tech-pkg
Date: 07/08/2003 10:38:46
On Tue, 8 Jul 2003, Todd Vierling wrote:

> [...more pkgsrc dreamland...]
>
> On Mon, 7 Jul 2003, Frederick Bruckman wrote:
>
> : Do you mean to allow two different run-times to be installed,
> : reserving the ".so" symlink for the devel package?
>
> Yes.  Several people, myself included, proposed this a while back, but the
> general response is typically that pkgsrc isn't robust enough to do it yet.
> The issues:
>
> * multiple pkg registrations per pkgsrc directory (this would be nice in may
>   ways, e.g. with info, libtool, gtar, where we would no longer need a
>   separate build tree for "-base" and "-doc")
>
> * ability to install more than one version of a pkg (libfoo-1.0, libfoo-2.0)
>   without the implicit CONFLICTS (=libfoo-[0-9]*)
>
> One workaround suggested was a pkg_delete/"pkg_add -u" additional flag that
> would leave any .so.major and .so.major.minor[.teeny] in place.  This
> doesn't help installing a binary package dependent on the older libs on
> another machine, but it at least allows code to keep running.

You know, I've actually been doing the first thing myself for quite a
while. The scripts and "pkgdb" source are available from my directory
on ftp.netbsd.org via ftp. It does work for binary packages, too, as
long as you install the old dependency by hand (with "pkg_hack -a").
The biggest problem is with auto-generated PLISTS, but that's an
artifact of using a script wrapper. If the superceded files were
unregistered directly by pkg_add, it wouldn't be an issue. Feedback,
enthusiasm have been pretty thin though.

> And all this somewhat relates, again, to the fact that pkgsrc doesn't
> address dependencies against shlibs -- i.e. "Which major version of this
> shlib do I need?  Which package version has that?  Oops, this newer version
> of my dewey dependency has the wrong major version; I'll crash now."

Well, they mostly refuse to run. I fielded a brainstorm to add
@needs or @needslib tags, calculated via "ldd" at build time, to
the +CONTENTS, and no one strongly objected. I just need to find
time and get focused.

> : That could work, if we're allowed to drop support for a.out.
>
> You don't have to do that, because a.out doesn't care about .so symlinks --
> only that the major number is monotonically increasing (i.e. the devel
> package headers that are installed correspond to the highest installed major
> number).
>
> If someone on a.out were to install a newer runtime and older devel, "you
> get what you deserve".  Although this case could be handled by implementing
> dewey CONFLICTS (with >, not >= in this case) -- and making an implicit
> conflict on devel packages:
>
> CONFLICTS+=	${PKGBASE:C/-devel$//}>${PKGVERSION}
>
> This would disallow installing any newer runtime without deinstalling the
> older devel package first.  It would be useful even on ELF, to indicate
> exactly why it's best to have a devel package that is at least as new as the
> newest installed runtime.

I think you're saying that we can "support" a.out, as long you all
realize that it's broken. I'd rather say, "We don't support a.out",
but if it sort of works for somebody, good for them.

I actually had "pkg_hack" checked for tainted a.out dependencies and
force upgrading them, but it adds too much time for the rest of us
(well, me), now that the upgrade to ELF is complete, so I took it out.

> : Oh, I see we're not vulnerable to the same lossage that affected
> : OpenSSL -- package links against base libssl, gets run-path of
> : ${PREFIX}/lib for some other package, OpenSSL package gets installed,
> : package now runs against package OpenSSL -- because libstdc++ isn't
> : installed to ${PREFIX}/lib.
>
> Thank goodness we have buildlink2, which, I've found, does indeed work
> around this issue happily.  (Ignore all my previous comments about it "not
> working"; it does.  It's just some programs here and there which try to
> "reach outside the buildlink2 box" in the pkgsrc bits.)

1) No, it doesn't. The issue is that RPATH is searched before base for
all NEEDED shared libraries. You can't tell ld.elf_so, "Wait, not for
this one." [Or maybe you can, with some obcure ld scripts, be we don't
do that.] The fix was to give pkg openssl a unique soname, generated
by adding "300" to the major.

2) buildlink2 doesn't work for gcc. The idea of wrapping the tools by
inserting a script in the path *does* *not* *work* for "./xgcc" which
is called with a full path in any case. The end result is that a
hidden dependency on iconv is introduced where there was none before
(because of the addition of -I/usr/pkg/include to CFLAGS, which a
script would transform if there were a script, but there isn't).

> : It'll still be a mess if a package links against a library that runs
> : against package libstc++, and also another library that runs against base
> : libstdc++, won't it?
>
> That depends on whether the two dependencies are exporting C++ APIs to the
> thing liking to them.  Yes, this particular case is a bad problem.  But,
> you're likely to know it at link time because of symbol conflicts.

I believe that the linker may complain that the two files "conflict",
but it'll link it anyway. Because of my experience with "pkg_hack",
I've run into this a lot with "gnome" packages. For this lossage,
since the two files are almost the same, it'll almost work, except
when it doesn't.

> Ideally, pretty much all C++ code on a system needs to be recompiled after
> src/ switches to GCC3 -- though the GCC & CodeSourcery people are trying
> Hard to get it to an ABI-stable state.  It's close, but no cigar (gcc-3.4
> uses major number 6).

:-( I didn't know that.

Frederick