pkgsrc-Bugs archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
Re: pkg/56906: libbsd: update devel/libbsd to 0.11.0
The following reply was made to PR pkg/56906; it has been noted by GNATS.
From: David Holland <dholland-pbugs%netbsd.org@localhost>
To: gnats-bugs%netbsd.org@localhost
Cc:
Subject: Re: pkg/56906: libbsd: update devel/libbsd to 0.11.0
Date: Sun, 10 Jul 2022 00:33:03 +0000
Several messages in this discussion weren't sent to gnats.
(Send replies to bug reports to gnats-bugs@; it then remails them.
Mails sent to just the mailing list or to gnats-admin don't get into
the bug database unless someone notices and forwards a copy.)
(Generally it seems that if you reply to your own message it does not
go to the right place by default...)
------
On Sun, Jul 03, 2022 at 11:32:33AM +0200, Paolo Vincenzo Olivo wrote:
> Apparently, also libbsd contains Beerware-licensed code [1], so the
> LICENSE entry of the devel/libbsd package (which is `original-bsd'),
> should be changed to reflect the fact libbsd it's multi-licensed.
>
> [1] https://libbsd.freedesktop.org/wiki/
>
> - PVO
On Sat, Jul 02, 2022 at 04:37:22PM +0200, Paolo Vincenzo Olivo wrote:
> So, an update on this topic is due.
>
> I modified wip/libbsd 's Makefile to enable link-time optimization when
> supported, and I forced autoreconf at pre-configure stage, since the
> README for 0.11.6 recommends running `autogen'.
>
> The package now explicitly depends on devel/libmd (available as wip/libmd)
> through the buildlink methodology.
>
> wip/libbsd (0.11.6) compiles fine on Linux, but any time I try to link
> against it in a package (with -lbsd or pkg-config), the compiler hangs
> indefinitely at build stage. This didn't ahppen with previous versions
> of libbsd.
>
> I was able to reproduce it by attempting to compile the same programs
> manually, both on my workstation (Slackware 15.0) and on SDF's MetaArray
> (Debian 11), using a very standard mk.conf. Still same issue, as soon as
> the compiler tries to link against libbsd, the build freezes forever,
> without printing any error.
>
> Apparently nobody else reported this so far. Other distributions do not
> apply any noteworthy patch to libbsd-0.11.6, aside from removing the
> libtool archives, which for them is common practice.
>
> I wonder whether this has anything to do with the pkgsrc infrastructure
> (default compiler flags, linked libs, mitigations) or if I'm missing
> something very stupid in the Makefile, which somebody more knowledgeable
> would catch immediately while looking at libbsd's build log.
>
> Anyway, at least I've nailed down the issue to some change introduced
> between 0.11.3 and 0.11.4, as libbsd-0.11.3 allows me to pass -lbsd
> safely in other packages, while 0.11.4 does not (resulting in the
> aforementioned 'hang').
>
> The ChangeLog for libbsd is available at:
> https://cgit.freedesktop.org/libbsd/log/?h=main
>
> I'd look at any commit between 0.11.3 and 0.11.4. One may be:
> "Switch md5 compatibility logic back to direct linking"
> https://cgit.freedesktop.org/libbsd/commit/?h=main&id=e7cf8c5785b14fc8fbd37bb665a5f9a4f28c7888
>
> Would it make sense, for the time being, to pull libbsd-0.11.3, together
> to libmd-1.0.4 (latest) in the main tree, and try to understand what's
> the issue with 11.0.6 in the meantime?
>
> Speaking of libmd, I successfully built wip/libmd on NetBSD and Solaris
> 11 (and Linux). It's a very simple package and unless someone has
> something to object, I'd consider it ready to import.
> The only problem with libmd is that it contains code licensed under
> 'beer-ware' [1], an extremely permissive and substantially copyright-
> only license, which, while being supposedly BSD/GPL compatible, is not
> included among the accepted licenses in licenses.mk
>
> I found libbsd to be very linux-centric instead (not something
> unheard of freedesktop's), in spite of their supposed focus on
> portability. Ii'm confident that libbsd is routinely tested on Linux,
> macOS (and perhaps MingW) only.
>
> They recently dropped OpenBSD support, since their obsd-specific
> arc4random implementation wouldn't compile in it.
>
> On Solaris 11.4 the build fails early on, due to incompatible
> arc4random_addrandom(). I suppose this would turn out the same way on
> illumos too.
>
> On NetBSD, arc4random.c, getopt.c and some others compile fine after
> patching bsd/unistd.h for closefrom(), but I found that multiple other
> functions need major rework. And at this point, it's probably better to
> rely on a ad-hoc compat library when porting openbsd software, as done
> for openssh, or devel/got.
>
> At the same time, a good portion of *BSD utils ported to Linux adn OS X
> relies on libbsd, so it does make sense to have a working and up to date
> version in repo.
>
> [1] https://en.wikipedia.org/wiki/Beerware
>
> - PVO
On Fri, Jul 08, 2022 at 02:39:23PM +0200, Paolo Vincenzo Olivo wrote:
> (Splitting this reply in 2 mails; one regarding the libbsd bug, the
> other dedicated to discussing the observations made in the parent message).
>
> On 22/07/03 07:50PM, David H. Gutteridge wrote:
> >
> > If there are regressions with 0.11.6 vs. 0.11.5 (etc.), those should
> > really be reported back upstream (regardless of what we'd do).
> >
>
> And we have a culprit.
> The bug was introduced indeed between 0.11.3 and 0.11.4 [1]. In
> particular, this added chunk of code in the `install-exec-hook' target of
> the Makefile.am:
>
> ```
> + if NEED_TRANSPARENT_LIBMD
> + # The "GNU ld script" magic is required so that GNU ldconfig does not complain
> + # about an unknown format file.
> + soname=`readlink $(DESTDIR)$(libdir)/libbsd.so`; \
> + $(RM) $(DESTDIR)$(libdir)/libbsd.so; \
> + (echo '/* GNU ld script'; \
> + echo ' * The MD5 functions are provided by the libmd library. */'; \
> + cat format.ld; \
> + echo "GROUP($(runtimelibdir)/$$soname AS_NEEDED($(MD5_LIBS)))"; \
> + )>$(DESTDIR)$(libdir)/libbsd.so
> ```
>
> Results in having the target shared objects of @libbsd.so (the
> libbsd.so.0.11.6 shared lib) overwritten with the content echoed in
> quotes, whenever $(RM) is undefined or null and @libbsd.so isn't
> consequently preemptively removed.
>
> This leads to a circular reference between the @libbsd.so the libbsd
> library file, whereby the latter itself points to a non-existing object.
> And this accounts for the hangings witnessed while trying to link
> against libbsd.
>
> By skimming through the DESTDIR, I had noticed that libbsd.so.0.11.6
> curiously consisted of a plain text file, but I couldn't make up my mind
> about it. It was @RVP, in a private mail exchange, to point me to
> where the problem actually lied.
>
> The reason why this happens on pkgsrc, whereas nobody else reported it
> in the Linux ecosystem, is that GNU Make defines RM among its implicit
> variables [2] and aliases it to `rm -f'. When the install-exec-hook
> target is executed, $(RM) is lost and bmake tries to execute @libbsd.so
> rather than forcibly removing it.
>
> I wouldn't strictly consider this a bug, as much as a portability
> concern, especially considering freedesktop doesn't mention gmake as a
> dependency for libbsd. This further reinforces my previous impression of
> libbsd being very linux-centric and hardly tested outside of Linux, in
> open contrast with the supposed goals of the project.
>
> I've committed a very simple a patch to wip/libbsd to address the
> problem (replacing $(RM) with `rm -f'in that single line) . Now libbsd
> works just as expected, at least with my packages. Another way to
> address to problem would be to redefine pkgsrc's RM in the package
> Makefile and pass it to libbsd as a MAKEFLAG. Or add gmake to USE_TOOLS.
> Which approach is considered best?
>
> Putting the other considerations made in this thread temporarily aside I
> now consider libbsd (+libmd) ready to import.
>
> [1] 'Switch md5 compatibility logic back to direct linking' \
> https://gitlab.freedesktop.org/libbsd/libbsd/-/commit/e7cf8c5785b14fc8fbd37bb665a5f9a4f28c7888
>
> [2] https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html
>
> --
> ----------------------------+----------------------------
> vms[-at]retrobsd.ddns.net | https://retrobsd.ddns.net
>
On Fri, Jul 08, 2022 at 04:46:29PM +0200, Paolo Vincenzo Olivo wrote:
> On 22/07/03 07:50PM, David H. Gutteridge wrote:
> >
> > Something to think about with both libbsd and libmd is if they should
> > provide proper built-in detection. libbsd has an attempt at it, but
> > seems wrong to me, as it doesn't work on BSDs. (I guess the assumption
> > is that no one would choose to build it in that context, and depending
> > packages would conditionally handle this.
>
> Yes, I haven't touched the original builtin.mk of libbsd-0.10.0. Indeed,
> it only checks if libbsd is already available on the host system.
> There's no wrapper to test supported functions/calls and verify their
> compatibility} with those provided by libbsd.
>
> My take? Somebody more experienced than me could look at the libbsd code
> and port whatever function is lacking to libncompat. Then all packages
> relying on libbsd could have a conditional to include the required
> headers whenever `HAVE_NBCOMPAT_H' is defined.
>
> NetBSD could for example benefit from having compatibility for
> explicit_bzero(), readpassprhase(), strtofflags(), pledge(),
> freezero(), and recallocarray(). iAtthe same time, libbsd includes
> support for NetBSD-specific functions like strtoi() and strtou(), and
> uses NetBSD's vins/unvis.
>
> > though they don't seem to entirely consistently or correctly, e.g.,
> > audio/rtunes or graphics/scrot.)
>
> The packages currently depending on libbsd usually only include
> libbsd's buildlnk3.mk if the target platform is either Linux or
> different from *BSD. No clear standardised behavior. Undefined *BSD is a
> too generic assumption and not very robust imho. It works with older
> packages, but not with new software designed on and for any of the BSDs,
> which typically relies on system-specific features, making it non
> portable across BSDs. Packages currently requiring libbsd on Linux are
> usually tools ported from OpenBSD, and sometimes from other BSDs. It's
> not like they will simply compile on NetBSD without patches.
>
> A major shortcoming of linking to libbsd anywhere outside of *BSD (which
> is for example what audio/rtunes does), is that the target platform may
> not need libbsd as much as Linux does or not need it at all. This is the
> case of illumos for example, and to a good extent, Solaris 11. libbsd
> won't compile here. macOS is an exception (libbsd can be found in
> MacPorts), but this is probably the consequence of a stronger focus and
> attention to the platform.
>
> I think this thread as a whole raises the question about the potential
> need of a reworked userspace shim for BSD functions capable of building
> on NetBSD first, and other platforms later on.
>
> > Trying to build the current libbsd 0.10 on NetBSD 9.99.97/amd64 fails
> > with:
> >
> > In file included from arc4random.c:33:
> > ../include/bsd/unistd.h:62:6: error: conflicting types for 'closefrom'
> > 62 | void closefrom();
> > | ^~~~~~~~~
> > In file included from ../include/bsd/unistd.h:31,
> > from arc4random.c:33:
> > /usr/include/unistd.h:329:6: note: previous declaration of 'closefrom'
> > was here
> > 329 | int closefrom(int);
> > | ^~~~~~~~~
> > *** [arc4random.lo] Error code 1
> >
> > make[2]: stopped in
> > /home/disciple/pkgsrc/devel/libbsd/work/libbsd-0.10.0/src
> > 1 error
> >
> > I see you found the same, and it sounds like it may be complicated to
> > resolve.
>
>
> Yes, and pretty much any other function will have some conflicting
> declaration. I had come to the point of successfully building arc4random
> and getopt, but then gave up in front of the fact the rest also required a
> non-negligible amount of work (at least for me).
>
> An alternative approach would be to try to compile only what's needed on
> NetBSD (as I did for wip/signify, but it was a much simpler package).
>
> > Built-in handling for libmd would be a bit more complicated, I think,
> > given there are more OSes involved (and SunOS variants, perhaps).
>
> It would; however libmd is a very small package and compiles fast, even
> on SunOS (tested). So the effort required for a homebrew library could
> be spared?
>
> > I'd prefer it to have a working built-in guard, as above, but that may
> > be non-trivial. Others may have a different opinion than me.
>
> I'm fine with them staying in wip, as long as they work on Linux.
> Platform-specific stuff doesn't really fit pkgsrc.
>
> > Sure, though I don't see many actual dependencies in pkgsrc itself, and
> > presumably 0.10 satisfies them all (no idea), so I don't see a
> > particular urgency to this, personally. It's a nice to have. (Maybe I'm
> > missing something?) Please don't get me wrong, certainly the efforts
> > you're making are appreciated!
>
> Oh, sure, I simply notified it,in light of the relevant TODO note in
> pkgsrc and the fact you asked me about getting te latest version in
> repo. I was playing at packaging a couple of things which required
> libbsd>=0.11.0, and that's why I started the thread in the first place.
>
> The only possibly relevant package to require libbsd>=0.11.0 and libmd
> is the portable version of the got VCS from OpenBSD (wip/got-portable),
> which I packaged a while ago and works fine on Linux as long as these
> dependencies are satisfied. It should be noted, however, that the
> package builds just fine on NetBSD without neither libbsd nor libmd and
> so so does on Darwin.
>
> Then again, I'm perfectly fine with libbsd 0.11.6 staying in wip, given
> the very small amount of packages requiring it. My objective was to get
> a working up to date version in pkgsrc, and apparently this is the case
> now (albeit it needs testing)
>
> Side Note: as far as I cant tell net/ucspi-tools is the only
> optionally-libbsd-dependent package which builds on Linux. I remember
> having some issue with graphics/scrot, which I worked out with a local
> patch.
>
> > Yes, I noticed the entry was wrong for the package, too. It looks like
> > it should actually list four or five different licenses, similar to
> > what you've done for libmd.
>
> To be 100% honest, I find this license (and other similar, like wtf),
> just a way to complicate the life of whomever wants to use author's code
> and implement it in a differently licensed project. The developer may
> just stick with 2-CLAUSE-BSD if complete freedom and minimalism are the
> goal, since the license is well-known, broadly adopted and has no
> pending compliance evaluation on it. I haven't investigated further in
> this topic, but I think that, since all Linux distros provide libbsd and
> libmd binaries, it should really be a concern.
>
> Kindest Regards,
> PVO
>
> --
> ----------------------------+----------------------------
> vms[-at]retrobsd.ddns.net | https://retrobsd.ddns.net
>
Home |
Main Index |
Thread Index |
Old Index