pkgsrc-Bugs archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: pkg/56906: libbsd: update devel/libbsd to 0.11.0



The following reply was made to PR pkg/56906; it has been noted by GNATS.

From: David Holland <dholland-pbugs%netbsd.org@localhost>
To: gnats-bugs%netbsd.org@localhost
Cc: 
Subject: Re: pkg/56906: libbsd: update devel/libbsd to 0.11.0
Date: Sun, 10 Jul 2022 00:33:03 +0000

 Several messages in this discussion weren't sent to gnats.
 
 (Send replies to bug reports to gnats-bugs@; it then remails them.
 Mails sent to just the mailing list or to gnats-admin don't get into
 the bug database unless someone notices and forwards a copy.)
 
 (Generally it seems that if you reply to your own message it does not
 go to the right place by default...)
 
    ------
 
 On Sun, Jul 03, 2022 at 11:32:33AM +0200, Paolo Vincenzo Olivo wrote:
  > Apparently, also libbsd contains Beerware-licensed code [1], so the
  > LICENSE entry of the devel/libbsd package (which is `original-bsd'),
  > should be changed to reflect the fact libbsd it's multi-licensed.
  > 
  > [1] https://libbsd.freedesktop.org/wiki/
  > 
  > - PVO
 
 On Sat, Jul 02, 2022 at 04:37:22PM +0200, Paolo Vincenzo Olivo wrote:
  > So, an update on this topic is due. 
  > 
  > I modified wip/libbsd 's Makefile to enable link-time optimization when
  > supported, and I forced autoreconf at pre-configure stage, since the
  > README for 0.11.6 recommends running `autogen'.
  > 
  > The package now explicitly depends on devel/libmd (available as wip/libmd)
  > through the buildlink methodology.
  > 
  > wip/libbsd (0.11.6) compiles fine on Linux, but any time I try to link
  > against it in a package (with -lbsd or pkg-config), the compiler hangs
  > indefinitely at build stage. This didn't ahppen with previous versions
  > of libbsd.
  > 
  > I was able to reproduce it by attempting to compile the same programs
  > manually, both on my workstation (Slackware 15.0) and on SDF's MetaArray
  > (Debian 11), using a very standard mk.conf. Still same issue, as soon as
  > the compiler tries to link against libbsd, the build freezes forever,
  > without printing any error.
  > 
  > Apparently nobody else reported this so far. Other distributions do not
  > apply any noteworthy patch to libbsd-0.11.6, aside from removing the
  > libtool archives, which for them is common practice. 
  > 
  > I wonder whether this has anything to do with the pkgsrc infrastructure
  > (default compiler flags, linked libs, mitigations) or if I'm missing
  > something very stupid in the Makefile, which somebody more knowledgeable
  > would catch immediately while looking at libbsd's build log.
  > 
  > Anyway, at least I've nailed down the issue to some change introduced
  > between 0.11.3 and 0.11.4, as libbsd-0.11.3 allows me to pass -lbsd
  > safely in other packages, while 0.11.4 does not (resulting in the
  > aforementioned 'hang').
  > 
  > The ChangeLog for libbsd is available at:
  > https://cgit.freedesktop.org/libbsd/log/?h=main
  > 
  > I'd look at any commit between 0.11.3 and 0.11.4. One may be:
  > "Switch md5 compatibility logic back to direct linking"
  > https://cgit.freedesktop.org/libbsd/commit/?h=main&id=e7cf8c5785b14fc8fbd37bb665a5f9a4f28c7888
  > 
  > Would it make sense, for the time being, to pull libbsd-0.11.3, together
  > to libmd-1.0.4 (latest) in the main tree, and try to understand what's
  > the issue with 11.0.6 in the meantime?
  > 
  > Speaking of libmd, I successfully built wip/libmd on NetBSD and Solaris
  > 11 (and Linux). It's a very simple package and unless someone has
  > something to object, I'd consider it ready to import. 
  > The only problem with libmd is that it contains code licensed under
  > 'beer-ware' [1], an extremely permissive and substantially copyright-
  > only license, which, while being supposedly BSD/GPL compatible, is not
  > included among the accepted licenses in licenses.mk  
  > 
  > I found libbsd to be very linux-centric instead (not something
  > unheard of freedesktop's), in spite of their supposed focus on
  > portability. Ii'm confident  that libbsd is routinely tested on Linux,
  > macOS (and perhaps MingW) only.
  > 
  > They recently dropped OpenBSD support, since their obsd-specific
  > arc4random implementation wouldn't compile in it.
  > 
  > On Solaris 11.4 the build fails early on, due to incompatible
  > arc4random_addrandom(). I suppose this would turn out the same way on
  > illumos too.
  > 
  > On NetBSD, arc4random.c, getopt.c and some others compile fine after
  > patching bsd/unistd.h for closefrom(), but I found that multiple other
  > functions need major rework. And at this point, it's probably better to
  > rely on a ad-hoc compat library when porting openbsd software, as done
  > for openssh, or devel/got.
  > 
  > At the same time, a good portion of *BSD utils ported to Linux adn OS X
  > relies on libbsd, so it does make sense to have a working and up to date
  > version in repo.
  >   
  > [1] https://en.wikipedia.org/wiki/Beerware
  > 
  > - PVO
 
 
 
 On Fri, Jul 08, 2022 at 02:39:23PM +0200, Paolo Vincenzo Olivo wrote:
  > (Splitting this reply in 2 mails; one regarding the libbsd bug, the
  > other dedicated to discussing the observations made in the parent message).
  > 
  > On 22/07/03 07:50PM, David H. Gutteridge wrote:
  > >  
  > >  If there are regressions with 0.11.6 vs. 0.11.5 (etc.), those should
  > >  really be reported back upstream (regardless of what we'd do).
  > >  
  > 
  > And we have a culprit.
  > The bug was introduced indeed between 0.11.3 and 0.11.4 [1]. In
  > particular, this added chunk of code in the `install-exec-hook' target of
  > the Makefile.am:
  > 
  > ```
  > + if NEED_TRANSPARENT_LIBMD
  > + # The "GNU ld script" magic is required so that GNU ldconfig does not complain
  > + # about an unknown format file.
  > +   soname=`readlink $(DESTDIR)$(libdir)/libbsd.so`; \
  > +   $(RM) $(DESTDIR)$(libdir)/libbsd.so; \
  > +   (echo '/* GNU ld script'; \
  > +	 echo ' * The MD5 functions are provided by the libmd library. */'; \
  > +	 cat format.ld; \
  > +	 echo "GROUP($(runtimelibdir)/$$soname AS_NEEDED($(MD5_LIBS)))"; \
  > +	)>$(DESTDIR)$(libdir)/libbsd.so
  > ```
  > 
  > Results in having the target shared objects of @libbsd.so (the
  > libbsd.so.0.11.6 shared lib) overwritten with the content echoed in
  > quotes, whenever $(RM) is undefined or null and @libbsd.so isn't
  > consequently preemptively removed.
  > 
  > This leads to a circular reference between the @libbsd.so the libbsd
  > library file, whereby the latter itself points to a non-existing object.
  > And this accounts for the hangings witnessed while trying to link
  > against libbsd.
  > 
  > By skimming through the DESTDIR, I had noticed that libbsd.so.0.11.6
  > curiously consisted of a plain text file, but I couldn't make up my mind
  > about it. It was @RVP, in a private mail exchange, to point me to 
  > where the problem actually lied.
  > 
  > The reason why this happens on pkgsrc, whereas nobody else reported it
  > in the Linux ecosystem, is that GNU Make defines RM among its implicit
  > variables [2] and aliases it to `rm -f'. When the install-exec-hook
  > target is executed, $(RM) is lost and bmake tries to execute @libbsd.so
  > rather than forcibly removing it.
  > 
  > I wouldn't strictly consider this a bug, as much as a portability
  > concern, especially considering freedesktop doesn't mention gmake as a
  > dependency for libbsd. This further reinforces my previous impression of
  > libbsd being very linux-centric and hardly tested outside of Linux, in
  > open contrast with the supposed goals of the project.
  > 
  > I've committed a very simple a patch to wip/libbsd to address the
  > problem (replacing $(RM) with `rm -f'in that single line) . Now libbsd
  > works just as expected, at least with my packages.  Another way to
  > address to problem would be to redefine pkgsrc's RM in the package
  > Makefile and pass it to libbsd as a MAKEFLAG. Or add gmake to USE_TOOLS.
  > Which approach is considered best? 
  > 
  > Putting the other considerations made in this thread temporarily aside I
  > now consider libbsd (+libmd) ready to import. 
  > 
  > [1] 'Switch md5 compatibility logic back to direct linking' \
  > https://gitlab.freedesktop.org/libbsd/libbsd/-/commit/e7cf8c5785b14fc8fbd37bb665a5f9a4f28c7888
  > 
  > [2] https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html
  > 
  > -- 
  > ----------------------------+----------------------------
  > vms[-at]retrobsd.ddns.net   |   https://retrobsd.ddns.net
  > 
 
 
 
 On Fri, Jul 08, 2022 at 04:46:29PM +0200, Paolo Vincenzo Olivo wrote:
  > On 22/07/03 07:50PM, David H. Gutteridge wrote:
  > >  
  > >  Something to think about with both libbsd and libmd is if they should
  > >  provide proper built-in detection. libbsd has an attempt at it, but
  > >  seems wrong to me, as it doesn't work on BSDs. (I guess the assumption
  > >  is that no one would choose to build it in that context, and depending
  > >  packages would conditionally handle this.
  > 
  > Yes, I haven't touched the original builtin.mk of libbsd-0.10.0. Indeed,
  > it only checks if libbsd is already available on the host system.
  > There's no wrapper to test supported functions/calls and verify their
  > compatibility} with those provided by libbsd.  
  > 
  > My take? Somebody more experienced than me could look at the libbsd code
  > and port whatever function is lacking to libncompat. Then all packages
  > relying on libbsd could have a conditional to include the required
  > headers whenever `HAVE_NBCOMPAT_H' is defined. 
  > 
  > NetBSD could for example benefit from having compatibility for
  > explicit_bzero(), readpassprhase(), strtofflags(), pledge(),
  > freezero(), and recallocarray(). iAtthe same time, libbsd includes
  > support for NetBSD-specific functions like strtoi() and strtou(), and
  > uses NetBSD's vins/unvis.
  > 
  > > though they don't seem to entirely consistently or correctly, e.g.,
  > > audio/rtunes or graphics/scrot.)
  > 
  > The packages currently depending on libbsd usually only include
  > libbsd's buildlnk3.mk if the target platform is either Linux or
  > different from *BSD. No clear standardised behavior. Undefined *BSD is a
  > too generic assumption and not very robust imho. It works with older
  > packages, but not with new software designed on and for any of the BSDs,
  > which typically relies on system-specific features, making it non
  > portable across BSDs. Packages currently requiring libbsd on Linux are
  > usually tools ported from OpenBSD, and sometimes from other BSDs. It's
  > not like they will simply compile on NetBSD without patches. 
  > 
  > A major shortcoming of linking to libbsd anywhere outside of *BSD (which
  > is for example what audio/rtunes does), is that the target platform may
  > not need libbsd as much as Linux does or not need it at all. This is the
  > case of illumos for example, and to a good extent, Solaris 11. libbsd
  > won't compile here.  macOS is an exception (libbsd can be found in
  > MacPorts), but this is probably the consequence of a stronger focus and
  > attention to the platform. 
  > 
  > I think this thread as a whole raises the question about the potential
  > need of a reworked userspace shim for BSD functions capable of building
  > on NetBSD first, and other platforms later on.
  > 
  > >  Trying to build the current libbsd 0.10 on NetBSD 9.99.97/amd64 fails
  > >  with:
  > >  
  > >  In file included from arc4random.c:33:
  > >  ../include/bsd/unistd.h:62:6: error: conflicting types for 'closefrom'
  > >      62 | void closefrom();
  > >         |      ^~~~~~~~~
  > >  In file included from ../include/bsd/unistd.h:31,
  > >                    from arc4random.c:33:
  > >  /usr/include/unistd.h:329:6: note: previous declaration of 'closefrom' 
  > >  was here
  > >     329 | int  closefrom(int);
  > >         |      ^~~~~~~~~
  > >  *** [arc4random.lo] Error code 1
  > >  
  > >  make[2]: stopped in 
  > >  /home/disciple/pkgsrc/devel/libbsd/work/libbsd-0.10.0/src
  > >  1 error
  > >  
  > >  I see you found the same, and it sounds like it may be complicated to
  > >  resolve.
  > 
  > 
  > Yes, and pretty much any other function will have some conflicting
  > declaration. I had come to the point of successfully building arc4random
  > and getopt, but then gave up in front of the fact the rest also required a
  > non-negligible amount of work (at least for me). 
  > 
  > An alternative approach would be to try to compile only what's needed on
  > NetBSD (as I did for wip/signify, but it was a much simpler package).
  > 
  > >  Built-in handling for libmd would be a bit more complicated, I think,
  > >  given there are more OSes involved (and SunOS variants, perhaps).
  > 
  > It would; however libmd is a very small package and compiles fast, even
  > on SunOS (tested). So the effort required for a homebrew library could
  > be spared?
  > 
  > >  I'd prefer it to have a working built-in guard, as above, but that may
  > >  be non-trivial. Others may have a different opinion than me.
  > 
  > I'm fine with them staying in wip, as long as they work on Linux.
  > Platform-specific stuff doesn't really fit pkgsrc.
  > 
  > >  Sure, though I don't see many actual dependencies in pkgsrc itself, and
  > >  presumably 0.10 satisfies them all (no idea), so I don't see a
  > >  particular urgency to this, personally. It's a nice to have. (Maybe I'm
  > >  missing something?) Please don't get me wrong, certainly the efforts
  > >  you're making are appreciated!
  > 
  > Oh, sure, I simply notified it,in light of the relevant TODO note in
  > pkgsrc and the fact you asked me about getting te latest version in
  > repo. I was playing at packaging a couple of things which required
  > libbsd>=0.11.0, and that's why I started the thread in the first place.
  > 
  > The only possibly relevant package to require libbsd>=0.11.0 and libmd
  > is the portable version of the got VCS from OpenBSD (wip/got-portable),
  > which I packaged a while ago and works fine on Linux as long as these
  > dependencies are satisfied. It should be noted, however, that  the
  > package builds just fine on NetBSD without neither libbsd nor libmd and
  > so so does on Darwin.
  > 
  > Then again, I'm perfectly fine with libbsd 0.11.6 staying in wip, given
  > the very small amount of packages requiring it. My objective was to get
  > a working up to date version in pkgsrc, and apparently this is the case
  > now (albeit it needs testing)
  > 
  > Side Note: as far as I cant tell net/ucspi-tools is the only
  > optionally-libbsd-dependent package which builds on Linux. I remember 
  > having some issue with graphics/scrot, which I worked out with a local
  > patch. 
  > 
  > >  Yes, I noticed the entry was wrong for the package, too. It looks like
  > >  it should actually list four or five different licenses, similar to
  > >  what you've done for libmd.
  > 
  > To be 100% honest, I find this license (and other similar, like wtf),
  > just a way to complicate the life of whomever wants to use author's code
  > and implement it in a differently licensed project. The developer may
  > just stick with 2-CLAUSE-BSD if complete freedom and minimalism are the
  > goal, since the license is well-known, broadly adopted and has no
  > pending compliance evaluation on it. I haven't investigated further in
  > this topic, but I think that, since all Linux distros provide libbsd and
  > libmd binaries, it should really be a concern.  
  > 
  > Kindest Regards,
  > PVO
  > 
  > -- 
  > ----------------------------+----------------------------
  > vms[-at]retrobsd.ddns.net   |   https://retrobsd.ddns.net
  > 
 


Home | Main Index | Thread Index | Old Index