Subject: experiences with using xlibs (instead of x11-links) and X11BASE as
To: None <tech-pkg@netbsd.org>
From: Jeremy C. Reed <reed@reedmedia.net>
List: tech-pkg
Date: 02/10/2004 22:46:28
Hello again,
The following documents some of my experiences with using xlibs as an
alternative to using x11-links (and no native X and no XFree96-libs). Also
X11BASE is set to LOCALBASE.
xlibs is not a perfect success, but good enough for a wide variety of X
software.
This is NetBSD/i386 1.6.2_RC3.
So first I started with an empty /usr/pkg and empty /var/db/pkg.
I started without any X. And I finished without using any XFree86-*
package (other than non-X-needing imake).
My mk.conf has:
bigfour:/usr/pkgsrc/wip$ cat /etc/mk.conf
PACKAGES=
${_PKGSRCDIR}/packages/${OPSYS}/${OS_VERSION}/${MACHINE_ARCH}
WRKOBJDIR= /tmp/pkgsrc # build here instead of in pkgsrc
OBJHOSTNAME= yes # use work.`hostname`
SU_CMD=/usr/pkg/bin/sudo /bin/sh -c
#SU_CMD=/usr/bin/sudo /bin/sh -c
PYTHON_VERSION_DEFAULT=23pth
X11BASE= ${LOCALBASE}
USE_PKGSRC_X= xlibs
And later added (see below):
# needed for fontconfig
PREFER_PKGSRC= freetype2
# needed for gle
PREFER_PKGSRC+= MesaLib glu
# needed for gnome-lib
BUILDLINK_USE_BUILTIN.xpm= NO
The USE_PKGSRC_X is my creation.
I started with no packages. My installed packages (some don't work
perfect) are:
digest-20021220 Message digest wrapper utility
sudo-1.6.7.5 Allow others to run commands as root
libtool-base-1.5.2 Generic shared library support script (the script itself)
jpeg-6bnb1 IJG's jpeg compression utilities
png-1.2.5nb2 Library for manipulating PNG images
pth-2.0.0 GNU Portable Thread library
glib-1.2.10nb5 Some useful routines for C programming
perl-5.6.1nb10 Practical Extraction and Report Language
xextensions-1.0.1 Various extension headers for xlibs from freedesktop.org
xproto-6.6.1 X protocol and ancillary headers
xtrans-0.1 Network API translation layer to insulate X
pkgconfig-0.15.0 System for managing library compile/link flags
libXau-0.1.1 Authorization Protocol for X from freedesktop.org
gmake-3.80nb2 GNU version of 'make' utility
libX11-6.2.1 Base X libraries from freedesktop.org
libICE-6.3.1nb1 Inter Client Exchange (ICE) library for X
libSM-6.0.1 SM Extension
libXt-0.1.4 X Toolkit Intrinsics library
libXmu-6.2.1nb2 X Miscellaneous Utilities library
glu-6.0 GLU polygon tesselation facility for Mesa
gal-0.22nb4 Gnome Application Libraries
createbuildlink-3.1 shell script to help creating buildlink[23].mk files
pkgdiff-0.113 Tools to ease creating and maintaining patches for pkgsrc
libXv-2.2.1 Xv Extension library
libXfont-1.4.1 Xfont extension (Library)
resourceext-1.0 X Resource usage extension headers
libXres-1.0.1 X Resource usage library
libXext-6.4.2 X Extension library
libXi-6.0.1nb1 X Input extension library
gtk+-1.2.10nb6 The GIMP Toolkit - libraries for building X11 user interfaces
dillo-0.7.3 Very small and fast graphical web-browser
vim-share-6.2.0 Data files for the vim editor (vi clone)
manedit-0.5.6nb2 UNIX Manual (man, manpage) page editor, viewer, and browser for X11
libslang-1.4.9nb1 Routines for rapid alpha-numeric terminal applications development
xjed-0.99.16 Extensible folding editor with Emacs/WordStar/EDT emulations, X11 version
freetype2-2.1.5 Font rendering engine and library API
tiff-3.6.1 Library and tools for reading and writing TIFF data files
gdk-pixbuf-0.22.0nb2 The GNOME image loading library
ORBit-0.5.15nb6 High-performance CORBA ORB with support for the C language
unzip-5.50nb2 List, test and extract compressed files in a ZIP archive
zip-2.3nb2 Create/update ZIP files compatible with pkzip
m4-1.4nb1 GNU version of Unix m4 macro-processor
autoconf213-2.13 Generates automatic source code configuration scripts (old version)
mozilla-1.6 Full featured gecko-based browser
imake-4.3.0nb1 Imake and other utilities from XFree86
xpm-3.4knb1 The X Pixmap library
aterm-0.4.2nb4 Aterm (Afterstep XVT) - a VT102 emulator for the X window system
rxvt-2.7.10nb3 Low memory usage xterm replacement that supports color
xclip-0.08nb1 Command line interface to the X windows clipboard
pkglint-3.67 Verifier for NetBSD packages and complete pkgsrc tree
libXres-1.0.1nb1 X Resource usage library
ion-20020207 Keyboard friendly tiling window manager
expat-1.95.6nb1 XML parser library written in C
fontconfig-2.2.1nb3 Library for configuring and customizing font access
render-0.8 Headers for X11 render extension
Xrender-0.8.3nb1 Client library for the X Rendering Extension protocol
Xft2-2.1.2nb1 Library for configuring and customizing font access
libungif-4.1.0.1 Tools and library routines for working with GIF images
imlib2-1.1.0nb1 Image manipulation library
waimea-0.4.0nb6 Fast and highly customizable virtual window manager
ratpoison-1.2.2 Simple window manager with no fat library dependencies
libiconv-1.9.1nb1 Character set conversion library
netpbm-10.11.8nb1 Toolkit for conversion of images between different formats
imlib-1.9.14nb5 Image manipulation library for X11
gnome-dirs-1.2 Shared GTK/GNOME (v1 and v2) directories
gnome1-dirs-1.2 Shared GTK/GNOME (v1) directories
libaudiofile-0.2.5 Sound library for SGI audio file
esound-0.2.32 The Enlightened sound daemon
gnome-libs-1.4.2nb1 Libraries for the GNU Network Object Model Environment
icewm-gnome-1.2.13nb1 Small, fast and light-weight window manager (GNOME version)
cxunzip-0.98nb1 Cloned Xunzip is a GNOME zip-archive uncompressing program
popt-1.7nb4 Command line option parsing library
xdg-dirs-1.0 Shared XDG directories
gnome2-dirs-1.2nb1 Shared GTK/GNOME (v2) directories
gnome-mime-data-2.4.1 MIME and Application database for GNOME2
db3-3.11.2nb1 Sleepycat Software's Berkeley DB version 3
libxml-1.8.17nb1 XML parser, mainly used by the GNOME project
oaf-0.6.10nb4 New object activation framework for GNOME
GConf-1.0.9nb5 Configuration database system used by GNOME
gnome-vfs-1.0.5nb5 GNOME Virtual File System
xmlcatmgr-2.0beta1 XML and SGML catalog manager
docbook-xml-4.2nb5 XML DTD designed for computer documentation
libxml2-2.6.5 XML parser library from the GNOME project
libxslt-1.1.2nb1 XSLT parser library from the GNOME project
p5-XML-Parser-2.31nb1 Perl extension interface to James Clark's XML parser, expat
intltool-0.30 Toolbox for internationalisation
scrollkeeper-0.3.14nb1 Open Document Cataloging Project
control-center-1.4.0.4nb5 GNOME control-center development library
readline-4.3pl5 GNU library that can recall and edit previous input
guile-1.6.4 GNU's Ubiquitous Intelligent Language for Extension
MesaLib-6.0 Graphics library similar to SGI's OpenGL
glut-6.0 GLUT Graphics library similar to SGI's OpenGL
gle-3.1.0 GL subroutines for drawing tubing and extrusions
libglade-0.17nb5 Runtime interpreter for GLADE GUI files
xlibs-1.0nb1 Meta-package for X Libraries and Protocol Headers Project
xscreensaver-4.14nb2 Screen saver and locker for the X window system
xscreensaver-gnome-4.14nb3 Screen saver and locker for the X window system (GNOME applet)
gdk-pixbuf-gnome-0.22.0nb2 The GNOME image loading library GNOME canvas support add-on
gnome-pixmaps-1.4.2 Pixmaps for gnome-core
ghostscript-fonts-6.0 Postscript fonts for Aladdin Ghostscript
ghostscript-gnu-7.05nb1 GNU Postscript interpreter
libunicode-0.4 Library for manipulating Unicode characters and strings
gnome-print-0.36nb4 Prototype implementation of the Gnome Printing Architecture
bison-1.875nb1 GNU yacc(1) replacement
bonobo-1.0.18nb7 Architecture for creating reusable software components
soup-0.7.10nb7 Callback-based mechanism for sending and servicing SOAP requests
automake14-1.4.6 GNU Standards-compliant Makefile generator (old version)
autoconf-2.59 Generates automatic source code configuration scripts
libwww-5.4.0nb1 The W3C Reference Library
vim-gtk-6.2.0nb1 Vim editor (vi clone) with X11 GTK+ GUI
xvier-1.0nb1 Four-in-a-row game
Xaw3d-1.5 3-D widgets based on the Athena Widget set
xcolorsel-1.1anb1 Browse and select named X colors from rgb.txt
(I tried to get gnome-games installed, but glibwww failed to build
probably due to my libwww problems as mentioned in other email.)
I modified mk/bsd.pkg.mk. mk/buildlink?/bsd.buildlink?.mk to work with
USE_PKGSRC_X=xlibs to use xlibs instead of x11-links. I imported xlibs
meta-package into pkgsrc-wip.
Below are three patches; they need improvements, but they helped. Notice
that some of the following patches contain some changes (also posted last
week) about allowing X11BASE to be same as LOCALBASE.
(I have more comments below the patches.)
diff -b -u -r1.1376 bsd.pkg.mk
--- mk/bsd.pkg.mk 2004/02/06 18:45:02 1.1376
+++ mk/bsd.pkg.mk 2004/02/11 00:40:11
@@ -197,6 +197,21 @@
. endif
.endif
+.if defined(USE_X11) && (${USE_PKGSRC_X} == "xlibs") && \
+ (!defined(USE_X11_LINKS) || ${USE_X11_LINKS} == YES)
+DEPENDS+= xlibs>=1.0:../../wip/xlibs
+. if empty(USE_BUILDLINK2:M[nN][oO])
+. include "../../wip/xlibs/buildlink2.mk"
+. else
+USE_BUILDLINK3= YES
+. include "../../wip/xlibs/buildlink3.mk"
+. endif
+.endif
+
+.if defined(USE_IMAKE) && (${USE_PKGSRC_X} == "xlibs")
+BUILD_DEPENDS+= imake>=4.0:../../x11/imake
+.endif
+
.if empty(DEPOT_SUBDIR)
PKG_FAIL_REASON+= "DEPOT_SUBDIR may not be empty."
.endif
Index: mk/buildlink2/bsd.buildlink2.mk
===================================================================
RCS file: /cvsroot/pkgsrc/mk/buildlink2/bsd.buildlink2.mk,v
retrieving revision 1.112
diff -b -u -r1.112 bsd.buildlink2.mk
--- mk/buildlink2/bsd.buildlink2.mk 2004/02/06 04:37:02 1.112
+++ mk/buildlink2/bsd.buildlink2.mk 2004/02/11 00:40:17
@@ -88,6 +88,12 @@
.endif
.if defined(USE_X11)
+.if ${USE_PKGSRC_X} == "xlibs"
+USE_X11_LINKS= NO
+# XXX the following doesn't work; this is done in mk/bsd.pkg.mk instead.
+# DEPENDS+= xlibs>=1.0:../../wip/xlibs
+#.include "../../wip/xlibs/buildlink2.mk"
+.else
USE_X11_LINKS?= YES
. if empty(USE_X11_LINKS:M[nN][oO])
BUILD_DEPENDS+= x11-links>=0.20:../../pkgtools/x11-links
@@ -99,6 +105,7 @@
_BLNK_LDFLAGS+= -Wl,${_OPSYS_RPATH_NAME}${X11BASE}/lib
.endif
.endif
+.endif
CONFIGURE_ENV+= BUILDLINK_CPPFLAGS="${_BLNK_CPPFLAGS}"
MAKE_ENV+= BUILDLINK_CPPFLAGS="${_BLNK_CPPFLAGS}"
@@ -210,7 +217,11 @@
_LT_ARCHIVE_TRANSFORM_SED= \
-e "s|$/usr\(/lib/[^ ]*\.la\)|${BUILDLINK_DIR}\1|g" \
-e "s|${LOCALBASE}\(/lib/[^ ]*\.la\)|${BUILDLINK_DIR}\1|g" \
+
+.if ${LOCALBASE} != ${X11BASE}
+_LT_ARCHIVE_TRANSFORM_SED+= \
-e "s|${X11BASE}\(/lib/[^ ]*\.la\)|${BUILDLINK_DIR}\1|g"
+.endif
_LT_ARCHIVE_TRANSFORM= \
${SED} ${_LT_ARCHIVE_TRANSFORM_SED} $${file} > $${dest}
@@ -256,14 +267,14 @@
if [ ! -f $${cookie} ]; then \
${ECHO_BUILDLINK_MSG} "Linking ${.TARGET:S/-buildlink//}
files into ${BUILDLINK_DIR}."; \
${MKDIR} ${BUILDLINK_DIR}; \
- case "${BUILDLINK_PREFIX.${.TARGET:S/-buildlink//}}" in \
+ buildlink_dir="${BUILDLINK_DIR}"; \
+ if [ "${X11BASE}" != "${LOCALBASE}" ] ; then \
+ case "${BUILDLINK_PREFIX.${.TARGET:S/-buildlink//}}" in\
${X11BASE}) \
buildlink_dir="${BUILDLINK_X11_DIR}"; \
;; \
- *) \
- buildlink_dir="${BUILDLINK_DIR}"; \
- ;; \
esac; \
+ fi; \
pkg_prefix=; \
if [ -n "${BUILDLINK_PKGBASE.${.TARGET:S/-buildlink//}}" ]; then \
pkg_prefix=`${PKG_INFO} -qp ${BUILDLINK_PKGBASE.${.TARGET:S/-buildlink//}} | ${AWK} '{ sub("${BUILDLINK_PREFIX.${.TARGET:S/-buildlink//}}", "", $$2); sub("/", "", $$2); print $$2; exit }'`/; \
@@ -351,24 +362,30 @@
#
# Convert direct paths to shared libraries into "-Ldir -llib" equivalents.
#
+.if ${X11BASE} != ${LOCALBASE}
_BLNK_TRANSFORM+= p:${X11BASE}
+.endif
_BLNK_TRANSFORM+= p:${LOCALBASE}
_BLNK_TRANSFORM+= p:/usr/lib
#
# Convert direct paths to static libraries in ${LOCALBASE} or ${X11BASE}
# into references into ${BUILDLINK_DIR}.
#
+.if ${X11BASE} != ${LOCALBASE}
_BLNK_TRANSFORM+= static:${X11BASE}:${_BLNK_MANGLE_DIR.BUILDLINK_X11_DIR}
+.endif
_BLNK_TRANSFORM+= static:${LOCALBASE}:${_BLNK_MANGLE_DIR.BUILDLINK_DIR}
#
# Transform references into ${X11BASE} into ${BUILDLINK_X11_DIR} but if
# the package doesn't use X11, then just remove these references altogether.
#
-.if defined(USE_X11)
+.if ${X11BASE} != ${LOCALBASE}
+. if defined(USE_X11)
_BLNK_TRANSFORM+= I:${X11BASE}:${_BLNK_MANGLE_DIR.BUILDLINK_X11_DIR}
_BLNK_TRANSFORM+= L:${X11BASE}:${_BLNK_MANGLE_DIR.BUILDLINK_X11_DIR}
-.else
+. else
_BLNK_TRANSFORM+= r:${X11BASE}
+. endif
.endif
#
# Transform references into ${LOCALBASE} into ${BUILDLINK_DIR}.
@@ -924,8 +941,10 @@
_BLNK_CHECK_PATTERNS+= -e "-I${LOCALBASE}/[a-rt-z]"
_BLNK_CHECK_PATTERNS+= -e "-L${LOCALBASE}/[a-rt-z]"
+.if ${X11BASE} != ${LOCALBASE}
_BLNK_CHECK_PATTERNS+= -e "-I${X11BASE}/"
_BLNK_CHECK_PATTERNS+= -e "-L${X11BASE}/"
+.endif
buildlink-check:
@if [ -f ${_BLNK_WRAP_LOG} ]; then \
Index: mk/buildlink3/bsd.buildlink3.mk
===================================================================
RCS file: /cvsroot/pkgsrc/mk/buildlink3/bsd.buildlink3.mk,v
retrieving revision 1.78
diff -b -u -r1.78 bsd.buildlink3.mk
--- mk/buildlink3/bsd.buildlink3.mk 2004/02/06 18:45:03 1.78
+++ mk/buildlink3/bsd.buildlink3.mk 2004/02/11 00:40:25
@@ -68,8 +68,18 @@
#
BUILDLINK_DEPENDS?= # empty
-X11_LINKS_SUBDIR= share/x11-links
.if defined(USE_X11)
+.if ${USE_PKGSRC_X} == "xlibs"
+USE_X11_LINKS= NO
+BUILDLINK_DEPENDS+= xlibs
+BUILDLINK_PACKAGES+= xlibs
+# need to include this or list every dependency here
+.include "../../wip/xlibs/buildlink3.mk"
+BUILDLINK_DEPENDS.xlibs+= xlibs>=1.0
+BUILDLINK_DEPMETHOD.xlibs= full
+BUILDLINK_PKGSRCDIR.xlibs= ../../wip/xlibs
+.else
+X11_LINKS_SUBDIR= share/x11-links
USE_X11_LINKS?= YES
. if !empty(USE_X11_LINKS:M[yY][eE][sS])
BUILDLINK_DEPENDS+= x11-links
@@ -84,6 +94,7 @@
.else
_BLNK_X11_LINKS_PACKAGE= # empty
.endif
+.endif
# _BLNK_DEPENDS contains all of the unique elements of BUILDLINK_DEPENDS.
# _BLNK_PACKAGES contains all of the unique elements of BUILDLINK_PACKAGES.
@@ -205,7 +216,7 @@
BUILDLINK_USE_BUILTIN.${_pkg_}?= no
. endif
#
-# If we prefer the pkgsrc version, then don't user the built-in package.
+# If we prefer the pkgsrc version, then don't use the built-in package.
#
.if !empty(PREFER_PKGSRC:M[yY][eE][sS]) || \
!empty(PREFER_PKGSRC:M${_pkg_})
@@ -504,10 +515,12 @@
${_BLNK_COOKIE.${_pkg_}}:
${_PKG_SILENT}${_PKG_DEBUG} \
+ buildlink_dir="${BUILDLINK_DIR}"; \
+ if [ "${X11BASE}" != "${LOCALBASE}" ] ; then \
case ${BUILDLINK_PREFIX.${_pkg_}} in \
${X11BASE}) buildlink_dir="${BUILDLINK_X11_DIR}" ;; \
- *) buildlink_dir="${BUILDLINK_DIR}" ;; \
esac; \
+ fi; \
cd ${BUILDLINK_PREFIX.${_pkg_}}; \
${_BLNK_FILES_CMD.${_pkg_}} | \
while read file; do \
I had many issues when working on this. For example, the pkgsrc-wip xlibs
packages needed work, such as properly having correct dependencies.
Sometimes I get a dependency outputted for xlibs multiple times (but it
still works fine).
Packages using xmkmf stopped working, so I made it so imake would get
installed. (imake doesn't need any X to build. It should probably be
located at devel/imake or sysutils/imake.)
mozilla installed, but many components were not built and not installed.
Missing for example are: libgfxxprint.so and libxlibrgb.so and related.
(These were also missing under Linux.) I didn't look at build logs to see
what was missing yet. It worked partially, but later hung.
Also, rxvt failed to run. rxvt said:
**Pth** SCHEDULER INTERNAL ERROR: no more thread(s) available to schedule!?!?
Abort trap
I think I have a problem with xlibs using threads.
(Please note that on Linux systems, using glibc with its glibc threads
patch, rxvt worked fine with only xlibs.)
I also had to patch my installed /usr/pkg/include/X11/Xos_r.h -- it was
forcing use of getpwuid_r and getpwnam_r which I don't think NetBSD has. I
don't have a solution for this yet, so my fix was to not use these
reenterant functions. (I didn't have this problem with Linux.)
wm/ion built and installed fine, but is missing run dependencies like
xterm and run-mailcap. (Where is run-mailcap?)
Here is an example of libraries used by working x11/aterm:
$ sudo ldd /usr/pkg/bin/aterm
/usr/pkg/bin/aterm:
-lXau.0 => /usr/pkg/lib/libXau.so.0
-lX11.6 => /usr/pkg/lib/libX11.so.6
-lpthread.20 => /usr/pkg/lib/libpthread.so.20
-lXext.6 => /usr/pkg/lib/libXext.so.6
-lICE.6 => /usr/pkg/lib/libICE.so.6
-lSM.6 => /usr/pkg/lib/libSM.so.6
-lXpm.4 => /usr/pkg/lib/libXpm.so.4
-lc.12 => /usr/lib/libc.so.12
Another problem was that fontconfig brought in freetype2, but then
freetype2 didn't get buildlink'd because BUILDLINK_IS_BUILTIN.freetype2
was YES and BUILDLINK_USE_BUILTIN.freetype2 was YES. So in my mk.conf I
added:
PREFER_PKGSRC= freetype2
(This should not be needed. My LOCALBASE is /usr/pkg/.
../../graphics/freetype2/buildlink3.mk assumes that X11BASE is native?)
I also had a problem with xpm with the error:
make: don't know how to make _BUILDLINK_USE. Stop But that disappeared
from some other change I made. I don't know how I caused it in the first
place. (But the error, I found out, is documented in the buildlink3
documentation.)
Later gnome-libs needed xpm but wasn't buildlinked. (Although other
packages using xpm were fine.) I don't see any PREFER_PKGSRC option for
this in the ../../graphics/xpm/buildlink3.mk file.
I added to my mk.conf:
BUILDLINK_USE_BUILTIN.xpm= NO
Why did I have to do that? (Maybe graphics/xpm/buildlink3.mk assumes
X11BASE is native?)
Then I had similar issue with gle -- it needed libGL. But MesaLib was not
buildlinked. Then I found that glu wasn't buildlinked. I guess both think
X11BASE is builtin. My workaround is:
PREFER_PKGSRC+= MesaLib glu
Another problem was Xaw3d needed xpm buildlinked (see my other email).
And another issue is x11/xcolorsel needs a RgbTxtFile (like
/usr/X11R6/lib/X11/rgb.txt) available.
Also, I will need to install XFree86-* fonts. These packages need to be
made so that they don't need any XFree86-libs to build; this may be hard
because they need XFree86-clients which utilize some XFree86-libs parts.
(xlibs developers -- who also work on fontconfig, Xft, and freetype2 --
have been plannig to provide fonts using more up-to-date formats and
modern tools.)
I am hoping that USE_PKGSRC_X can be implemented for pkgsrc, with a first
working choices to be NONE (or empty) to use native and also have
"XFree86" (or "XFree") to use the x11/XFree86-libs which is now in pkgsrc
and works fine. x11-links should only be used if native X libararies are
used.
xlibs is not ready for NetBSD for everything, but in my experiences
(including Linux) it has worked fine to provide X libraries for many X
applications. xlibs under NetBSD is a good replacement for standard X
libraries used by many applications.
Also, xlibs helps with problems with conflicting files of libGL*, and
render-related files (when installing X11BASE to LOCALBASE or probably
using xpkgwedge too) because they aren't installed twice.
An example of something that doesn't build properly with xlibs is
openmotif. (Why does KDE needed openmotif? Isn't QT good enough?)
By the way, icewm also doesn't work, because of:
iconv doesn't supply (sufficient) 646 to Unicode converters.
issues. This has been reported before and is unrelated to xlibs.
Jeremy C. Reed
http://bsd.reedmedia.net/
p.s. I need to import to pkgsrc-wip libXaw (which I committed to
freedesktop.org) and a couple other xlibs, like libXdmcp (X Display
Manager Control Protocol library) and maybe xlibs' libXpm. Unless someone
volunteers first ...