Re: Cross-compilation problems

[Taylor R Campbell <campbell+netbsd-tech-pkg%mumble.net@localhost>]

> Date: Mon, 8 Jul 2019 09:11:23 +0200
> From: tlaronde%polynum.com@localhost
> 
> I need to set USE_CWRAPPERS=no or the compilation fails for the same
> reason as before.

Update: There are two issues:

1. cwrappers doesn't know about passing --sysroot=@CROSS_DESTDIR@, as
   mk/wrapper/cmd-sink-cross-gcc &c. set.

   For now, I've added USE_CWRAPPERS=no to HOWTO-use-crosscompile, for
   the packages that are being cross-compiled.

2. In mk/compiler/gcc.mk, the setting of _CC to the cross-compiler is
   passed down to recursive makes even when building native packages,
   which confused cwrappers when natively compiling dependencies of
   cross-compiled packages.

   I've committed a patch to avoid doing this by making the assignment
   `MAKEFLAGS+= _CC=${_CC:Q}' conditional on USE_CROSS_COMPILE=no.

> With USE_CWRAPPERS=no, then it succeeds when it can succeed. The
> /etc/mk.conf is then this:
> 
> Note: contrary to the howto, I revert the logic: explicitely tell to
> cross compile with this at the very beginning of /et/mk.conf:
> 
> USE_CROSS_COMPILE?= no

That probably won't work.  The comment I left says:

   # XXX This currently can't be set to `yes' on the command line,
   # which is a bug.
   USE_CROSS_COMPILE?=  yes

I don't remember exactly why passing USE_CROSS_COMPILE=yes on the
command line doesn't work.  But if I wrote that comment, there was
probably a reason that needs to be diagnosed.  If you change this
without fixing the underlying bug, you'll probably get inscrutable
errors later on.

> #========== XCOMPBEGIN
> # (From Taylor R. Campbell Cross-compilation in pkgsrc (user's guide).)
> 
> .if ${USE_CROSS_COMPILE:M[yY][eE][sS]}

This is still wrong.  You need copy & paste _exactly_ what the
documentation said:

   .if !empty(USE_CROSS_COMPILE:M[yY][eE][sS])

Otherwise you will get inscrutable errors later on.

> # This is a kludge for cross-libtool.
> #
> CROSSBASE= ${LOCALBASE}/cross-${TARGET_ARCH:U${MACHINE_ARCH}}

The CROSSBASE-related logic had bitrotted.  I've committed a fix for
the bitrot, eliminating the last vestiges of CROSSBASE.  It is no
longer necessary to set CROSSBASE in your mk.conf: cross-libtoool-base
will always install into $PREFIX/cross-$ARCH, and the pkgsrc
cross-compilation infrastructure will always look there for the
cross-libtool.

> procmail: FAILED
> 	fails because it tries to execute a test program it compiles.

I've committed a workaround for this on NetBSD by pregenerating the
configuration and eyeballing it to confirm it makes sense.

> motif: FAILED
> 	fails because it searches natively for X11R6 (while I have on
> 	the target X cross-compiled by NetBSD under X11R7; making a symlink
> 	on the destdir doesn't help since motif doesn't look there;
> 	shouldn't be a lot of work to fix since the OpenGroup sources were
> 	clean)

This failure was an easy bug in the configure script.  However, motif
has some internal tools that are more trouble to convert to
cross-building because it creates libXm for use at compie-time _and_
for installation on the target system.

I have a patch for this, but it's a little involved and may add some
maintenance burden even beyond the cross-compilation case to keep the
Makefile.in patch update so I haven't committed it.

> samba: FAILED
> 	not a surprise since it depends on a lot of things
> 	particularily Perl which compiles a test program and tries to run it;
> 	typical situation where cross-compilation can't work.

Cross-compiling Perl is a problem, yes.  What you can do is:

(a) cross-build all of the dependencies of Perl,
(b) copy the dependencies to the target machine,
(c) natively build a Perl package on the target machine,
(d) copy the Perl package back onto the cross-compiling system,
(e) continue building things that depend on Perl.

(Yes, this is a bad system.  It would be nice if someone could make
Perl and Python cross-compilable.)

> The fact too that a huge number of sources depend unnecessarily of a
> bunch of extra things (Perl to replace a magic string in a file because
> people don't know about ed(1) or because some systems don't provide
> ed(1) because it is POSIX and Gnu is Not Unix...) is no help at all.

This particular use of Perl -- that is, running a native Perl during
the build -- is not a problem.  Packages with

USE_TOOLS+= perl

work just fine.

From eae9ab10b7102343ab8dd752ff93f159303556a8 Mon Sep 17 00:00:00 2001
From: Taylor R Campbell <campbell+pkgsrc%mumble.net@localhost>
Date: Mon, 8 Jul 2019 17:54:17 +0000
Subject: [PATCH] Make motif cross-compilable again.

---
 x11/motif/Makefile                            | 26 ++++-
 x11/motif/distinfo                            |  2 +
 .../patches/patch-config_util_Makefile.in     | 42 ++++++++
 x11/motif/patches/patch-tools_wml_Makefile.in | 99 +++++++++++++++++++
 4 files changed, 167 insertions(+), 2 deletions(-)
 create mode 100644 x11/motif/patches/patch-config_util_Makefile.in
 create mode 100644 x11/motif/patches/patch-tools_wml_Makefile.in

diff --git a/x11/motif/Makefile b/x11/motif/Makefile
index 8e777c6c9d4..4c531e4328c 100644
--- a/x11/motif/Makefile
+++ b/x11/motif/Makefile
@@ -47,8 +47,30 @@ CONFIGURE_ARGS+=	--enable-xft
 CFLAGS.SunOS+=		-D_POSIX_PTHREAD_SEMANTICS
 
 .if !empty(USE_CROSS_COMPILE:M[yY][eE][sS])
-MAKE_FLAGS+=	CROSS_COMPILING=yes CC_FOR_BUILD=${NATIVE_CC:Q} \
-    CPPFLAGS_FOR_BUILD=-I${PREFIX}/include
+TOOL_DEPENDS=		${PKGNAME}:../../${PKGPATH}
+
+MAKE_FLAGS+=		CC_FOR_BUILD=${NATIVE_CC:Q}
+MAKE_FLAGS+=		CPPFLAGS_FOR_BUILD=-I${PREFIX}/include
+MAKE_FLAGS+=		LDFLAGS_FOR_BUILD=-L${PREFIX}/lib\ -Wl,-R${PREFIX}/lib
+
+# Motif's configure checks for /usr/X/include/X11/X.h and
+# /usr/X11R6/include/X11/X.h to decide whether the default prefix
+# should be /usr/X, /usr/X11R6, or /usr.  But we set the prefix anyway,
+# so this is moot.
+CONFIGURE_ARGS+=	ac_cv_file__usr_X_include_X11_X_h=false
+CONFIGURE_ARGS+=	ac_cv_file__usr_X11R6_include_X11_X_h=false
+.endif
+
+SUBST_CLASSES+=		cross
+SUBST_STAGE.cross=	pre-configure
+SUBST_MESSAGE.cross=	Choosing between cross-compilation or not.
+SUBST_FILES.cross+=	tools/wml/Makefile.in
+.if !empty(USE_CROSS_COMPILE:M[yY][eE][sS])
+SUBST_SED.cross+=	-e 's,@IF_NATIVE@,\#,g'
+SUBST_SED.cross+=	-e 's,@IF_CROSS@,,g'
+.else
+SUBST_SED.cross+=	-e 's,@IF_NATIVE@,,g'
+SUBST_SED.cross+=	-e 's,@IF_CROSS@,\#,g'
 .endif
 
 # 1) Fix the man pages to refer to the correct sections.
diff --git a/x11/motif/distinfo b/x11/motif/distinfo
index d44791d0b09..aede7ff184b 100644
--- a/x11/motif/distinfo
+++ b/x11/motif/distinfo
@@ -8,6 +8,7 @@ SHA1 (patch-Makefile.in) = 9f4573cd47989a821ea616df53da924766fe3d39
 SHA1 (patch-clients_uil_UilDB.c) = 781b52e915e667eeb02359a51cfeefdb9556c4dd
 SHA1 (patch-clients_uil_UilDefI.h) = 6447a306809e65b4993d79e14a989a01aed2e0bc
 SHA1 (patch-clients_uil_UilDiags.c) = 39ac8307219b435baf3d1bf829809fcb6536b1a9
+SHA1 (patch-config_util_Makefile.in) = f02af49e80a7df5db0bf51fff8c87bea81e1502b
 SHA1 (patch-doc_man_man3_Makefile.am) = b396f5e1235834f507a415c62b0efaa939943f1d
 SHA1 (patch-doc_man_man4_Makefile.am) = 154071494cf54a65e2e266861c81771a14156cae
 SHA1 (patch-doc_man_man5_Makefile.am) = 2f6e0cc239faf53e3391aef9da3def934023928e
@@ -17,3 +18,4 @@ SHA1 (patch-lib_Xm_Xmfuncs.h) = fef6a642988a8024da0bff00cdc40dc485695e2b
 SHA1 (patch-lib_Xm_Xmos.c) = 88af3627fad7c9ec40db0e8aca47e9617fe8c53b
 SHA1 (patch-lib_Xm_XpmI.h) = b27e0156bc2dd990a4d9642186b1638276054314
 SHA1 (patch-tools_wml_wmluiltok.l) = 1e79df5605ad8b25a55fbb617a707d27d27e3f2a
+SHA1 (patch-tools_wml_Makefile.in) = fd4d6385714e275337e9549e75c3fd23ffeb494d
diff --git a/x11/motif/patches/patch-config_util_Makefile.in b/x11/motif/patches/patch-config_util_Makefile.in
new file mode 100644
index 00000000000..24b358e91c3
--- /dev/null
+++ b/x11/motif/patches/patch-config_util_Makefile.in
@@ -0,0 +1,42 @@
+$NetBSD$
+
+--- config/util/Makefile.in.orig	2017-12-05 12:49:57.000000000 +0000
++++ config/util/Makefile.in
+@@ -89,7 +89,7 @@ POST_UNINSTALL = :
+ build_triplet = @build@
+ host_triplet = @host@
+ target_triplet = @target@
+-noinst_PROGRAMS = makestrs$(EXEEXT)
++noinst_PROGRAMS = makestrs
+ subdir = config/util
+ ACLOCAL_M4 = $(top_srcdir)/aclocal.m4
+ am__aclocal_m4_deps = $(top_srcdir)/ac_find_xft.m4 \
+@@ -186,6 +186,7 @@ AUTOHEADER = @AUTOHEADER@
+ AUTOMAKE = @AUTOMAKE@
+ AWK = @AWK@
+ CC = @CC@
++CC_FOR_BUILD = $(CC)
+ CCDEPMODE = @CCDEPMODE@
+ CDE_CONFIGURATION_TOP = @CDE_CONFIGURATION_TOP@
+ CDE_INSTALLATION_TOP = @CDE_INSTALLATION_TOP@
+@@ -193,6 +194,7 @@ CFLAGS = @CFLAGS@
+ CONFDIR = @CONFDIR@
+ CPP = @CPP@
+ CPPFLAGS = @CPPFLAGS@
++CPPFLAGS_FOR_BUILD = $(CPPFLAGS)
+ CURRENT = @CURRENT@
+ CYGPATH_W = @CYGPATH_W@
+ DEFS = @DEFS@
+@@ -383,9 +385,9 @@ clean-noinstPROGRAMS:
+ 	echo " rm -f" $$list; \
+ 	rm -f $$list
+ 
+-makestrs$(EXEEXT): $(makestrs_OBJECTS) $(makestrs_DEPENDENCIES) $(EXTRA_makestrs_DEPENDENCIES) 
+-	@rm -f makestrs$(EXEEXT)
+-	$(AM_V_CCLD)$(LINK) $(makestrs_OBJECTS) $(makestrs_LDADD) $(LIBS)
++makestrs: $(makestrs_DEPENDENCIES) $(EXTRA_makestrs_DEPENDENCIES) 
++	@rm -f makestrs
++	$(AM_V_CCLD)$(CC_FOR_BUILD) $(CPPFLAGS_FOR_BUILD) -o $@ makestrs.c
+ 
+ mostlyclean-compile:
+ 	-rm -f *.$(OBJEXT)
diff --git a/x11/motif/patches/patch-tools_wml_Makefile.in b/x11/motif/patches/patch-tools_wml_Makefile.in
new file mode 100644
index 00000000000..b3eb63b890f
--- /dev/null
+++ b/x11/motif/patches/patch-tools_wml_Makefile.in
@@ -0,0 +1,99 @@
+$NetBSD$
+
+Conditionalize the build of the wml tools when cross-compiling:
+
+- If we're compiling natively, link against the newly built libXm.la.
+
+- If we're cross-compiling, use the natively installed -lXm
+
+--- tools/wml/Makefile.in.orig	2017-12-05 12:49:59.000000000 +0000
++++ tools/wml/Makefile.in
+@@ -130,7 +130,7 @@ am__v_lt_0 = --silent
+ am__v_lt_1 = 
+ wmldbcreate_SOURCES = $(srcdir)/wmldbcreate.c
+ wmldbcreate_OBJECTS = wmldbcreate.$(OBJEXT)
+-wmldbcreate_DEPENDENCIES = ../../lib/Xm/libXm.la
++@IF_NATIVE@wmldbcreate_DEPENDENCIES = ../../lib/Xm/libXm.la
+ am_wmluiltok_OBJECTS = wmluiltok.$(OBJEXT)
+ wmluiltok_OBJECTS = $(am_wmluiltok_OBJECTS)
+ wmluiltok_DEPENDENCIES =
+@@ -150,8 +150,8 @@ DEFAULT_INCLUDES = -I.@am__isrc@ -I$(top
+ depcomp = $(SHELL) $(top_srcdir)/depcomp
+ am__depfiles_maybe = depfiles
+ am__mv = mv -f
+-COMPILE = $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) \
+-	$(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS)
++COMPILE = $(CC_FOR_BUILD) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) \
++	$(CPPFLAGS_FOR_BUILD) $(AM_CFLAGS) $(CFLAGS)
+ LTCOMPILE = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
+ 	$(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) \
+ 	$(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) \
+@@ -161,9 +161,10 @@ am__v_CC_ = $(am__v_CC_@AM_DEFAULT_V@)
+ am__v_CC_0 = @echo "  CC      " $@;
+ am__v_CC_1 = 
+ CCLD = $(CC)
+-LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
+-	$(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \
+-	$(AM_LDFLAGS) $(LDFLAGS) -o $@
++@IF_CROSS@LINK = $(CC_FOR_BUILD) $(LDFLAGS_FOR_BUILD) -o $@
++@IF_NATIVE@LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
++@IF_NATIVE@	$(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \
++@IF_NATIVE@	$(AM_LDFLAGS) $(LDFLAGS) -o $@
+ AM_V_CCLD = $(am__v_CCLD_@AM_V@)
+ am__v_CCLD_ = $(am__v_CCLD_@AM_DEFAULT_V@)
+ am__v_CCLD_0 = @echo "  CCLD    " $@;
+@@ -256,6 +257,7 @@ AUTOHEADER = @AUTOHEADER@
+ AUTOMAKE = @AUTOMAKE@
+ AWK = @AWK@
+ CC = @CC@
++CC_FOR_BUILD = $(CC)
+ CCDEPMODE = @CCDEPMODE@
+ CDE_CONFIGURATION_TOP = @CDE_CONFIGURATION_TOP@
+ CDE_INSTALLATION_TOP = @CDE_INSTALLATION_TOP@
+@@ -263,6 +265,7 @@ CFLAGS = @CFLAGS@
+ CONFDIR = @CONFDIR@
+ CPP = @CPP@
+ CPPFLAGS = @CPPFLAGS@
++CPPFLAGS_FOR_BUILD = $(CPPFLAGS)
+ CURRENT = @CURRENT@
+ CYGPATH_W = @CYGPATH_W@
+ DEFS = @DEFS@
+@@ -289,12 +292,13 @@ INSTALL_SCRIPT = @INSTALL_SCRIPT@
+ INSTALL_STRIP_PROGRAM = @INSTALL_STRIP_PROGRAM@
+ LD = @LD@
+ LDFLAGS = @LDFLAGS@
++LDFLAGS_FOR_BUILD = $(LDFLAGS)
+ LEX = @LEX@
+ LEXLIB = @LEXLIB@
+ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@
+ LIBDIR = @LIBDIR@
+ LIBOBJS = @LIBOBJS@
+-LIBS = @LIBS@
++#LIBS = @LIBS@
+ LIBTOOL = @LIBTOOL@
+ LIB_XP = @LIB_XP@
+ LIPO = @LIPO@
+@@ -424,7 +428,8 @@ wmluiltok_LDADD = @LEXLIB@
+ wml_SOURCES = wml.c
+ wml_LDADD = -L. -lwml
+ wml_DEPENDENCIES = libwml.a
+-wmldbcreate_LDADD = ../../lib/Xm/libXm.la
++@IF_CROSS@wmldbcreate_LDADD = -lXm
++@IF_NATIVE@wmldbcreate_LDADD = ../../lib/Xm/libXm.la
+ INCLUDES = -I../../lib -I$(srcdir)/../../lib -I$(srcdir)/../../include ${X_CFLAGS} 
+ noinst_LIBRARIES = libwml.a
+ libwml_a_SOURCES = wmlparse.y wml.c wmloutkey.c wmlouth.c wmloutmm.c wmloutp1.c wmlresolve.c wmlsynbld.c wmlutils.c
+@@ -534,11 +539,11 @@ distclean-compile:
+ @am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $<
+ 
+ wmldbcreate.o: $(srcdir)/wmldbcreate.c
+-@am__fastdepCC_TRUE@	$(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT wmldbcreate.o -MD -MP -MF $(DEPDIR)/wmldbcreate.Tpo -c -o wmldbcreate.o `test -f '$(srcdir)/wmldbcreate.c' || echo '$(srcdir)/'`$(srcdir)/wmldbcreate.c
++@am__fastdepCC_TRUE@	$(AM_V_CC)$(CC_FOR_BUILD) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS_FOR_BUILD) $(AM_CFLAGS) $(CFLAGS) -MT wmldbcreate.o -MD -MP -MF $(DEPDIR)/wmldbcreate.Tpo -c -o wmldbcreate.o `test -f '$(srcdir)/wmldbcreate.c' || echo '$(srcdir)/'`$(srcdir)/wmldbcreate.c
+ @am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/wmldbcreate.Tpo $(DEPDIR)/wmldbcreate.Po
+ @AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='$(srcdir)/wmldbcreate.c' object='wmldbcreate.o' libtool=no @AMDEPBACKSLASH@
+ @AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+-@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o wmldbcreate.o `test -f '$(srcdir)/wmldbcreate.c' || echo '$(srcdir)/'`$(srcdir)/wmldbcreate.c
++@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(CC_FOR_BUILD) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS_FOR_BUILD) $(AM_CFLAGS) $(CFLAGS) -c -o wmldbcreate.o `test -f '$(srcdir)/wmldbcreate.c' || echo '$(srcdir)/'`$(srcdir)/wmldbcreate.c
+ 
+ wmldbcreate.obj: $(srcdir)/wmldbcreate.c
+ @am__fastdepCC_TRUE@	$(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT wmldbcreate.obj -MD -MP -MF $(DEPDIR)/wmldbcreate.Tpo -c -o wmldbcreate.obj `if test -f '$(srcdir)/wmldbcreate.c'; then $(CYGPATH_W) '$(srcdir)/wmldbcreate.c'; else $(CYGPATH_W) '$(srcdir)/$(srcdir)/wmldbcreate.c'; fi`