RFC: distbb for cross-compiling packages

To: tech-pkg%NetBSD.org@localhost
Subject: RFC: distbb for cross-compiling packages
From: Aleksey Cheusov <cheusov%tut.by@localhost>
Date: Fri, 18 Nov 2011 15:01:53 +0300

Hello. I've implemented support for cross-compiling packages in
distbb. I have not released "stable" version yet, but the code is
available in wip/nih-current. I spent three days to get this work,
so I am in the very beginning.

Notes.

1) Cross-compiling in distbb is a pure "cross-compiling" and uses all
cross-tools, not just cross-compiler with native ld in combination
with distcc or something similar. Documentation for pkgsrc part is
in doc/HOWTO-crosscompile-xorg.

2) The goal is to compile build host tools automatically as soon as
they are needed. For this to work I assume that BUILD_DEPENDS is both
target host and build host dependency. For example, bison/flex needs
to be compiled for build host (we run them) and target platform too
(we need liby.a and libfl.a). In general this assumption is correct
but often generates huge amount of unnecessary dependencies. Very
often BUILD_DEPENDS in pkgsrc means either build host deps only
(coreutils and p5-Unicode-Map8 in textproc/dict-mueller7 or autoconf
for regenerating configure script) or target platform
dependency only. It'd be better if we had mark build/target platform
dependencies explicitly with a help of additional variable[s]. If
this happens I'll easily adapt distbb to use them.

3) I use BOOTSTRAP_DEPENDS variable just like BUILD_DEPENDS. Packages
listed there are installed before building the package.

How to configure distbb.

1) Configure normal distbb setup as it is documented in README.

2) build.sh -m ...

3) Create mk.xxx.conf for target platform and copy it to all slaves where
packages will be built. Configure and test manual cross builds on all
slaves. Example of a target mk.xxx.conf is in attachment.

4) Additional variables in distbb.conf
CROSS_COMPILE # set to non-empty string for cross-compiling
TARGET_PACKAGES # path to packages directory for target platform
TARGET_MAKECONF # path to target platform mk.conf

There are others but they are optional if you are happy with the
defaults. See distbb.default.conf for details.

5) A lot of ac_cv_xxx assignments in CONFIGURE_ENV in target platform
mk.conf either globally or on per-package basis is needed. Then
patching and testing...

How it works.
The only difference from normal bulk builds is that artificial
packages prefixed with BH-- (Build Host) are introduced. Then, a
dependency graph containing both target platform packages and build
platform packages is generated. BH-- packages are built using
standard mk.conf. Target packages are built with $TARGET_MAKECONF. Of
course target packages and pkg_summary(5) are uploaded instead of
build host packages.

Test runs
As a test I built some packages for NetBSD-5.99.56/sparc64 on
NetBSD-5.99.56/amd64.

http://mova.org/~cheusov/pub/pkgsrc-distbb/NetBSD-sparc64/current/logs/cross-test1/META/report.html

Here I started cross bulk build from scratch with empty $PACKAGES/All
and $TARGET_PACKAGES/All. I tried to build xterm, some of my packages
and some others. textproc/xmlcatmgr, lang/perl5 and
*pkgtools/pkg_install* failed classically. They run programs built
for target platform. See "really built" and "Progress messages"
links. BH-- there means "build host package".

http://mova.org/~cheusov/pub/pkgsrc-distbb/NetBSD-sparc64/current/logs/cross-test2/META/report.html

Here I tried to build sysutils/mc

http://mova.org/~cheusov/pub/pkgsrc-distbb/NetBSD-sparc64/current/logs/cross-test3/META/report.html

More packages from
http://mail-index.netbsd.org/tech-pkg/2011/01/06/msg006516.html
Very interesting results.

Examples for config files -- see attachments.

Questions.

Target mk.conf

- Without _GCC_VERSION cross-compiling didn't work. I did not inspect
where gcc-2.95 dependency comes from.

- TOOLS_PLATFORM.digest defaults to $CROSS_DESTDIR/... This is wrong
and needs to fixed.

- I'm not sure how to configure pkg-config properly, so I set
PKG_CONFIG_SYSROOT_DIR variable and exported it in mk.conf.
This probably should be done in pkgsrc.

- I think we need explicit built/target host build-time
dependencies. See above.

Comments?

#-*-mode:  sh -*-

renice 19 $$ 2>/dev/null 1>>/dev/null

#CLEAN_TARGET=None

PKGSRC_PREFIX=/usr/pkg
BMAKE=/usr/bin/make
PKGSRCDIR=/srv/pkgsrc
RMDIRS_CMD="/bin/rm -rf $PKGSRC_PREFIX/* /var/db/pkg"
BOOTSTRAP="/srv/pkgsrc_bin/NetBSD/bootstrap.tar"
EXTRACT_BOOTSTRAP_CMD="/home/cheusov/pkg_distbb/bin/sudo /bin/tar -xf 
$BOOTSTRAP -C /"
MASTER_MODE=yes
SLAVES='5-4'
REPORTS_DIR=$HOME/bulk-logs
PACKAGES=/srv/pkgsrc_bin/NetBSD-cross
RESPONSIBLE='Aleksey Cheusov'

UNPRIV_USER=cheusov

PKGSRCVER='current'

#UPLOAD_PKGS_PROG=:
#UPLOAD_LOGS_PROG=:
UPLOAD_PKGS_FILES="/home/cheusov/pkg_distbb2/libexec/distbb/upload_pkgs_no_bin_on_ftp"

USER_MAIL=cheusov

UPLOAD_PKGS_DEST=cheusov%mova.org@localhost:~/public_html/pub/pkgsrc-distbb/NetBSD-sparc64/current/packages/
UPLOAD_LOGS_DEST=cheusov%mova.org@localhost:~/public_html/pub/pkgsrc-distbb/NetBSD-sparc64/current/logs/

     
PACKAGES_URL="http://mova.org/~cheusov/pub/pkgsrc-distbb/NetBSD-sparc64/current/packages/";
REPORT_URL_PREFIX="http://mova.org/~cheusov/pub/pkgsrc-distbb/NetBSD-sparc64/current/logs/";

PAEXEC_EXTRA_OPTS='-Z300 -W1'

######################################################################
I_AM_READY=4

TRANSPORT='/home/cheusov/pkg_distbb/etc/distbb_chroot /srv/sandbox'

CROSS_COMPILE=1
TARGET_PACKAGES=/srv/pkgsrc_bin/NetBSD-cross-sparc64
TARGET_MAKECONF=/etc/mk.NetBSD-5.99.56-sparc64.conf

. /home/cheusov/pkg_distbb2/share/distbb/distbb.default.conf

PKG_SUMMARY="$PACKAGES/All/pkg_summary.txt"

# Example /usr/pkg/etc/mk.conf file produced by bootstrap-pkgsrc
# Wed Nov 16 01:41:52 FET 2011

.ifdef BSD_PKG_MK       # begin pkgsrc settings

OPSYS=NetBSD
OS_VERSION=5.99.56
# NATIVE_MACHINE_GNU_PLATFORM=sparc64--netbsd
# MACHINE_GNU_PLATFORM=sparc64--netbsd

PKG_DBDIR=              /var/db/pkg
LOCALBASE=              /usr/pkg
VARBASE=                /var
PKG_TOOLS_BIN=          /usr/sbin
PKGMANDIR=              man

TARGET_ARCH=            sparc64
USE_CROSS_COMPILE=      yes
SYSROOT=                /srv/obj-sparc64/destdir.sparc64
TOOLDIR=                /srv/obj-sparc64/tooldir.NetBSD-5.99.56-amd64
CPPFLAGS+=              --sysroot=${SYSROOT}
LDFLAGS+=               --sysroot=${SYSROOT}
CROSS_DESTDIR=          ${SYSROOT}

PKG_CONFIG_SYSROOT_DIR= ${SYSROOT}
.export PKG_CONFIG_SYSROOT_DIR

WRKOBJDIR=              /tmp/obj-pkgsrc-cross
PACKAGES=               /srv/pkgsrc_bin/NetBSD-cross-sparc64
DISTDIR=                /srv/pkgsrc_distfiles
PKGSRCDIR=              /srv/pkgsrc

X11_TYPE=               modular

TOOLS_PLATFORM.digest=  digest

_GCC_VERSION=           4.5.3

PKG_DEFAULT_OPTIONS=    -pulseaudio -arts

PKG_RESUME_TRANSFER=    yes

ALLOW_VULNERABLE_PACKAGES=      yes

SKIP_LICENSE_CHECK=     yes

PKG_DEFAULT_OPTIONS=    -arts -pulseaudio -aalib -nas

ACCEPTABLE_LICENSES+=   opera-850-license

.endif                  # end pkgsrc settings

.PHONY: list-vars
list-vars:
.for i in ${VARS}
        @printf '%s=%s\n' "${i}" ${${i}:Q}
.endfor

.sinclude "/home/cheusov/prjs/makefile-cvsdist/Makefile.cvsdist"

-- 
Best regards, Aleksey Cheusov.

Follow-Ups:
- Re: RFC: distbb for cross-compiling packages
  - From: Greg Troxel

Prev by Date: Re: OWN_DIRS doesn't protect directories as advertised
Next by Date: Re: RFC: distbb for cross-compiling packages
Previous by Thread: daily pkgsrc CVS update output
Next by Thread: Re: RFC: distbb for cross-compiling packages
Indexes:

Home | Main Index | Thread Index | Old Index