Re: Using pbulk with distcc

Just to keep you posted, I faced a "Cyclic dependency" even with the line added from "http://wiki.netbsd.org/tutorials/pkgsrc/cross_compile_distcc/" in my mk.conf. It seems the "if" condition is ignored/not working so I simply forced the execution of what's inside that condition and it seems to works :

# Skiped some lines...

PKGSRC_COMPILER=distcc ccache gcc
MAKE_JOBS=4
DISTCC_HOSTS=nbqc1-distcc1.neiluj.ca localhost
PKG_RCD_SCRIPTS=yes
ALLOW_VULNERABLE_PACKAGES=YES
PKG_DEFAULT_OPTIONS+=
KRB5_ACCEPTED=heimdal mit-krb5
USE_CWRAPPERS=yes
PKG_OPTIONS.irssi+= ssl perl inet6

DEPENDS_TARGET= bulk-install
BATCH= yes
BULK_PREREQ+= pkgtools/lintpkgsrc
BULK_PREREQ+= pkgtools/pkg_install
BULK_PREREQ+= devel/ccache
BULK_PREREQ+= devel/distcc
BULK_PREREQ+= devel/popt

# http://wiki.netbsd.org/tutorials/pkgsrc/cross_compile_distcc/

#.for DISTCCDEPS in devel/ccache sysutils/checkperms pkgtools/digest devel/distcc devel/popt devel/libtool-base lang/f2c devel/gmake
#. if "${PKGPATH}" == "${DISTCCDEPS}"
#IGNORE_DISTCC= yes
#IGNORE_CCACHE= yes
#. endif
#.endfor

# Force IGNORE here
IGNORE_DISTCC= yes
IGNORE_CCACHE= yes

By the way I keep trying to make pbulk use distcc for its build however I'm starting to think pbulk simply ignore the "PKGSRC_COMPILER=distcc ccache gcc" line of the mk.conf. Is that possible ? Is gcc hardcoded in pbulk ?

On Sun, Feb 4, 2018 at 10:10 PM, Julien Savard <juliensavard17%gmail.com@localhost> wrote:

Hi,
I installed pbulk simply by running "make install" in /usr/pkgsrc/pkgtools/pbulk. Then I ran "mksandbox --without-x /usr/sandbox" and finnaly " chroot $SANDBOX_ROOT_PATH/$SANDBOX_NAME /bin/sh -c "sh /usr/pkgsrc/mk/pbulk/pbulk.sh -n -c /root/mk.conf.frag" "

mk.conf.frag containing :

PKG_DEVELOPER=yes
MAKE_JOBS=1
SKIP_LICENSE_CHECK=yes
PKG_COMPILER=distcc ccache gcc
PKG_RCD_SCRIPTS=yes
ALLOW_VULNERABLE_PACKAGES=YES
PKG_DEFAULT_OPTIONS+=
KRB5_ACCEPTED=heimdal mit-krb5
USE_CWRAPPERS=yes
PKG_OPTIONS.irssi+= ssl perl inet6
PKGCHK_CONF?= /etc/pkgchk.conf
DEPENDS_TARGET= bulk-install
BATCH= yes
BULK_PREREQ+= pkgtools/lintpkgsrc
BULK_PREREQ+= pkgtools/pkg_install
BULK_PREREQ+= devel/ccache
BULK_PREREQ+= devel/popt
BULK_PREREQ+= devel/distcc
# http://wiki.netbsd.org/tutorials/pkgsrc/cross_compile_distcc/
.for DISTCCDEPS in devel/ccache sysutils/checkperms pkgtools/digest devel/distcc devel/popt devel/libtool-base lang/f2c devel/gmake
. if "${PKGPATH}" == "${DISTCCDEPS}"
IGNORE_DISTCC= yes
IGNORE_CCACHE= yes
. endif
.endfor
WRKOBJDIR= /tmp
PACKAGES= /srv/packages
DISTDIR= /srv/distfiles

Running "make install" in any package in /usr/pkgsrc from the sandbox will not work after running pbulk.sh :

=> Bootstrap dependency digest>=20010302: NOT found
=> Verifying bulk-install for ../../pkgtools/digest
make[1]: don't know how to make bulk-install. Stop

make[1]: stopped in /usr/pkgsrc/pkgtools/digest
*** Error code 2

Stop.

However creating a pacakge from the host and installing it inside the sandbox works :

$SANDBOX_ROOT_PATH/$SANDBOX_NAME/sandbox /chroot
cd /usr/pkgsrc/packages/All/
# this directory contiant the package compiled from the host bind monted inside the sandbox ( default behavior of mksandbox)
sparc1# pkg_add ccache-3.3.4.tgz
sparc1# pkg_add popt-1.16nb1.tgz
sparc1# pkg_add distcc-3.2.tgz

After adding the host in /root/.distcc/hosts I have working setup :

sparc1# cat main.c
#include <stdio.h>
int main() {
printf("Hello distcc!\n");
return 0;
}
sparc1# distcc -c main.c -o main.o
#... Skipped some lines
distcc[1718] compile from main.c to main.o
distcc[1718] (dcc_parse_hosts_file) load hosts from /root/.distcc/hosts
distcc[1718] (dcc_parse_hosts) found tcp token "nbqc1-distcc1.neiluj.ca"
#... Skipped some lines
distcc[1718] exec on nbqc1-distcc1.neiluj.ca: cc -c main.c -o main.o
distcc[1718] (dcc_note_state) note state 2, file "main.c", host "nbqc1-distcc1.neiluj.ca"
distcc[1718] (dcc_connect_by_name) connecting to nbqc1-distcc1.neiluj.ca port 3632
#... Skipped some lines
distcc[1718] compile main.c on nbqc1-distcc1.neiluj.ca completed ok
distcc[1718] elapsed compilation time 0.000001s
sparc1# gcc main.o -o hello-distcc
sparc1# ./hello-distcc
Hello distcc!

This will install all I need in /usr/pkg/ inside the sandbox. However it will be deleted as soon as I run "/usr/pbulk/bin/bulkbuild" ( pbulk.conf:prefix=/usr/pkg --> "rm -fr $prefix" on each bulkbuild ???).

Installing distcc inside the sandbox in another directory then the default directory or prefix if you prefer ( /usr/pbulk/ instead of/usr/pkg) seemed logical to me and this is what I though you told me to do. Guess I misunderstood...

So the question remain : How do you make pbulk use distcc ?

On Fri, Feb 2, 2018 at 8:39 AM, Thomas Klausner <wiz%netbsd.org@localhost> wrote:
On Fri, Feb 02, 2018 at 08:28:49AM -0500, Julien Savard wrote:
> So I installed distcc with the pbulk prefix. In other term :
>
> /usr/sandbox/sandbox chroot
> cd /usr/pkgsrc/packages/All
> pkg_add -p /usr/pbulk/ ccache-3.3.4.tgz
> pkg_add -p /usr/pbulk/ popt-1.16nb1.tgz
> pkg_add -p /usr/pbulk/ distcc-3.2.tgz

This is not recommended. Just build the packages for that prefix, like you did for pbulk (I hope)...
Thomas