Subject: Re: pkgsrc and distcc
To: Jukka Salmi <j+nbsd@2005.salmi.ch>
From: Lars Nordlund <lars.nordlund@hem.utfors.se>
List: tech-pkg
Date: 05/25/2005 23:26:02
Wed 2005-05-25 klockan 21:47 +0200 skrev Jukka Salmi:
> Hmm, just tried this, but if I use MAKE_FLAGS+=-j6 everything is compiled
> on localhost (with MAKEFLAGS all three hosts took part in compiling).
> Furthermore, building print/xpdf now fails with:
> 
> ===> build-message [xpdf-3.00pl3] ===> Building for xpdf-3.00pl3
> --- all ---
> cd goo; /usr/bin/make
> c++ -O2 -I/usr/pkg/include -I/usr/pkg/include/freetype2 -I/usr/X11R6/include -DHAVE_CONFIG_H -I.. -I. -c GHash.cc
> [...]
> ranlib libGoo.a
> cd fofi; /usr/bin/make
> cd: can't cd to fofi
> cd splash; /usr/bin/make
> cd: can't cd to splash
> cd xpdf; /usr/bin/make all
> cd: can't cd to xpdf
> => Unwrapping files-to-be-installed.
> 
> and I can't find the source of the problem...

It is the buildlink wrappers. While the build is progressing files in
work/.wrapper/tmp are updated with new values. The values are found
during the build and can not be "pre-calculated" (I think). Putting a
shell lock around reads/writes of these values will make the build
basically single-cpu bound (or introduce interesting merge problems..).

The only "SMP solution" as I see it is to build different packages (in
the hopefully large dependancy chain) at the same time, and not trying
to build a single package with the -j flag.

Dropping the buildlink framework in pkgsrc is a bad move. It gives a
nice extra level of checks to see that the package being built only uses
its dependancies and not stuff randomly found around the system..

Perhaps extending buildlink a bit so it supports chroot() builds instead
of the current symlink method. This side-steps the problem with having
to wrap the compilers and tools with scripts and sed/awk:ing the
arguments to them. The drawback is that pkgsrc builds will have to be
done as root, but that might be an ok tradeoff. Other drawbacks are that
chroot() might not be available on all pkgsrc platforms.


Well, I might have misunderstood how things work, and hopefully other
people will correct me.


Best regards
	Lars Nordlund