Subject: Re: pkgsrc on SMP machines
To: Hubert Feyrer <feyrer@cs.stevens.edu>
From: None <segv@netctl.net>
List: tech-pkg
Date: 12/15/2005 04:20:07
On Thu, 15 Dec 2005 04:39:51 +0100 (CET)
Hubert Feyrer <feyrer@cs.stevens.edu> wrote:
> On Thu, 15 Dec 2005 segv@netctl.net wrote:
> > Hi, can anyone suggest a good strategy for utilising the most of SMP
> > hardware when building software from pkgsrc??
> ...
> > Fixing all Makefiles to support -j > 1 seems quite unreasonable, I guess the
> > best solution would be to have a multithreaded compiler, which distributes
> > workload to a specified number of threads, i.e.:
>
> How about compiling the pkgs themselves (i.e. whatever gets extracted from
> sources) with 'make -j', e.g. as the patch below does. At least that's how
> Unix is supposed to work...
>
> Of course that's a crude hack, and a real solution would involving a
> user-settable variable plus a per-package maximum, and the real value
> would be the somewhere in between there.
>
> (There is that other idea to build pkgsrc packages in parallel which is
> stuck to some people's head, but it grossly violates the KISS principle
> and involves all sort of nasty things like locking and whatnot; I still
> suggest not going there)
>
>
> - Hubert
>
>
> Index: bsd.pkg.mk
> ===================================================================
> RCS file: /cvsroot/pkgsrc/mk/bsd.pkg.mk,v
> retrieving revision 1.1774
> diff -u -r1.1774 bsd.pkg.mk
> --- bsd.pkg.mk 5 Dec 2005 22:07:07 -0000 1.1774
> +++ bsd.pkg.mk 15 Dec 2005 03:34:35 -0000
> @@ -1895,7 +1895,7 @@
> . for _dir_ in ${BUILD_DIRS}
> ${_PKG_SILENT}${_PKG_DEBUG}${_ULIMIT_CMD} \
> cd ${WRKSRC}; cd ${_dir_}; \
> - ${SETENV} ${MAKE_ENV} ${MAKE_PROGRAM} ${BUILD_MAKE_FLAGS} \
> + ${SETENV} ${MAKE_ENV} ${MAKE_PROGRAM} -j17
> ${BUILD_MAKE_FLAGS} \ -f ${MAKEFILE} ${BUILD_TARGET}
> . endfor
> .endif
Hmm...this does not seems to work (I put -j16), it's not using all 4 CPUs. Any
ideas why?