Subject: Re: distcc and pkgsrc
To: wrzymski <wrzymski@gmail.com>
From: Gary Thorpe <gathorpe79@yahoo.com>
List: netbsd-help
Date: 02/04/2006 22:24:23
--- wrzymski <wrzymski@gmail.com> wrote:
> 2006/2/4, Jukka Salmi <j+nbsd@2006.salmi.ch>:
> > wrzymski --> netbsd-help (2006-02-04 00:01:03 +0100):
> > > 2006/2/3, Jarmo Jaakkola <jarmo.jaakkola@tut.fi>:
> > [...]
> > That's not true. If superuser privileges are required, ${SU_CMD} is
> > executed (which is ${ROOT_CMD} by default, which is `/usr/bin/su -
> > root -c' for most systems).
>
> Yes you are right.
>
> > > With such configuration you can't expect almost any boost with
> typical
> > > c programs.
> >
> > Why not?
>
> My experience with various pkgsrc programs tells me that high values
> of MAKEFLAGS=-j often ends with failed compilations.
Because pkgsrc is composed of third-party software, there is no
guarantee that the complete and correct dependency information is
present in a given software's makefile. Without this, "-jN" is not
reliable. There is the same limitation in FreeBSD's ports system.
So basically, you cannot parallelize it (reliably) and hence it cannot
be distributed effectively, regardless of how distcc works.
> Distcc isn't
> magic wand it "shares" only preprocessed code and sending it to
> slower
> machines shouldn't give you any noticeable and worth effort kick.
Actually, even slower machines can decrease the total time needed but
matching machines with large speed differences may be counterproductive
because of extra communication overhead.
> Of
> course distcc can be useful. For example I use it to cross compile
> packages at my Jornada (hpcsh) port. It has only 32mb memory and
> 133Mhz sh3 CPU. I mount pkgsrc tree and swap over NFS for
> compilation.
> With typical C programs I guess it's 20-40% faster than native
> compilation. The real kick is for low memory machines like Jornada
> with swap over NFS cross compiling C++ files means 10x faster or even
> more :) Without distcc I couldn't compile negatively kismet for
> Jornada, swap file was growing 4x memory and after couple of hours
> ending with panic. Distcc can do the trick in about 1h :)
>
> To be 100% sure you should compare timings by yourself and give us
feedback
__________________________________________________________
Find your next car at http://autos.yahoo.ca