Subject: Re: Multiprocessor builds anyone?
To: Chuck McManis <firstname.lastname@example.org>
From: Andrew Gillham <email@example.com>
Date: 08/05/2000 21:13:51
Chuck McManis writes:
> Hello everyone,
> This may be an impractical idea, but it seems doable.
> Building a release on the VAX is really slow. A typical build on my fastest
> machine takes several days to do from scratch. I have 4 VAXStation VLCs
> (little 6 VUP puppies) that are not doing anything (they test when I'm done
> building but don't help out in the build.) What I was wondering was if
> anyone had looked as non-overlapping multiprocessor builds. For example,
> building through the libraries, and then starting one machine on usr.bin,
> one on usr.sbin, one on gnu, one on share, and having them all build in
Have you looked at 'pkgsrc/parallel/clusterit' at all? It allows you to
do something like this using the normal build process. If you create a
small shell script, I call it 'dcc', to invoke clusterit, you can end up
doing something like this:
# cd /usr/src
# make CC='dcc' -j 4 build
This will end up distributing load out across your "cluster" by spawning
jobs off to run parallel whenever possible. Even though make thinks they
are all running locally it will end up working ok as long as each node
mounts the exact same /usr/src tree.
What I have been using is a single "master" NFS server that exports its
/usr to each node. This assures that each node has the same toolchain,
utilities, and source tree.