Subject: clustering builds
To: None <fair@clock.org, port-m68k@netbsd.org>
From: Tim Rightnour <root@garbled.net>
List: port-m68k
Date: 01/11/1999 10:05:18
I dont subscribe to this list.. but I sometimes browse the archives looking for
hosed messages.. and ran across this:

# Those of us who still use the various m68k machines are among the
# slowest processors that NetBSD runs on; a full build from nothing
# takes 3 days on a Sun 3/60 workstation (20 MHz mc68020).
#
# I was wondering if anyone here has been using multiple systems to
# build NetBSD in parallel, and thus cut down on build times?
#
#        Erik E. Fair    fair@clock.org

Yes.. I have a room full of i386's which I use to do paralell builds.

I currently have two very simple ways to do this:

#1 glunix.   Ok.. not very simple.. but there is a package that can be made to
work with very minimal effort.  It includes a program called "glumake" which is
a glunixified gnu make.  It is able to do builds in paralell across machines
with the -j flag.  I also have patches somewhere for our make to glunixify it..
 It's pretty much 5 lines, and you could easily look at what glunix does and
pretty much cut and paste it.

#2 cluster-it.  http://www.inficad.com/~garbled/clusterit.html
(there is a pkg for this as well)

This is a collection of clustering commands like dsh (paralell execution), seq
(sequential execution), run (random execution) and barrier (barrier sync for
shell scripting).  For paralell build's, I have used seq, by doing something
like:

CC= seq 'cd ${.CURDIR}; cc'

And it works nicely.. (you need to do some fiddling with the definition of make
as well, to get it to cd properly, but it is possible.)  Basicly you set up seq
like that, and run make -j #ofmachines, and it will hammer them all at once.

---
Tim Rightnour  -  root@garbled.net
Free Multi-Platform Operating System: http://www.netbsd.org
NetBSD Mailing lists on the web: http://mail-index.netbsd.org/mlist