Subject: Re: FreeBSD's package cluster
To: None <tech-pkg@NetBSD.org>
From: Klaus Heinz <k.heinz.mai.fuenf@onlinehome.de>
List: tech-pkg
Date: 05/28/2005 00:33:03
John Klos wrote:

> What'd be neat to see is a set of tools that could have one (fast) machine 
> preprocess, as much as possible, some or all of the packages, then have 
> the slower, real processors do the compiling. ClusterIt seems like it'd be 
> an ideal way to use one machine to coordinate all of them. It'd also be 

I think for a first shot we should concentrate on a simple model.
Multiple machines building parts of the same package, a fast central
machine preprocessing and then delegating compiler work seems rather
complicated.

> neat to have a way to have multiple machines of the same arch work on 
> different, non-interdependent packages.

This sounds more feasible to me. A single machine "checks out" a build
task from the servers TODO list and builds a single package at a time.
Prerequisite packages are downloaded (cached) from the central package
server. When the package is ready, upload it to the server so it can be
marked "completed". If I remember correctly the research I did some
months ago, this is roughly how Debian does it (see
http://www.debian.org/devel/buildd/).

> Has anyone worked on anything like this? Would there be interest?

I think Johnny Lam hinted at some work/design called "bobac" (?) he
already had.

ciao
     Klaus