Subject: Re: Fastest m68k box for bulk package builds
To: David Brownlee <abs@netbsd.org>
From: John Klos <john@sixgirls.org>
List: port-m68k
Date: 09/24/2001 12:52:25
> > Perhaps a status page would be in order so that people can get an small
> > occasional email telling them that it's been updated, so they can go
> > look at the status page and see if what they're looking for has been
> > built.
> >
> 	Sounds reasonable - it should probably be coordinated across
> 	architectures so you could go to the same place for m68k, mips,
> 	etc.

> 	Maybe just something that generates a web page per category with
> 	a table of package-name vs arch+osrev, with an indicator of built,
> 	failed, not-for-this-arch, etc?

Yes, definitely.

An issue, though: if one person / machine is doing a bulk build, then this
wouldn't be difficult, as the bulk-build scripts generate an index. But
what about multiple people / machines? And merges / updates?

Would this be most appropriate for the bulk scripts (that is, adding
something to output ewasily mergeable information), or for some code on
the server?

> > Yes; I've done some experimenting with two Quadras, that Amiga mentioned
> > above, and NFS to see what kind of interaction the machines would have.
> > I've also been talking with various people about implementing a database
> > controlled build server that would track pkgsrc and all of the binaries
> > for a particular architecture, so building can be distributed more loosely
> > (ie, over the Internet using ssh where NFS isn't possible).
> >
> 	That sounds very interesting, though pkgsrc version skews might
> 	start to become an issue.

I've been keeping the pkgsrc tree, distfiles, and finished binary tree on
one disk and sharing via NFS. Compile temp directories are local.

Is there a better way to do this? One thing that would be nice in the bulk
package scripts would be if, when entering a directory, if a test can be
done to see if another task is (trying to) build(ing) that package (and
hasn't failed or succeeded yet), and if so, skip it.

A few times when I was sharing via NFS, more than one computer would end
up in the same package directory (usually because of a dependency), and
things would get screwed up.

If "package locking" existed, then I think building with multiple machines
would be much closer to automatic.

> > I can use the already set up nodes of the Quadra cluster to start a bulk
> > build on 1.4.3, if this would help.
> >
> 	More binary packages for releases are always a good thing :)

I didn't realise that lots of people still run 1.4. But then again, I
suppose lots of non-NetBSD people don't realise that we run VAX and m68k
machines!

> > I'll get started on that. For now, the Quadras will do 1.4, and the Amiga
> > will do 1.5.
>
> 	Great - thanks!

Status as it comes...

Thanks,
John