Subject: Re: Fastest m68k box for bulk package builds
To: John Klos <john@sixgirls.org>
From: David Brownlee <abs@netbsd.org>
List: port-m68k
Date: 09/26/2001 12:39:22
On Mon, 24 Sep 2001, John Klos wrote:

> > > Perhaps a status page would be in order so that people can get an small
> > > occasional email telling them that it's been updated, so they can go
> > > look at the status page and see if what they're looking for has been
> > > built.
> > >
> > 	Sounds reasonable - it should probably be coordinated across
> > 	architectures so you could go to the same place for m68k, mips,
> > 	etc.
>
> > 	Maybe just something that generates a web page per category with
> > 	a table of package-name vs arch+osrev, with an indicator of built,
> > 	failed, not-for-this-arch, etc?
>
> Yes, definitely.
>
> An issue, though: if one person / machine is doing a bulk build, then this
> wouldn't be difficult, as the bulk-build scripts generate an index. But
> what about multiple people / machines? And merges / updates?
>
	The table should probably be built by a cronjob on ftp.netbsd.org,
	based on what packages have been uploaded.

> Would this be most appropriate for the bulk scripts (that is, adding
> something to output ewasily mergeable information), or for some code on
> the server?
>
	It might make sense to add it to pkgsrc, then have the cron job on
	ftp.netbsd.org run it - if you keep it with the bulk-packages then
	people running independent builds can make more use of it.

> > 	That sounds very interesting, though pkgsrc version skews might
> > 	start to become an issue.
>
> I've been keeping the pkgsrc tree, distfiles, and finished binary tree on
> one disk and sharing via NFS. Compile temp directories are local.
>
	Seems reasonable.

> Is there a better way to do this? One thing that would be nice in the bulk
> package scripts would be if, when entering a directory, if a test can be
> done to see if another task is (trying to) build(ing) that package (and
> hasn't failed or succeeded yet), and if so, skip it.
>
> A few times when I was sharing via NFS, more than one computer would end
> up in the same package directory (usually because of a dependency), and
> things would get screwed up.
>
> If "package locking" existed, then I think building with multiple machines
> would be much closer to automatic.
>
	I think package locking is useful in more than just bulk-build
	situations, both in building, and in installation/removal.
	It probably makes sense to keep a lockfile in the work directory,
	and maybe try to adjust the clean logic so that everything except
	the work directory itself and the lockfile would be deleted.

> > > I can use the already set up nodes of the Quadra cluster to start a bulk
> > > build on 1.4.3, if this would help.
> > >
> > 	More binary packages for releases are always a good thing :)
>
> I didn't realise that lots of people still run 1.4. But then again, I
> suppose lots of non-NetBSD people don't realise that we run VAX and m68k
> machines!
>
	:) - 1.5 is definitely more important, but there are still quite
	a few people whe haven't moved up (I still have one machine on 1.3
	for various reasons :)


-- 
		David/absolute		-- www.netbsd.org: No hype required --