Subject: Re: Fastest m68k box for bulk package builds
To: David Brownlee <abs@netbsd.org>
From: John Klos <john@sixgirls.org>
List: port-m68k
Date: 09/24/2001 12:52:25
> > Perhaps a status page would be in order so that people can get an small
> > occasional email telling them that it's been updated, so they can go
> > look at the status page and see if what they're looking for has been
> > built.
> >
> Sounds reasonable - it should probably be coordinated across
> architectures so you could go to the same place for m68k, mips,
> etc.
> Maybe just something that generates a web page per category with
> a table of package-name vs arch+osrev, with an indicator of built,
> failed, not-for-this-arch, etc?
Yes, definitely.
An issue, though: if one person / machine is doing a bulk build, then this
wouldn't be difficult, as the bulk-build scripts generate an index. But
what about multiple people / machines? And merges / updates?
Would this be most appropriate for the bulk scripts (that is, adding
something to output ewasily mergeable information), or for some code on
the server?
> > Yes; I've done some experimenting with two Quadras, that Amiga mentioned
> > above, and NFS to see what kind of interaction the machines would have.
> > I've also been talking with various people about implementing a database
> > controlled build server that would track pkgsrc and all of the binaries
> > for a particular architecture, so building can be distributed more loosely
> > (ie, over the Internet using ssh where NFS isn't possible).
> >
> That sounds very interesting, though pkgsrc version skews might
> start to become an issue.
I've been keeping the pkgsrc tree, distfiles, and finished binary tree on
one disk and sharing via NFS. Compile temp directories are local.
Is there a better way to do this? One thing that would be nice in the bulk
package scripts would be if, when entering a directory, if a test can be
done to see if another task is (trying to) build(ing) that package (and
hasn't failed or succeeded yet), and if so, skip it.
A few times when I was sharing via NFS, more than one computer would end
up in the same package directory (usually because of a dependency), and
things would get screwed up.
If "package locking" existed, then I think building with multiple machines
would be much closer to automatic.
> > I can use the already set up nodes of the Quadra cluster to start a bulk
> > build on 1.4.3, if this would help.
> >
> More binary packages for releases are always a good thing :)
I didn't realise that lots of people still run 1.4. But then again, I
suppose lots of non-NetBSD people don't realise that we run VAX and m68k
machines!
> > I'll get started on that. For now, the Quadras will do 1.4, and the Amiga
> > will do 1.5.
>
> Great - thanks!
Status as it comes...
Thanks,
John