Re: Support for build.sh with a cluster

[Curt Sampson <cjs%cynic.net@localhost>]

On Sat, 26 Jun 2004, Matthew Mondor wrote:

> Does anyone know of any current implementation of toolchain changes
> which would allow to distribute compilation among a cluster of
> networked machines?

I don't know how much this helps, but there is already some support in
make for executing jobs remotely when using parallel execution with
the "-j" flag. It would still need some work actually to be usable for
practical purposes, last time I looked, though.

As well, this seems a bit of a painful route to go, since you'd probably
have to use multiple methods of distribution of the various components.
(Some data would be transferred through rsh or ssh, some with a
distributed filesystem, and so on.)

It would be much nicer just to build some sort of remote execution
facility that could operate through a single link. In my ideal world,
you'd send a set of requests and resources to the remote machine, which
would essentially execute the request in a sandbox, with the remote
requesting further resources as it needs them. For example, the initial
data sent to the remote might contain a request to compile a file for
a certain architecture and the file itself. The remote would start the
compilaction and, as it found things like header files that it needed,
send requests for them back to the requester.

cjs
-- 
Curt Sampson  <cjs%cynic.net@localhost>   +81 90 7737 2974   
http://www.NetBSD.org
    Don't you know, in this new Dark Age, we're all light.  --XTC