tech-pkg archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

A new approach to package building




LPJS (https://github.com/outpaddling/LPJS/) is nearing the initial
release, and while it's primarily aimed at scientific computing, I've
had it in the back of my mind to use it for package building as well.
Using a job scheduler / resource manager this way has the potential to
improve hardware utilization in both single server and multiple server
package builds, thereby speeding up bulk builds.

Two things will be needed in order to feed builds into LPJS:

1. A correct build order.  I'm wondering if anyone has developed a tool
that can "sort" the pkgsrc tree, producing a linear list of
category/package entries where dependencies always precede their dependents.

2. Resource specs (processors and memory requirements) for each build.
This can be determined empirically, manually the first time, and updated
automatically once LPJS is able to track resource use (high on the to-do
list for a future release).

With that information, it will be easy to generate a batch script for
each package build and feed it into the queue.

Like pkgsrc, and unlike all other job schedulers, LPJS has portability
to all POSIX platforms as a core design principle.  So it could aid bulk
builds on any platform supported by pkgsrc.

--
Life is a game.  Play hard.  Play fair.  Have fun.


Home | Main Index | Thread Index | Old Index