Subject: Re: Reducing bsd.pkg.mk recursion
To: None <tech-pkg@netbsd.org>
From: Juan RP <juan@xtraeme.nopcode.org>
List: tech-pkg
Date: 11/12/2004 09:26:50
--Signature=_Fri__12_Nov_2004_09_26_50_+0100_+LE/ZSXOcB8LRgF7
Content-Type: text/plain; charset=US-ASCII
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
On Thu, 11 Nov 2004 22:48:13 -0500 (EST)
Todd Vierling <tv@duh.org> wrote:
> There's a lot of it. A whopping caboodle of it. The problem is, it's
> rather expensive to call ${MAKE} recursively (in some cases, a couple levels
> deep) within a package when there's really no need to do so.
>
> I have some changes in work which already improve bsd.pkg.mk build time
> appreciably, especially on platforms where fork() is not cheap. It's not
> just fork(), though; it takes nonzero time for make to go reparse and
> recalculate everything it just finished calculating in the package Makefile
> the last time through.
>
> I'll post the diffs for review soonish when they are reliable. I'd like to
> point out the specific points I'm addressing before diving fully into this,
> in case I'm missing something here. Please comment freely.
>
> * PKG_PHASE: This whole concept forces a layer of ${MAKE} recursion. The
> only things that use this, AFAICT, are bits in the wrapper, bl3, and
> reduce-depends code to avoid calculating things when the user hasn't
> requested a build mode requiring them.
>
> I believe these should all be possible to handle through proper use of
> ".if make()" conditionals rather than recursing just to set PKG_PHASE.
> The "real-*" targets could then be collapsed right onto their respective
> cookie timestamp targets.
>
> To accomplish this, I plan to add bits to bsd.pkg.mk to detect the
> "maximum phase" of the package build requested by the user, so that
> anything previously needing PKG_PHASE can simply check if a certain phase
> *would* be executed in the current make instance. This should provide
> equivalent functionality all within a single make invocation.
>
> * Many following targets are recursed, but could be collapsed into a
> sequence of subtargets for a certain build phase. In some cases these
> would involve slight shellcode modifications (and some of those are
> actually simplifications). In other cases it may involve splitting a do-*
> target, which is currently recursing, into several separate targets
> instead.
Yay, that sounds great! because pkgsrc is "slow" right now... IMHO :)
--Signature=_Fri__12_Nov_2004_09_26_50_+0100_+LE/ZSXOcB8LRgF7
Content-Type: application/pgp-signature
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (NetBSD)
iD8DBQFBlHPKypkLYVDran0RAkznAKCDZNp/5mYEsWpu/aLM4uv87zYaDwCgut5w
vawheIkBD8wU9d81XQBTqNs=
=mejO
-----END PGP SIGNATURE-----
--Signature=_Fri__12_Nov_2004_09_26_50_+0100_+LE/ZSXOcB8LRgF7--