Subject: Re: Reducing bsd.pkg.mk recursion
To: None <tech-pkg@netbsd.org>
From: Juan RP <juan@xtraeme.nopcode.org>
List: tech-pkg
Date: 11/12/2004 09:26:50
--Signature=_Fri__12_Nov_2004_09_26_50_+0100_+LE/ZSXOcB8LRgF7
Content-Type: text/plain; charset=US-ASCII
Content-Disposition: inline
Content-Transfer-Encoding: 7bit

On Thu, 11 Nov 2004 22:48:13 -0500 (EST)
Todd Vierling <tv@duh.org> wrote:

> There's a lot of it.  A whopping caboodle of it.  The problem is, it's
> rather expensive to call ${MAKE} recursively (in some cases, a couple levels
> deep) within a package when there's really no need to do so.
> 
> I have some changes in work which already improve bsd.pkg.mk build time
> appreciably, especially on platforms where fork() is not cheap.  It's not
> just fork(), though; it takes nonzero time for make to go reparse and
> recalculate everything it just finished calculating in the package Makefile
> the last time through.
> 
> I'll post the diffs for review soonish when they are reliable.  I'd like to
> point out the specific points I'm addressing before diving fully into this,
> in case I'm missing something here.  Please comment freely.
> 
> * PKG_PHASE: This whole concept forces a layer of ${MAKE} recursion.  The
>   only things that use this, AFAICT, are bits in the wrapper, bl3, and
>   reduce-depends code to avoid calculating things when the user hasn't
>   requested a build mode requiring them.
> 
>   I believe these should all be possible to handle through proper use of
>   ".if make()" conditionals rather than recursing just to set PKG_PHASE.
>   The "real-*" targets could then be collapsed right onto their respective
>   cookie timestamp targets.
> 
>   To accomplish this, I plan to add bits to bsd.pkg.mk to detect the
>   "maximum phase" of the package build requested by the user, so that
>   anything previously needing PKG_PHASE can simply check if a certain phase
>   *would* be executed in the current make instance.  This should provide
>   equivalent functionality all within a single make invocation.
> 
> * Many following targets are recursed, but could be collapsed into a
>   sequence of subtargets for a certain build phase.  In some cases these
>   would involve slight shellcode modifications (and some of those are
>   actually simplifications).  In other cases it may involve splitting a do-*
>   target, which is currently recursing, into several separate targets
>   instead.

Yay, that sounds great! because pkgsrc is "slow" right now... IMHO :)


--Signature=_Fri__12_Nov_2004_09_26_50_+0100_+LE/ZSXOcB8LRgF7
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (NetBSD)

iD8DBQFBlHPKypkLYVDran0RAkznAKCDZNp/5mYEsWpu/aLM4uv87zYaDwCgut5w
vawheIkBD8wU9d81XQBTqNs=
=mejO
-----END PGP SIGNATURE-----

--Signature=_Fri__12_Nov_2004_09_26_50_+0100_+LE/ZSXOcB8LRgF7--