pkg/28764: Using bin-install under pkg_comp fails badly

[kre%munnari.OZ.AU@localhost]

>Number:         28764
>Category:       pkg
>Synopsis:       Using bin-install under pkg_comp fails badly
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    pkg-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Dec 23 18:09:00 +0000 2004
>Originator:     Robert Elz
>Release:        NetBSD 1.6U   (pkgsrc -current as of yesterday)
>Organization:
        Prince of Songkla University
>Environment:
        Irrelevant info from system used to send-pr follows
System: NetBSD fuchsia.noi.kre.to 1.6U NetBSD 1.6U (FUCHSIA) #51: Wed Jul 23 
16:06:08 ICT 2003 kre%fuchsia.cs.mu.OZ.AU@localhost:/usr/obj/sys/FUCHSIA i386
Architecture: i386
Machine: i386
>Description:
        I am using pkg_comp on a 2.0F (or something like that) system to
        build binary packages for 2.0. (using libkver)   I was using
        pkg_comp in the "normal" way (BUILD_TARGET=package) with everthing
        working perfectly.   Then perl came to be upgraded, and a bunch of
        stuff I hadn't yet built demanded the new perl version.   I really
        doubt that any of that really needed it (other stuff was still happy
        with the older version, but that's not important for now).

        Updating perl meand blowing away a large fraction of what was
        already compiled and installed - of course, this is all in the
        pkg_comp playground, so blowing it all away is harmless.  So,
        I did that.    Now I know that I have *lots* of binary packages
        already built that can simply be re-installed after the new
        perl is there (the latter part -new perl- is trivial of course).  
        Of course, there's no need to reinstall any of them, unless
        they're needed as dependencies for some other package (once all
        the compiles are done, the whole playground will be nuked).

        This should be a perfect time to use bin-install - so I change
        BUILD_TARGET to bin-install (set AUTO_TARGET that way too, just
        for completeness), and go back to building all the stuff I
        hadn't done yet when the perl upgrade happened (this was a few
        days ago of course, it has taken me this long to wake up to
        what the result was).

        What I expected was that any of my pre-compiled packages which
        was needed as a dependency of something I was newly building
        would be pkg_add'd, and anything that actually needed to be
        compiled from source (either an updated version since the
        prefious compile, or anything I hadn't compiled previously)
        would be the subject of "make package" (or the equivalent in
        action) and I'd end up with binary packages for those.

        That's not what happened.

        For packages I named on the command line for pkg_comp build
        everything looked as if it would have worked as expected.
        Certainly binary packages were built.   It looked as if
        they would have been used to install - but of course, I wasn't
        compiling anything I'd already made, so there never was a
        binary package existing for anything I put on the command line.

        But for dependencies everything happens exactly the opposite
        of what I'd wanted and expected.   No pkg_add was ever done,
        instead hundreds (perhaps a thousand) packages were all
        recompiled from source,    OK, that's just a waste of time,
        no great harm done (and one might even claim its safer that
        way).   But, at the same time, no binary packages were built
        for anything that was a dependency of any of the packages
        named on pkg_comp's command line.

        What's worse of course, is that where I listed packages on the
        command line in the "wrong" order (I was mostly just using
        lexical order out of the various category directories in pkgsrc)
        so that I say "pkg_comp build A B" and A depends upon B, then
        B would be compiled and installed as a dependency of A, but no
        binary package made, A gets compiled, installed, and packaged,
        then everything gets cleaned (A and B, with cleandepends=YES)
        then B gets compiled again, but now the install fails, as B
        is already installed, so no package gets made for it, once cgain
        (even though it was listed on the command line).

>How-To-Repeat:
        As above, make a few binary packages using pkg_comp (pkg_comp
        may not be important here, I haven't tried the similar thing
        for a "native" install to check it), pkg_delete a whole bunch
        of the basic ones of them from the pkg_comp playground, (ones
        that "everything" needs), then go and start making some other
        packages with pkg_comp set to use bin-install instead of
        "package" as its target.

        Notice that pkg_add never happens, or not unless you name a
        package to build, explicitly, that you had built before,
        Also notice that the only binary packages that are built are
        the ones that are explicitly named to pkg_comp build.

        Observation (just using ps to see what was actually happenening)
        suggests that something is causing make to be run with
        DEPENDS_TARGET set to reinstall rather than bin-install (or
        package).

>Fix:
        No idea.   The NetBSD mk system is way beyond my understanding,
        and the pkgsrc version of it goes eons beyond even that.

        As a workaround, I'm just removing everything I built in the past
        few days, and building again the old way (with BUILD_TARGET=package)
        which works fine.   Or at leat, I'm expecting that it will, this part
        is just starting... (but no reason to believe it won't).