pkg/38232: pkgtools/pkg_comp botches testing for adequate pkg_install

[kre%munnari.OZ.AU@localhost]

>Number:         38232
>Category:       pkg
>Synopsis:       pkgtools/pkg_comp botches testing for adequate pkg_install
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    pkg-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Mar 12 14:50:04 +0000 2008
>Originator:     Robert Elz
>Release:        NetBSD 3.99.15  (pkgsrc current within past 8 hours)
>Organization:
        Prince of Songkla University
>Environment:
System: NetBSD jade.coe.psu.ac.th 3.99.15 NetBSD 3.99.15 
(GENERIC-1.696-20060125) #8: Wed Jan 25 04:59:39 ICT 2006 
kre%jade.coe.psu.ac.th@localhost:/usr/obj/current/kernels/JADE_ASUS i386
Architecture: i386
Machine: i386
>Description:
        Sometime recently pkgsrc has begun using pkg_info -E
        That doesn't exist in the NetBSD 3.0 version of pkg_install
        (which is version 20051122)

        pkg_comp tests if it has an adequate version of pkg_install
        installed on the system, and installs the latest from
        pkgsrc if not, but the way it does this doesn't work (or
        not with the current environment).

>How-To-Repeat:
        Start with an empty pkg_comp sandbox
                pkg_comp removeroot
                pkg_comp makeroot

        That's probably enough to see the problem, if you're using
        an old enough version of NetBSD as the base system (I use
        NetBSD 3.0 so my binary packages work just about anywhere - and
        are all using a consistent set of internal interfaces) and if
        you have "makeroot" auto-install anything.

        What I see is ...

PKG_COMP ==> Mounting sandboxed filesystems
PKG_COMP ==> Checking if pkg_install is up to date
PKG_COMP ==> Building and installing editors/vim-share
===> Binary install for vim-share-7.1.270
pkg_info: unknown option -- E
usage: pkg_info [-BbcDdFfhIikLmNnpqRrSsVv] [-e package] [-K pkg_dbdir] [-l prefi
x]
                pkg-name [...]
       pkg_info [-a | -u] [flags]
       pkg_info -Q variable pkg-name [pkg-name ...]

       Notice the "Checking if pkg_install is up to date" (which did
       nothing) and then the lack of -E.  If I had the error log
       of the "pkg_comp makeroot" which was the previous command
       before this build, you'd see the same kind of noise there.

>Fix:
        No fix, but an explanation.

        pkg_comp does ...


    cat >> $script <<EOF
cd /usr/pkgsrc/pkgtools/pkg_comp
fail=\$(make show-var VARNAME=PKG_FAIL_REASON)
if echo \$fail | grep "package tools installed on this system are out of date" >
/dev/null; then
    echo "PKG_COMP ==> pkg_install is out of date; rebuilding"
    cd /usr/pkgsrc/pkgtools/pkg_install
    make && make install && make clean
fi
EOF
    chmod +x $script
    chroot $DESTDIR /pkg_comp/tmp/`basename $script`
    rm $script

        That is, it looks at the result from PKG_FAIL_REASON
        and if the "make" says the pkg tools are out of date,
        it installs new ones.   Seems good enough, but ..

        (from bsd.pkg.mk)


# Check that we are using up-to-date pkg_* tools with this file.
.if !defined(NO_PKGTOOLS_REQD_CHECK)
.  if ${PKGTOOLS_VERSION} < ${PKGTOOLS_BASE_REQD}
PKG_FAIL_REASON+='The package tools installed on this system are out of date.'

        That message only appears if the version of the pkgtools
        is < PKGTOOLS_BASE_REQD and

bsd.pkg.mk:PKGTOOLS_BASE_REQD=  20051103

        which is a truly ancient version - only a system older than
        NetBSD 3.0 (ie: a 2.* or earlier) can possibly have such old
        tools installed (for NetBSD, I doubt any of the other systems
        that pkgsrc supports can ever have anything so ancient).

        That is, the test in check_pkg_install() in pkg_comp is
        essentially (these days) a waste of space and time.  Only
        someone installing on ancient NetBSD is going to get any
        benefit from it at all.

        I don't know if the correct solution is to bump
        PKGTOOLS_BASE_REQD to something a little more recent,
        or to redo the test in pkg_comp to something that catches
        more problems, or to simply fix pkgsrc so it isn't using
        pkg_info -E anywhere before it has a chance to update the
        pkgtools as part of the normal update, ie: using this from bsd.pkg.mk


.  elif ${PKGTOOLS_VERSION} < ${PKGTOOLS_REQD}
BOOTSTRAP_DEPENDS+=     pkg_install>=${PKGTOOLS_REQD}:../../pkgtools/pkg_install
.  endif

        Of course, even there

bsd.pkg.mk:PKGTOOLS_REQD=               20070802

        so that's not that new a version either (I don't know
        whether that one implemented pkg_info -E or not, I don't
        have it around to check).

        For me, that one does eventually kick in, and a new pkg_install
        is built & installed as a dependency of whatever package is
        being built, but even doing that results in a bunch more
        usage messages from the old pkg_info (which is why I suspect that
        pkg_info -E is being used too soon in pkgsrc, that is, before
        the pkgtools have a chance to update themselves to a version
        that is expected to support that).