upgrading pkgsrc, and variations on the theme of bin-install

[Taylor R Campbell <campbell%mumble.net@localhost>]

Hello!  Today as I was upgrading my NetBSD machines to pkgsrc-2008Q3,
I noticed a small issue with the way that bin-install behaves,
suggesting that what I want is a variant of bin-install.  bin-install
installs whatever PKGNAME_REQD indicates is required of it, but I want
a variant of bin-install that installs a binary only if it *exactly*
matches the version in pkgsrc, independent of PKGNAME_REQD.  Or
perhaps I want something different, so I shall expound upon the
context.

I upgrade pkgsrc by first building binary packages inside a pkg_comp
chroot.  To do this, I run `pkgdepgraph -o -a -a -s' to get a
topologically sorted list of pkgsrc paths of installed packages on my
system.  I then run `make checksum' in all of those paths in order to
fetch any distribution files that I don't have and to check
vulnerabilities.  After I have run `make checksum' in all of those
paths, I pass them all to `pkg_comp auto' to build them all in a
chroot.  I could run `pkg_comp auto' once for *each* package I wish to
upgrade, but this would solve only one of two problems I encounter.
(This would have some other nice advantages, by better isolating the
build environments of the packages, and it would also take much longer
to repeatedly rebuild the chroot environment -- some three hundred
times.)

The first problem is that when building and installing a large number
of packages into the chroot environment, some of the packages have
loose dependencies that later packages tighten; for example,
x11/libX11 (and many other packages that simply include
./../x11/inputproto/buildlink3.mk) depends on inputproto>=1.4, while
x11/modular-xorg-server depends on inputproto>=1.4.4.  Since
x11/libX11 happened to be built first into the chroot environment, it
installed inputproto-1.4.3.tgz, of which I already had a binary in my
local directory of binary packages.  The subsequent attempt to build
x11/modular-xorg-server failed because inputproto-1.4.3 was already
installed when x11/modular-xorg-server asked that inputproto-1.4.4 be
installed.

The second problem is that this process won't build newer versions of
some packages, if (1) I didn't have those packages installed, (2) I
had binary packages lying around, and (3) a package I newly installed
[perhaps with different options, or just because of a new version]
depends on one.  For example, in 2008Q2 on one machine, I had
editors/emacs built with xaw graphics, and then I changed it to be
built with gtk graphics.  But since I already had an older binary
package of gtk2+, which was good enough for Emacs, I didn't wind up
with the latest version in pkgsrc.

One way to fix both of these problems would be to set DEPENDS_TARGET
to my proposed variant of bin-install in the chroot environment's
mk.conf; then the only binary packages that would be used in the
upgrade process would be those whose versions match the upgraded
pkgsrc's exactly -- which was the object of using the old binary
packages in first place: to immensely reduce the number of packages
that need to be rebuilt from source.

Possibly there are other ways to upgrade pkgsrc, but I'd like to
maintain the following criteria:

1. that the process rebuild only those packages whose rebuilding is
   necessary, by using existing binary packages when their versions
   match pkgsrc's exactly;

2. that the whole process occur inside a chroot, so that my working
   system remains untouched, particularly if anything fails; and

3. that the process be incremental, so that if it fails halfway
   through after ten hours, I need not repeat those ten hours.

Thoughts?  Suggestions?