Re: pkg_rolling-replace unexpected behaviour

[Greg Troxel <gdt%lexort.com@localhost>]

Chavdar Ivanov <ci4ic4%gmail.com@localhost> writes:

> Perhaps I am wrong; I haven't read the source code yet. My
> understanding for pkg_rolling replace was that it finds and rebuild
> first the deepest in the dependence tree packages, ending with the
> leaves.

Yes, except that it's more complicated than "deepest"

  it only considers rebuilding things that are unsafe_depends or
  mismatched (or rebuild=YES)

  the initial calculation of the graph only looks at package
  dependencies, not build dependencies

  before building a package, pkg_rr finds build deps and adds them and
  recalculates.  usually a few rounds of this happen before it actually
  rebuilds anything, and then it stablizes.

  you said "deepest", but in general there is a complicated graph that
  is not a tree, in that there are multiple edges where a tree doesn't
  allow them, and there can be disconnected parts.  pkg_rr does a tsort
  and picks the first thing.  This is, absent bugs, guaranteed to be
  something such that there is no package that should be rebuilt before,
  but there are usually other packages that didn't tsort first that
  would also be correct first choices.

> I just cvs updated my pkkgsrc tree and started 'pkg_rolling-replace
> -uvk', a mere 10 days after my last go, so didn't expect that many
> packages to get updated.

It really depends.  a perl update needs a lot of rebuilds (but this is
usually not exciting).

> I noticed the second package to be rebuilt
> was
> ...
> -rw-r--r-- 2 root wheel   32355261 Nov 22 16:36 gimp-2.10.14nb1.tgz
> ...
> Then it proceeded with perl-5.30.1 and some modules, and again
> ...
> -rw-r--r-- 2 root wheel   32354941 Nov 22 17:19 gimp-2.10.14nb1.tgz
> ....
>
> I have noticed similar behaviour before, but this time I was able to
> catch it on screen (previously the repeated build package files would
> have been overwritten).
>
> Any idea why?

No, and probably either you have found a bug or perhaps have a corrupt
pkgdb of some kind.  I am happy to look into any data.  By now, it works
ok the way I run it, but it surely has bugs that trip in other
circumstances.

This sort of smells like failing to get all the dependency info and thus
replacing things too early, then having them marked unsafe_depends and
then doing it again.

I would suggest:

  run "pkg_admin rebuild-tree" (now, since I am suspicious, and this is
  a good thing periodically).  Let me know if it reports unresolved
  dependencies.

  (always) clean up all work directories, every time you do anything
  with pkgsrc, and certainly before a pkg_rr

  run "pkg_rolling-replace -uvk < /dev/null > RR.000 2>&1 &" so that you
  can look later


I also suggest using ccache so that extra rebuilds don't hurt that much.