Re: pkg_rolling-replace

[Chavdar Ivanov <ci4ic4%gmail.com@localhost>]

On Sun, 30 Aug 2020 at 20:24, Greg Troxel <gdt%lexort.com@localhost> wrote:
>
>
> Chavdar Ivanov <ci4ic4%gmail.com@localhost> writes:
>
> > The package being replaced at the moment was clang-10.0.1
> > (clang-10.0.0 was installed at the moment). This obviously requires
> > the latest llvm, which had to be built, but was not in any of the
> > three lists (MISMATCH_TODO, REBUILD_TODO and UNSAFE_TODO). So
> > llvm-10.0.1 got successfully built, but as llvm-10.0.0nb1 was
> > installed, it didn't replace it.
>
> Are you really sure your tree has no local changes?  If there is a
> conflict and conflict markers, various tools behave badly.

I ran pkg_chk, there were very few problems. One package had mixed-up
Makefile, perhaps due to an early local change; I reloaded all files
for it. There are a number of packages installed from wip, some of
them with dependencies from wip, which later have been moved to the
main pkgsrc or even removed; I dealt with these manually. There were
also a bunch of obsolete packages installed, which were only leaf, I
removed them (two from x11, related to printing, and about 10-12 tex-
packages).

>
> Run "pkg_chk -uq" manually.  llvm should be in it, but it seems it is
> not.   llvm should show up in MISMATCH_TODO.
>
> Another bug is that llvm didn't show up as a dependency.  read the pkg_rr
> code, around line 463.   Run those commands in clang, and see if llvm is
> listed as a dependency.

This is what comes from the pkg_rr log:
......

===> Installing dependencies for clang-10.0.1
==========================================================================
The supported build options for clang are:

        tests

You can select which build options to use by setting PKG_DEFAULT_OPTIONS
or the following variable.  Its current value is shown:

        PKG_OPTIONS.clang (not defined)

==========================================================================
==========================================================================
The following variables will affect the build process of this package,
clang-10.0.1.  Their current value is shown below:

        * PYTHON_VERSION_DEFAULT = 37
        * TERMINFO_DEFAULT = terminfo

Based on these variables, the following variables have been set:

        * PYPACKAGE = python37
        * TERMINFO_TYPE = terminfo

You may want to abort the process now with CTRL-C and change their value
before continuing.  Be sure to run `/usr/bin/make clean' after
the changes.
==========================================================================
=> Tool dependency cmake>=2.8.1nb1: found cmake-3.18.2
=> Build dependency cwrappers>=20150314: found cwrappers-20180325
=> Full dependency llvm-10.0.1{,nb*}: NOT found
=> Verifying reinstall for ../../lang/llvm
=> Bootstrap dependency digest>=20010302: found digest-20190127
=> Fetching llvm-10.0.1.src.tar.xz
=> Total size: 35270168 bytes
....

pkg_rr correctly selects clang for upgrade; clang 10.0.1 depends on
llvm-10.0.1, it builds llvm-10.0.1 and tries to install it, but fails,
as llvm-10.0.0nb1 is already installed.

pkg_rr somehow has failed to notice that llvm also needs an update and
it needs to be done ahead of the clang update - as a result of the
sort.

>
> >> Can you find out from the output (that I hope you saved) whether llvm
> >> was on the MISMATCH list?

That is. llvm was not listed in any of the lists.

> >
> >  I can post the three lists, but they are rather long. For whatever
> > reason pkg_rr hadn't recognized llvm as a package in need of an
> > upgrade. AS above, llvm was not in any of the lists.
>
> That's what I needed to know.  It should have gotten into MISMATCH.
>
> > I used to do this as well; now I prefer using only '-uv'. With '-k'
> > one can let it go over as many as packages it can, but it may leave
> > the packages in inconsistent state, if I ca say this - I use this
> > build host as a pkgin server and if I use '-uvk', then get the list
> > with the '-' at the end and go over each of them, thinking that
> > everything would be ok, there are still unresolved upgrades from
> > packages depending on these last ones. So at the moment I run only
> > '-uv', sorting out each failure as it goes, then cleaning the work
> > directories and carrying on. Perhaps it is not the right thing, but it
> > seems better then the previous.
>
> What you are doing is perfectly reasonable.  I use -k because my
> computer makes more progress when I am not paying attention.  In theory,
> and I think mostly in practivce, anything replaced later after fixing
> will result in further replaces in order, so things should end up ok.

This worked for me for a while, but broke somewhere in the switch to
the later kde frameworks.

>
> > Anyway, this seems to happen more often now, I have several similar
> > logs (I keep the pkg_rr logs, if they are of interest), e.g. with
> > python3.7 as well ( some python package using the default 3.7 gets
> > queued for rebuild before python3.7, which leads to python3.7 being
> > rebuilt and its installation fail, as the previous version is
> > obviously installed).
>
> You said "queued before"; do you mean "python37 is in MISMATCH but
> things are misordered" or "python37 is not in MISMATCH"?


Here is the example (the first failure yesterday):
....
16803 rr> MISMATCH_TODO=[engrampa karchive py37-zipp quazip
xfce4-thunar-archive-plugin bmp fluidsynth jack libcanberra
libmatemixer libsamplerate libsndfile m      oc musicpd openal-soft
pulseaudio sox xfce4-mixer xfce4-xmms-plugin librecad openscad qcad
TECkit py37-sqlite3 py37-sqlite3 GConf-ui SDL SDL2 at-spi auto
moc4 bison cbindgen cmake dconf doxygen gdbus-codegen gdl glib2 gmtk
grantlee kbookmarks kcmutils kconfig kcoreaddons kcrash kdeclarative
kdesdk-strigi-a      nalyzers kdesdk-thumbnailers kdoctools ki18n
kidletime kio kitemmodels knotifications knotifyconfig kpackage kparts
kpeople kpty kross krunner kservice k      texteditor kwayland
libappindicator libbonoboui libdazzle libdbusmenu-glib
libdbusmenu-gtk3 libdbusmenu-jsonloader libdbusmenu-qt libdbusmenu-qt5
libgee       libglade libgnome libgnomeui libhandy libindicator
libkgapi libpeas libuv libwnck libwnck3 m17n-lib mate-common meson
ninja-build nspr nss okteta p5-pang      o pango pangomm py37-cffi
py37-cffi py37-curses py37-curses py37-faker py-gobject-shared
py37-gobject3 py37-gobject3 py37-idle py37-jupyter_client py37-p
ip py37-pip py37-readline py37-setuptools_scm py37-setuptools_scm
py37-test-cov qjson spdlog threadweaver xfce4-conf xfce4-dev-tools
pkgsrc-guide leafpad       lyx pluma vim-gtk3 xfce4-mousepad xournal
qemu simh gnucash gnucash-docs SDL2_ttf fontforge gucharmap amor
bzflag dhewm3 openhexagon simgear gpsd viking       ImageMagick
ImageMagick6 breeze-icons cairo cairo-gobject cairomm clutter
clutter-gtk clutter-gtk0.10 cogl dvipng eog eom gd gegl gexiv2 gimp
gimp-color      -manager gimp-exif-browser gimp-fix-ca gimp-jxr
gimp-liquid-rescale gimp-rawphoto gimp-refocus-it gimp-resynthesizer
gimp-ufraw girara gnome-themes-stand      ard gnuplot gpicview
graphviz gtkimageview inkscape]
    1 rr> REBUILD_TODO=[]
    2 rr> UNSAFE_TODO=[dmenu qt5-qtbase windowmaker xapps poppler
xterm tk m17n-lib spectrwm dwm pango cairo gd sox podofo librsvg
mplayer libbluray slim pulse      audio gimp gtk3+ openjdk8 mencoder
libass openjdk11 vte gtk2+ bmp transcode xnedit xetex xine-ui mlterm
mjpegtools wld cups-filters alacritty-v0.5.0 moti      f openbox
musicpd mate-settings-daemon st-term ffmpeg2 mate-control-center
qt4-libs py37-apsw ffmpeg4 kde-runtime4 ffmpeg3 p5-PerlMagick
ImageMagick6 cbi      ndgen openscad libreoffice gst-plugins1-lame
xscreensaver graphviz fltk hs-X11-xft ghostscript-agpl rxvt-unicode
ImageMagick emacs28]

The lists above do not show python3.7. The presently installed version
is 3.7.8. py37-sqlite3 shows dependency on python3.7.9 and builds it,
but the installation fails again, as python3.7.8 is installed.
Basically the same as with clang/llvm. I then manually replaced python
and reran pkg_rr, which then caught all other py37 packages for
replacement.

>
> Also, pkg_rr, pkg_chk, and tools in general, do not deal well with
> multiversion things like pyNN-foo, but they are msotly ok for the
> default version.

They also do not deal well with packages having the same name. but
different underlying versions - e.g. pkg_chk showed

archivers/py-zipp1 - py37-zipp-3.1.0nb1 > py37-zipp-1.2.0 - ignoring

(it sees that py-zipp1 package should be also called 'py37-zipp' and
as the installed version is higher, it ignores it - but it will miss a
new version of py-zipp this way, or so I think).

But one deals manually with such issues.


-- 
----