Re: Python 2 fade-out

[Greg Troxel <gdt%lexort.com@localhost>]

Thomas Klausner <wiz%NetBSD.org@localhost> writes:

> Hi!
>
> Python 2 support upstream was stopped in 2020[1], and most software
> projects have used the time to switch to Python 3. Many upstreams have
> stopped supporting Python 2 and in pkgsrc we held out longer than
> most, leading to packaging old versions in parallel (which get no bug
> fixes or security updates), and too much breakage happening again and
> again because of incorrectly declared Python version support in
> packages.
>
> I don't think the additional effort spent on this is worth our time,
> so long-term, I'd like to remove Python 2 from pkgsrc. There is no
> definitive date for that.

This is mostly fine as "long term", but

  people use pkgsrc to get software that they use with non-pkgsrc
  software

  python 2.7 itself is going to be needed by various people for a very
  long time

  Just having python27 in pkgsrc (vs py-foo just for 27, py-foo
  supporting 27, 27 accepted by default) does not cause problems, or at
  least no one has made an argument that it does.   As I understand
  things, the problems are from having to mark things not-for-27 as
  updates happen, and really the problem is bulk build fallout from not
  marking depending packages.

As to the first point, for a long time there was a notion of large,
non-public codebases in 27, for which people wanted to use pkgsrc as the
base.  I think we've reached the point that we are no longer to spend
effor to accomodate that, especially for environments with paid staff.
Perhaps there are no longer any such environments anyway.

Therefore I think we should keep python27 itself in the normal practice
that removal is appropriate when there is a genuine belief that there
are no users, or the pain of having it outweighs the benefit.  I would
guess that is some years away; it's certainly not going to be in 2024.

> As an intermediate step, I'd like to know what pkgsrc packages that
> need Python 2 have an active user base, and remove the others or
> remove their Python 2 support and limit them to Python 3 if they
> support both. I think that most software using Python 2 is dead anyway
> (otherwise it would have been ported to Python 3 in the meantime).

"not maintained upstream" and "no users" are different, but I concur
that it is reasonable to prune 27 support from packages when there is a
genuine belief that there are no users, and indeed asking here is how we
determine that :-)

> So, if you have python27 installed, please let me know what packages
> you have installed that need it.

For me, I think it's that unifi needs mongodb3, and mongodb3 needs
python27 itself and py27-scons.  I think it's likely that other things
need py27-scons.  scons is almost as regrettable as python3's disregard
for API stability, but it's a fact of life.  (Yes, I know there's an
option for mongodb4, and also that unifi on NetBSD is quite troubled
these days.)

That leads to "keep py27-scons" as the immediate exception to "prune 27
support".

Looking at my binary packages (pruning earlier ones from the list)

-rw-r--r--  1 gdt  wheel    86506 Apr 20 21:14 /links/packages/All/py27-cairo-1.18.2nb6.tgz
-rw-r--r--  1 gdt  wheel   775630 Apr 20 21:15 /links/packages/All/py27-gobject-2.28.7nb5.tgz
-rw-r--r--  1 gdt  wheel  3377514 Apr 20 23:21 /links/packages/All/py27-gtk2-2.24.0nb48.tgz
-rw-r--r--  1 gdt  wheel  1273415 May  8 08:03 /links/packages/All/py27-setuptools-44.1.1nb1.tgz
-rw-r--r--  1 gdt  wheel   201423 May  8 08:03 /links/packages/All/py27-yubiauth-0.3.9nb1.tgz

# pkg_delete python27
Package `python27-2.7.18nb15' is still required by other packages:
        rdiff-backup-1.2.8nb13
        py27-cairo-1.18.2nb6
        py27-gobject-2.28.7nb5
        py27-gtk2-2.24.0nb48
        py27-setuptools-44.1.1nb1
        py27-yubiauth-0.3.9nb1

that amounts to

  py27-gtk2 (pinenty-gtk)

  py27-yubiauth (not sure on status of that, will check)

  rdiff-backup (needs an update, but just dropping it is not reasonable
  as backup formats live forever)

as the "keepable" users.

> I'm also thinking of removing 27 from the default
> PYTHON_VERSIONS_ACCEPTED (so that packages supporting Python 2 have to
> actively declare that), but I don't know how much this would
> break. Can someone please try this?

This makes sense to me.

In theory such a change would drop 27 from default and add it back where
needed.  devel/py-scons and devel/py-scons4 are already hard-coded to
27.

I've just applied this to my tree and will continue pkg_rr as updates
happen and see what breaks.  I'll also try to build unifi, pinenty,
rdiff-backup after removing everything that only it needs (pkgin ar).

--- pyversion.mk.~1.157.~	2023-11-09 19:51:25.505773460 -0500
+++ pyversion.mk	2024-05-08 07:56:02.766332604 -0400
@@ -31,7 +31,7 @@
 #	preferred over later ones.
 #
 #	Possible values: 312 311 310 39 38 27
-#	Default: 312 311 310 39 38 27
+#	Default: 312 311 310 39 38
 #
 # PYTHON_VERSIONS_INCOMPATIBLE
 #	The Python versions that are NOT acceptable for the package.
@@ -107,7 +107,7 @@ PYTHON_VERSION_DEFAULT?=		311
 .else
 PYTHON_VERSION_DEFAULT?=		310
 .endif
-PYTHON_VERSIONS_ACCEPTED?=		312 311 310 39 38 27
+PYTHON_VERSIONS_ACCEPTED?=		312 311 310 39 38
 PYTHON_VERSIONS_INCOMPATIBLE?=		# empty by default
 
 # transform the list into individual variables