Re: PLIST erros on ibus with python312 "*-cpython-312.opt-2.pyc" files

[Izumi Tsutsui <tsutsui%ceres.dti.ne.jp@localhost>]

I wrote:

> I'm trying to update ibus to 1.5.32 but get the following errors
> (on NetBSD/i386 10.1 and pkgsrc-2025Q1 tree):
> 
> ---
> => Automatic manual page handling
> => Generating post-install file lists
> pkg_create: can't stat `/s/cvs/pkgsrc/inputmethod/ibus/work.i386/.destdir/usr/pkg/share/ibus/setup/__pycache__/emojilang.opt-2.cpython-312.pyc'
> pkg_create: can't stat `/s/cvs/pkgsrc/inputmethod/ibus/work.i386/.destdir/usr/pkg/share/ibus/setup/__pycache__/engineabout.opt-2.cpython-312.pyc'

 :

> => Checking file-check results for ibus-1.5.32
> ERROR: ************************************************************
> ERROR: The following files are in the PLIST but not in /s/cvs/pkgsrc/inputmethod/ibus/work.i386/.destdir/usr/pkg:
> ERROR:         /s/cvs/pkgsrc/inputmethod/ibus/work.i386/.destdir/usr/pkg/share/ibus/setup/__pycache__/emojilang.opt-2.cpython-312.pyc
> ERROR:         /s/cvs/pkgsrc/inputmethod/ibus/work.i386/.destdir/usr/pkg/share/ibus/setup/__pycache__/engineabout.opt-2.cpython-312.pyc

 :

> ERROR: The following files are in /s/cvs/pkgsrc/inputmethod/ibus/work.i386/.destdir/usr/pkg but not in the PLIST:
> ERROR:         /s/cvs/pkgsrc/inputmethod/ibus/work.i386/.destdir/usr/pkg/share/ibus/setup/__pycache__/emojilang.cpython-312.opt-2.pyc
> ERROR:         /s/cvs/pkgsrc/inputmethod/ibus/work.i386/.destdir/usr/pkg/share/ibus/setup/__pycache__/engineabout.cpython-312.opt-2.pyc

It turns out:

- the ibus developers have switched to using automake 1.17 (from 1.16.5)
  to generate release tar balls since 1.5.32-rc1
  (generated files by automake etc. are not in GitHub repository)

- automake 1.17 uses "$PYTHON -OO -c" on Byte-compiling python modules:

  https://github.com/autotools-mirror/automake/commit/bde43d0481ff540418271ac37012a574a4fcf097#diff-4715c6e06bb441763f93490e3f7ae81b3ba009b54fb1f6353599ace83af86e28R210

---
# Then byte compile w/more optimization.
case $python_major.$python_minor in
2.*|3.[0-4])
  ;;
*)  # Python 3.5+
  # See https://bugs.gnu.org/38043 for background.
  $PYTHON -OO -c "
import sys, os, py_compile, imp
print('Byte-compiling python modules (-OO version) ...')
for file in sys.argv[1:]:
    $pathtrans
    $filetrans
    if not os.path.exists(filepath) or not (len(filepath) >= 3
                                            and filepath[-3:] == '.py'):
        continue
    print(file, end=' ', flush=True)
    if hasattr(imp, 'get_tag'):
        py_compile.compile(filepath, imp.cache_from_source(filepath), path)
    else:
        py_compile.compile(filepath, filepath + 'o', path)
print()" "$@" 2>/dev/null || exit $?
  ;;
esac
---

- Python 3.5+ generates "foo.cpython-312.opt-2.pyc" with "-OO"
  (rather than "foo.cpython-312.opt-1.pyc" with "-O") per PEP 488:

  https://peps.python.org/pep-0488/#proposal

---
For example, a source file named foo.py in CPython 3.5 could have
the following bytecode files based on the interpreterfs optimization
level (none, -O, and -OO):

    0: foo.cpython-35.pyc (i.e., no change)
    1: foo.cpython-35.opt-1.pyc
    2: foo.cpython-35.opt-2.pyc
---

- it looks currently pkgsrc/lang/python/plist-python.awk only handles
  "foo.cpython3*.opt-1.pyc" case for "-O"

  https://github.com/NetBSD/pkgsrc/blob/pkgsrc-2025Q1/lang/python/plist-python.awk

---
### For each Python bytecode file entry, convert directory and file name.
###
/^[^@]/ && /[^\/]+\.py[co]$/ {
	sub(/[^\/]+\.py[co]$/, "__pycache__/&")
	sub(/\.py[co]$/, ".cpython-" PYVERS "&")
	if (PYVERS ~ /^3[0-9]+$/ && $0 ~ /\.pyo$/) {
		sub(/\.pyo$/, ".opt-1.pyc")
	}
}
---

I'm not sure how "-OO" case should be handled (to generate
foo.cpython3*.opt-2.pyc in PLIST), but if noone will handle it soon
I will patch ibus-1.5.32/py-compile to force using "-O" rather
than "-OO" as autoconf 1.17 defaults.

Thanks,
---
Izumi Tsutsui