pkg/53673: Building Python 3.7 on Linux fails with error "pkg_create: can't stat" at stage "Generating post-install file lists"

To: pkg-manager%netbsd.org@localhost,gnats-admin%netbsd.org@localhost,pkgsrc-bugs%netbsd.org@localhost
Subject: pkg/53673: Building Python 3.7 on Linux fails with error "pkg_create: can't stat" at stage "Generating post-install file lists"
From: aikchar%codeghar.com@localhost
Date: Wed, 17 Oct 2018 01:50:00 +0000 (UTC)

>Number:         53673
>Category:       pkg
>Synopsis:       Building Python 3.7 on Linux fails with error "pkg_create: can't stat" at stage "Generating post-install file lists"
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    pkg-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Oct 17 01:50:00 +0000 2018
>Originator:     Hamza Sheikh
>Release:        trunk - git commit c18957a255f58b90000cad752b5c9a2764faf62d
>Organization:
>Environment:
Linux ec41ea7b3621 4.9.93-linuxkit-aufs #1 SMP Wed Jun 6 16:55:56 UTC 2018 x86_64 Linux
>Description:
I am building Python 3.7 on Alpine Linux 3.8 within ``docker build``. The build fails with the following messages. The full build log is available from https://mushtarka.codeghar.com/netbsd/2018/gnats/python37/build.log.


        running install_scripts
        copying build/scripts-3.7/pydoc3.7 -> /root/pkg/work/lang/python37/work/.destdir/usr/pkg/bin
        copying build/scripts-3.7/2to3-3.7 -> /root/pkg/work/lang/python37/work/.destdir/usr/pkg/bin
        copying build/scripts-3.7/pyvenv-3.7 -> /root/pkg/work/lang/python37/work/.destdir/usr/pkg/bin
        changing mode of /root/pkg/work/lang/python37/work/.destdir/usr/pkg/bin/pydoc3.7 to 755
        changing mode of /root/pkg/work/lang/python37/work/.destdir/usr/pkg/bin/2to3-3.7 to 755
        changing mode of /root/pkg/work/lang/python37/work/.destdir/usr/pkg/bin/pyvenv-3.7 to 755
        rm /root/pkg/work/lang/python37/work/.destdir/usr/pkg/lib/python3.7/lib-dynload/_sysconfigdata_linux.py
        rm -r /root/pkg/work/lang/python37/work/.destdir/usr/pkg/lib/python3.7/lib-dynload/__pycache__
        Creating directory /usr/pkg/man
        Creating directory /usr/pkg/man/man1
        /usr/bin/install -c -o root -g root -m 644 ./Misc/python.man  /root/pkg/work/lang/python37/work/.destdir/usr/pkg/man/man1/python3.7.1
        if test "xno" != "xno"  ; then  case no in  upgrade) ensurepip="--altinstall --upgrade" ;;  install|*) ensurepip="--altinstall" ;;  esac;  LD_LIBRARY_PATH=/root/pkg/work/lang/python37/work/Python-3.7.0 ./python -E -m ensurepip  $ensurepip --root=/root/pkg/work/lang/python37/work/.destdir/ ;  fi
        => Automatic manual page handling
        => Generating post-install file lists
        pkg_create: can't stat `/root/pkg/work/lang/python37/work/.destdir/usr/pkg/lib/python3.7/lib-dynload/nis.so'
        => Checking file-check results for python37-3.7.0
        ERROR: ************************************************************
        ERROR: The following files are in the PLIST but not in /root/pkg/work/lang/python37/work/.destdir/usr/pkg:
        ERROR:         /root/pkg/work/lang/python37/work/.destdir/usr/pkg/lib/python3.7/lib-dynload/nis.so
        *** Error code 1

        Stop.
        bmake[1]: stopped in /root/pkgsrc/lang/python37
        *** Error code 1

        Stop.
        bmake: stopped in /root/pkgsrc/lang/python37
>How-To-Repeat:
I was working with pkgsrc from https://github.com/NetBSD/pkgsrc and commit c18957a255f58b90000cad752b5c9a2764faf62d made on 2018-10-15.

The mk.conf file is available from https://mushtarka.codeghar.com/netbsd/2018/gnats/python37/mk.conf. It's also pasted below.


        # Example /usr/pkg/etc/mk.conf file produced by bootstrap-pkgsrc
        # Tue Jun 26 23:08:17 UTC 2018

        .ifdef BSD_PKG_MK	# begin pkgsrc settings

        ABI=			64

        PKG_DBDIR=		/usr/pkg/pkgdb
        LOCALBASE=		/usr/pkg
        VARBASE=		/var
        PKG_TOOLS_BIN=		/usr/pkg/sbin
        PKGINFODIR=		info
        PKGMANDIR=		man

        TOOLS_PLATFORM.awk?=		/usr/bin/awk
        TOOLS_PLATFORM.sh?=		/bin/bash

        WRKOBJDIR=/root/pkg/work
        DISTDIR=/root/pkg/distfiles
        PYTHON_VERSION_DEFAULT=37

        .endif			# end pkgsrc settings


The pkg_install.conf file is available from https://mushtarka.codeghar.com/netbsd/2018/gnats/python37/pkg_install.conf. It's also pasted below.


        IGNORE_URL=https://nvd.nist.gov/vuln/detail/CVE-2018-10754
        IGNORE_URL=https://www.openssl.org/news/secadv/20180612.txt
        IGNORE_URL=https://nvd.nist.gov/vuln/detail/CVE-2018-1000035
        IGNORE_URL=https://nvd.nist.gov/vuln/detail/CVE-2018-1060
        IGNORE_URL=https://nvd.nist.gov/vuln/detail/CVE-2018-1061


The profile file is available from https://mushtarka.codeghar.com/netbsd/2018/gnats/python37/profile. It's also pasted below.


        export PATH
        PATH="/usr/pkg/sbin:/usr/pkg/bin:${PATH}"

        export MANPATH
        MANPATH="/usr/pkg/man:${MANPATH}"

        export LC_CTYPE
        LC_CTYPE="en_US.UTF-8"
        export LC_COLLATE
        LC_COLLATE="C"
        export LC_TIME
        LC_TIME="C"
        export LC_NUMERIC
        LC_NUMERIC="C"
        export LC_MONETARY
        LC_MONETARY="C"
        export LC_MESSAGES
        LC_MESSAGES="en_US.UTF-8"
        export LC_ALL
        LC_ALL=""


The Dockerfile I'm using is available from https://mushtarka.codeghar.com/netbsd/2018/gnats/python37/Dockerfile. It's also pasted below.

With this Dockerfile, run ``docker build -t localhost/tox:latest .``. It should fail in the RUN stanza on line 37 of the Dockerfile.


        FROM alpine as builder

        # https://github.com/justincormack/alpine-pkgsrc/blob/master/Dockerfile
        ENV \
        PATH=/usr/pkg/bin:$PATH \
        LIBABISUFFIX=""

        RUN apk --update add automake autoconf bash bzip2 curl file gcc g++ gawk git gmp-dev grep gzip libffi-dev libtool linux-headers m4 make ncurses-dev openssl-dev rsync sed tar unzip wget zlib-dev

        RUN    mkdir -p /root/pkgsrc \
            && curl -L -o /root/pkgsrc.tar.gz https://github.com/NetBSD/pkgsrc/archive/c18957a255f58b90000cad752b5c9a2764faf62d.tar.gz \
            && tar -C /root/pkgsrc -xzf /root/pkgsrc.tar.gz --strip=1

        RUN    export SH=/bin/bash \
            && cd ~/pkgsrc/bootstrap \
            && ./bootstrap

        RUN    mkdir -p /root/pkg/work \
            && mkdir -p /root/pkg/distfiles

        COPY profile /etc/profile.d/pkgsrc
        COPY mk.conf /usr/pkg/etc/mk.conf
        COPY pkg_install.conf /usr/pkg/etc/pkg_install.conf

        RUN    . /etc/profile.d/pkgsrc \
            && pkg_admin -K /usr/pkg/pkgdb fetch-pkg-vulnerabilities

        RUN    . /etc/profile.d/pkgsrc \
            && cd /root/pkgsrc/lang/nawk \
            && bmake install clean clean-depends \
            && sed -i.bak 's!TOOLS_PLATFORM\.awk.*!TOOLS_PLATFORM.awk?=		/usr/pkg/bin/nawk!' /usr/pkg/etc/mk.conf

        RUN    . /etc/profile.d/pkgsrc \
            && cd /root/pkgsrc/pkgtools/pkg_alternatives \
            && bmake install clean clean-depends

        RUN    . /etc/profile.d/pkgsrc \
            && cd /root/pkgsrc/lang/python37 \
            && bmake install clean clean-depends \
            && cd /root/pkgsrc/devel/py-pip \
            && bmake install clean clean-depends \
            && cd /root/pkgsrc/devel/py-virtualenv \
            && bmake install clean clean-depends

        RUN    . /etc/profile.d/pkgsrc \
            && cd /root/pkgsrc/lang/python36 \
            && bmake install clean clean-depends \
            && cd /root/pkgsrc/devel/py-pip \
            && bmake install clean clean-depends PYTHON_VERSION_DEFAULT=36 \
            && cd /root/pkgsrc/devel/py-virtualenv \
            && bmake install clean clean-depends PYTHON_VERSION_DEFAULT=36

        RUN    . /etc/profile.d/pkgsrc \
            && cd /root/pkgsrc/lang/python27 \
            && bmake install clean clean-depends \
            && cd /root/pkgsrc/devel/py-pip \
            && bmake install clean clean-depends PYTHON_VERSION_DEFAULT=27 \
            && cd /root/pkgsrc/devel/py-virtualenv \
            && bmake install clean clean-depends PYTHON_VERSION_DEFAULT=27

        RUN    . /etc/profile.d/pkgsrc \
            && cd /root/pkgsrc/lang/python35 \
            && bmake install clean clean-depends \
            && cd /root/pkgsrc/devel/py-pip \
            && bmake install clean clean-depends PYTHON_VERSION_DEFAULT=35 \
            && cd /root/pkgsrc/devel/py-virtualenv \
            && bmake install clean clean-depends PYTHON_VERSION_DEFAULT=35

        RUN    . /etc/profile.d/pkgsrc \
            && cd /root/pkgsrc/lang/python34 \
            && bmake install clean clean-depends \
            && cd /root/pkgsrc/devel/py-pip \
            && bmake install clean clean-depends PYTHON_VERSION_DEFAULT=34 \
            && cd /root/pkgsrc/devel/py-virtualenv \
            && bmake install clean clean-depends PYTHON_VERSION_DEFAULT=34

        RUN tar czf /root/pkg.tgz /usr/pkg


        FROM alpine as distro
        RUN    apk update \
            && apk upgrade \
            && apk add bash
        COPY --from=builder /root/pkg.tgz /root/pkg.tgz
        RUN tar xzf /root/pkg.tgz -C /usr --strip=1
        COPY profile /etc/profile.d/pkgsrc
        COPY bashrc_pkgsrc_edits.sh /root
        RUN    chmod +x /root/bashrc_pkgsrc_edits.sh \
            && /root/bashrc_pkgsrc_edits.sh
        RUN rm -rf /root/bashrc_pkgsrc_edits.sh /root/pkg.tgz /etc/profile.d/pkgsrc
        RUN /usr/pkg/bin/python3.7 -m pip install tox


The bashrc_pkgsrc_edits.sh file is available from https://mushtarka.codeghar.com/netbsd/2018/gnats/python37/bashrc_pkgsrc_edits.sh. It's also pasted below.


        #!/bin/sh
        cat /etc/profile.d/pkgsrc >> /etc/bash.bashrc



>Fix:

Prev by Date: Re: pkg/53671 (lang/rust build breaks on read-only source tree)
Next by Date: Re: pkg/53673: Building Python 3.7 on Linux fails with error "pkg_create: can't stat" at stage "Generating post-install file lists"
Previous by Thread: Re: pkg/53671 (lang/rust build breaks on read-only source tree)
Next by Thread: Re: pkg/53673: Building Python 3.7 on Linux fails with error "pkg_create: can't stat" at stage "Generating post-install file lists"
Indexes:

Home | Main Index | Thread Index | Old Index