pkgsrc-Users archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: RHEL/Centos 7 builtin vs pkgsrc zlib clash for Makefiles that include devel/zlib/buildlink3.mk

On 2020-08-15 10:56, Lai, Peter PW wrote:

-----Original Message-----
From: Jason Bacon [mailto:outpaddling%yahoo.com@localhost]
Sent: Friday, August 14, 2020 5:23 PM
To: Lai, Peter PW <peter.lai2%prattwhitney.com@localhost>; pkgsrc-users%netbsd.org@localhost
Subject: [External] Re: RHEL/Centos 7 builtin vs pkgsrc zlib clash for Makefiles
that include devel/zlib/buildlink3.mk

On 2020-08-10 12:05, Lai, Peter PW wrote:

-----Original Message-----
From: Jason Bacon [mailto:outpaddling%yahoo.com@localhost]
Sent: Monday, August 10, 2020 10:00 AM
To: Lai, Peter PW <peter.lai2%prattwhitney.com@localhost>;
pkgsrc-users%netbsd.org@localhost
Subject: [External] Re: RHEL/Centos 7 builtin vs pkgsrc zlib clash
for Makefiles that include devel/zlib/buildlink3.mk

On 2020-08-10 05:50, Lai, Peter PW wrote:

Environment:

* RHEL 7.7 (Centos 7 for those playing at home I guess) in Azure.
* Toolchain: Software collection (SCL) devtoolset-9

* Yum installs native zlib-devel as a dependency for openssl-devel

* openssl-devel was installed in order to support PREFER_NATIVE=

openssl

Observations:

In this environment because zlib-devel is installed, bootstrap by
default

picks up native zlib (/usr/include/zlib.h). When devel/libxml2 is
then built, it is built against native zlib (libz.so.1 => /lib64/libz.so.1).

Next, some packages that also include
"../../devel/zlib/buildlink3.mk" such

nginx is built, it pulls in devel/zlib. This is weird because RHEL 7
includes zlib
1.2.7 which satisfies zlib>=1.2.3 in buildlink3.

What happens after is that libxml2 and downstream libxml2
dependencies

(e.g. databases/postgresql) break because they see the pkgsrc zlib first:

/mnt/datadisk01/pkg/bin/postgres: /mnt/datadisk01/pkg/lib/libz.so.1:
no version information available (required by
/mnt/datadisk01/pkg/lib/libxml2.so.2)

# ldd /mnt/datadisk01/pkg/lib/libxml2.so.2
/mnt/datadisk01/pkg/lib/libxml2.so.2:
/mnt/datadisk01/pkg/lib/libz.so.1: no version information available
(required by /mnt/datadisk01/pkg/lib/libxml2.so.2)
/mnt/datadisk01/pkg/lib/libxml2.so.2:
/mnt/datadisk01/pkg/lib/libz.so.1: no version information available
(required by /mnt/datadisk01/pkg/lib/libxml2.so.2)

I worked around this by adding zlib to PREFER_NATIVE when
re-building

nginx, but is there a better way to resolve this? This looks somewhat
related to the continuing "pkg_install fails when archivers/xz is
installed" thread because RPATH manipulation could be involved, but I
am opening a separate thread in case it is not.
This does look like the same issue that we've been discussing
regarding xz and it's a great example of the kind of problems that
are avoided by using PREFER_PKGSRC=yes (and/or PREFER_NATIVE=no).

What's your motive for using native openssl instead of pkgsrc?  The
only issue I've seen with the latter is the need to manually install
security/mozilla-rootcerts* or equivalent to make curl, etc. work with

https.

If there's some other disadvantage to pkgsrc openssl, maybe we should
try to correct it.

The RHEL version of OpenSSL is backported/patched 1.0.2k with the FIPS

module compiled (and validated through vendor affirmed basis as per
https://urldefense.com/v3/__https://www.openssl.org/docs/fips.html__;!!
MvWE!V8OL2UV1NdFLyP770_BxDCbRbIfJxyghAQxVL9hpOXoMsXRu58-
41nFt2k-C-GdOu-Q44Q$  ). So provisional compliance with FIPS 140-2 level 1
can be met for binaries linked to wrapper module when the system & kernel
are fips enabled and the binary uses the required calling convention to
autoconfigure or manually configure FIPS operation mode: We fork the
pkgsrc repo and patch Makefiles for downstream builds' configure knobs
where applicable to enable the calling of openssl's fips mode for those apps.
I don't think this is worth correcting on the pkgsrc/NetBSD level for folks who
need this niche requirement but I sent the message because the xz thread
seemed related when dealing with a non-openssl-related base vs pkgsrc
library issue...
I bootstrapped on CentOS 7 using my modified bootstrap script (attached)
with zlib-devel and openssl-devel preinstalled, so pkgsrc should be able to
use native versions of both.  After installing libxml2, I verified that it's using
pkgsrc zlib.  Packages that depend on openssl are not picking up the Yum
installation, though, because it's one revision too old.  Yum's zlib meets the
current requirements, though, so as a test I added it to PREFER_NATIVE and
checked dependencies for libarchive before and after.  This confirmed that
my mk.conf configuration is working as expected.

So I'm not sure why your installation was using pkgsrc zlib.  Did you have
anything in PREFER_* besides PREFER_NATIVE=openssl?

Linux centosdev.ceas  bacon ~ 1072: (pkgsrc): yum list installed|grep zlib
zlib.x86_64 1.2.7-18.el7                     @anaconda
zlib-devel.x86_64 1.2.7-18.el7                     @base

Linux centosdev.ceas  bacon ~ 1073: (pkgsrc): yum list installed | grep openssl
openssl.x86_64 1:1.0.2k-19.el7                  @anaconda
openssl-devel.x86_64 1:1.0.2k-19.el7                  @base
openssl-libs.x86_64 1:1.0.2k-19.el7                  @anaconda

Linux centosdev.ceas  bacon ~/Pkgsrc/pkgsrc-test/archivers/libarchive
1074: (pkgsrc): grep PREFER ~/Pkgsrc/pkg-test/etc/mk.conf
PREFER_PKGSRC=        yes PREFER_NATIVE=        openssl

[ Install pkgsrc libxml2 ]

Linux centosdev.ceas  bacon ~/Pkgsrc/pkgsrc-test/archivers/libarchive
1075: (pkgsrc): ldd ~/Pkgsrc/pkg-test/lib/libxml2.so|grep libz
      libz.so.1 => /home/bacon/Pkgsrc/pkg-test/lib/libz.so.1
(0x00007fb48f8df000)

Linux centosdev.ceas  bacon ~/Pkgsrc/pkgsrc-test/archivers/libarchive
1076: (pkgsrc): bmake show-depends
bzip2>=1.0.3:../../archivers/bzip2
xz>=5.0.0:../../archivers/xz
zlib>=1.2.3:../../devel/zlib
openssl>=1.1.1dnb2:../../security/openssl

[ Add zlib to PREFER_NATIVE ]

Linux centosdev.ceas  bacon ~/Pkgsrc/pkgsrc-test/archivers/libarchive
1078: (pkgsrc): grep PREFER ~/Pkgsrc/pkg-test/etc/mk.conf
PREFER_PKGSRC=        yes PREFER_NATIVE=        openssl zlib

Linux centosdev.ceas  bacon ~/Pkgsrc/pkgsrc-test/archivers/libarchive
1079: (pkgsrc): bmake show-depends
bzip2>=1.0.3:../../archivers/bzip2
xz>=5.0.0:../../archivers/xz
openssl>=1.1.1dnb2:../../security/openssl


Libxml2 will pick up native libz fine when built first. The problem was www/nginx which pulled in pkgsrc libz while still linking to native openssl. After installing pkgsrc libz to satisfy nginx, libxml2 broke. Adding zlib to PREFER_NATIVE resolved the issue. I didn't have anything else in PREFER_NATIVE except for openssl
Right, the problem, put another way, is that in the absence of PREFER_PKGSRC or PREFER_NATIVE, the current run time logic will link to pkgsrc libz if it's there, but won't install it if it's not. Since libz is a dependency of something installed during bootstrap, we have to explicitly set PREFER_PKGSRC during bootstrap to make sure every dependent package is using pkgsrc libz.  One theoretical fix would be to automatically mark dependencies PREFER_NATIVE when they're implicitly chosen a link time.  I'm not sure how difficult this would be.  In any case, setting PREFER_PKGSRC=yes + PREFER_NATIVE=list during bootstrap should prevent this problem.
Home | Main Index | Thread Index | Old Index