pkg/50317: Determining SDK under Darwin is problematic

[mmayer%mmayer.net@localhost]

>Number:         50317
>Category:       pkg
>Synopsis:       Determining SDK under Darwin is problematic
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    pkg-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Oct 08 21:10:00 +0000 2015
>Originator:     Markus Mayer
>Release:        pkgsrc tip as of Oct 8, 2015
>Organization:
>Environment:
Darwin edmonds.xxxx 14.5.0 Darwin Kernel Version 14.5.0: Wed Jul 29 02:26:53 PDT 2015; root:xnu-2782.40.9~1/RELEASE_X86_64 x86_64

>Description:
The problem is in mk/platform/Darwin.mk and how it determines the value for OSX_SDK_PATH.

All is well as long as Xcode and OS X "match", meaning the OS X version installed on the machine is the same that Xcode targets as its main/primary target. (It looks like Apple is now only shipping 1 OS X SDK with Xcode. I think they used to ship newest plus previous version in the past.)

In my case, I had Xcode 7 (which was automatically updated), but was still running OS X 10.10 (which needs manual intervention to upgrade). In preparation for said upgrade, I started re-building pkgsrc, so it'll be installed in /opt/pkg rather than /usr/pkg, to be ready for the write-protected /usr directory in El Capitan.

The problem I ran into was that it couldn't determine the SDK, which resulted in the somewhat misleading error:

$ bmake
ERROR: This package has set PKG_FAIL_REASON:
ERROR: libtool-base-2.4.2nb10 requires a working dlopen().
*** Error code 1

Stop.
bmake: stopped in /Users/markus/pkgsrc/devel/libtool-base

And hence, pretty much nothing would build. The reason for the error is that it can't find dlfcn.h, because the OS_SDK_PATH is incorrectly set (i.e. empty).

What is happening is the following (all lines from mk/platform/Darwin.mk):

OSX_VERS!=      sw_vers -productVersion

This is set to 10.10.5, since that's what the machine is running.

Next, it tries this:

OSX_SDK_PATH!= /usr/bin/xcrun --sdk macosx${OSX_VERS} --show-sdk-path 2>/dev/null || echo /nonexistent

That returns an error, since there's no SDK for 10.10.5 on my machine. All it has (due to the automatic Xcode 7 update) is the SDK for 10.11.

Running the command manually proves this:

$ /usr/bin/xcrun --sdk macosx10.10.5 --show-sdk-path
xcodebuild: error: SDK "macosx10.10.5" cannot be located.
xcodebuild: error: SDK "macosx10.10.5" cannot be located.
xcrun: error: unable to lookup item 'Path' in SDK 'macosx10.10.5'
$ /usr/bin/xcrun --sdk macosx10.11 --show-sdk-path
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk

However, more importantly, this also works:

$ /usr/bin/xcrun --sdk macosx --show-sdk-path
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk

If one does NOT specify the OS X version when querying for the SDK, it'll return the right one!


>How-To-Repeat:
Take any Mac running a newer Xcode (e.g. Xcode 7) and an older OS X (e.g. OS X 10.10) such that the SDK provided under /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/ does NOT match the version of the OS that is running.

pkgsrc will be unable to find the proper SDK and pretty much any and all attempts to build anything (beyond bootstrapping) will fail.
>Fix:
I can't verify that it'll do the right thing for all older OS X versions. From what I've found, it would seem the following change might be sensible:

-OSX_SDK_PATH!= /usr/bin/xcrun --sdk macosx${OSX_VERS} --show-sdk-path 2>/dev/null || echo /nonexistent
+OSX_SDK_PATH!= /usr/bin/xcrun --sdk macosx --show-sdk-path 2>/dev/null || echo /nonexistent

If this doesn't quite work due to older OS X releases, it would be good to check the return value of "/usr/bin/xcrun --sdk macosx${OSX_VERS} ..." and run "/usr/bin/xcrun --sdk macosx ..." if the former failed. I am pretty sure doing this will save others quite a bit of time, as this problem isn't particularly easy to find.