[ On Friday, October 22, 1999 at 22:40:05 (-0500), Frederick Bruckman wrote: ]
> Subject: Re: dilema with use of LDFLAGS=-L${LOCALBASE}/lib vs. private application libraries
>
> Alright, that's correct, but it's irrelevant. GNU configure packages
> don't usually pick up LDFLAGS from the environment at compile time;
> configure, instead, sets LDFLAGS in the generated Makefile at
> configure time.

Yes, but I'm not talking only about packages using GNU Autoconf.  If all
packages used GNU Autoconf then I could influence the GNU Autoconf
development to make the job of pkgsrc even easier than it already does,
particularly effectively too if pkgrsc itself provided local hacks to
the Autoconf it used for building packages!

> > > If configure is picking up the wrong library entirely, and not simply
> > > the wrong version, it's a bug.
> > 
> > How's configure to know?  It's not a 100% solvable problem.  Configure
> > can not be expected to test every interface in the library
> 
> What are you talking about? Why would you have to test every interface
> in the library? Just rename the library, or re-order the arguments in
> Makefile.in.

There are actually two very closely related problems, if my guess about
windowmaker and other similar packages is correct.

The first problem is that there will always be another package that
happens to reuse the same name for an internal library -- i.e. a name
that clashes with the name of another library that just happens to have
been installed in the build environment but which of course has a
different API.  This is not normally a problem, except when bsd.pkg.mk
imposes its additional flags on the new package being built.

The second problem is that case that I'm guessing about for windowmaker.
In these cases the package source tree is designed to build in such a
way that it very carefully ignores any of its own predecessor libraries
that may have been installed in the build environment (and in these
specific cases it does so by carefully placing '-L' arguments in exactly
the right places on the command line).  However bsd.pkg.mk comes along
and throws them a curve by telling them that they should look *first* in
$PREFIX/lib where they may accidentally find ancient and incompatible
versions of their predecessor's libraries.

> > Note that it's probably safe to comletely remove the use of LDFLAGS in
> > setting CONFIGURE_ENV, though not from MAKE_ENV for the sake of X11
> > packages, suggesting the following more complete change:
> > 
> > - LDFLAGS+=		-Wl,-R${LOCALBASE}/lib -L${LOCALBASE}/lib
> >   MAKE_ENV+=		LDFLAGS="${LDFLAGS}"
> > - CONFIGURE_ENV+=		LDFLAGS="${LDFLAGS}"
> 
> No thank you! If we revert to the old behaviour to make a couple
> packages work, are you going to supply fixes for the _hundreds_ that
> would be broken? Keep in mind that most make setups don't supply
> '-Wl,-R' arguments unless they've been libtoolized, so you would have
> to provide a custom solution for every package that uses shared
> libraries in order for it to work on elf ports.

Perhaps you don't seem to understand the problem?  I'm not talking about
thw '-Wl-R' argument -- it is only secondary to the issue.  The real
problem is the *order* in which "LDFLAGS+=" necessarily puts the (in
many cases completely unnecessary) "-L" argument.

If there were a common convention of PRE_LDFLAGS and POST_LDFLAGS which
would allow '-L' options to be appended to the entire compile command
line then this wouldn't be an issue, but such a convention does not
exist (though it partly does in the GNU Makefile Standards world if you
bend some of their rules a wee bit).

> More generally, you're going about it all wrong. You should fix the
> thing that's broken, and then, if you find yourself repeating the same
> fix in many different places, you generalize. 

I'm trying to fix a problem that *will* happen again and again and again
and which is completely and totally unpredictable (i.e. if you treat
packages as black boxes).  I'm proposing to fix it in a way that I claim
will require the least amount of effort on behalf of those involved, and
which will only require effort on behalf of those who are most likely
willing and able to provide it.

I've not done a scientific servey yet, but my off-the-cuff examination
of the packages currently in the pkgsrc collection, combined with my
very extensive 10-years of experience at porting code to Unix
environments, I feel quite certain that my proposal will do what I claim
it will.  I have indeed already begun proving my case within my own
build environment an so far I can report only success (though I'm only
about 1% through so far! ;-)

Remember that pkgsrc can only influence the actual build system of a
package by patching its makefiles.  Although we could have total control
by completely replacing the build system of every package with one of
our own design, this is not practical.  However without doing so there's
no way to control how a package will use a value of LDFLAGS (in either
the configuration or build stages).  Unfortunately the normal way is to
put it right after "$(CC)" on the compile command line and this is where
the problem lies.

> That's how the LDFLAGS
> evolved into it's present form. ISTR the '-Wl,-R' for elf ports came
> first, but then it was a natural to mirror the behavior for aout, as
> it permitted removing special handling in many, many packages.

That's very fine, and I agree, but it MUST NOT be done in the way that
it is being done with LDFLAGS in particular because that causes
unexpected things to happen in upredictable places and times (eg. the
sudden appearance of build problems with a new version of windowmaker).

> There's an even better reason for setting LDFLAGS in configure's
> environment. Most of the configure scripts search for libraries in
> conventional places, "conventional" meaning, "conventional on other
> systems".

Yes, but the number of packages that require "third-party" libraries is
not 100% of them.  For those that do not the search into $PREFIX/lib can
cause very unexpected and unpredicatble failures.

If packages do require third-party components then those components can
be identified very very easily at the time they are incorporated into
pkgsrc and thus can be dealt with properly.

Indeed if they do then they *MUST* be identified in order that build and
binary package dependencies can be dealt with in an explicit manner such
that binary packages do indeed stand a chance of working when external
shared libraries come into play (though shared libraries are only the
most visible form of such external component dependencies).

I *NEVER* want anything built from pkgsrc to just go out and find the
things it needs unless those things are 1000% guaranteed to be a part of
the base OS.  Period.  I hope that all users of binary packages can
agree with me on that.  The entire prupose of using pkgsrc to build
binary packages requires that pkgsrc foil the packages internal build
system into thinking that it found the components it needs while at the
same time knowing exactly what components the package will need and
exactly where it should find them.  I have built entire production
systems of multiple servers based on NetBSD and pkgsrc binary packages
and I can assure you that this is a most important critical requirement,
especially with shared libraries but also with any kind of externally
required component.

Whether those packages that need external components are foiled into
finding the right components by influencing their configuration tools,
or whether patches are applied to their build control files, is
irrelevant just so long as one of those things is done in a fashion that
ensures the pkgsrc maintainer understands the actions and requirements
of that package.

Finally please note that for the explicit purposes of managing packages
built using GNU Autoconf it is far more desirable to influence the
package using "--with-lib=/path/to/lib" where possible than with
"LDFLAGS=-L/path/to/lib".  Using the former generally causes an explicit
path to be used to the library while the latter may allow the package to
find and use unexpected libraries (as I've been trying to demonstrate!).

Perhaps as one final point in favour of my argument I might point out
that the (somewhat more advanced in some repects) FreeBSD ports system
does not ever set LDFLAGS.  In fact the only case where it introduces
'-L' to the package's build system is to handl the rather messy case of
HAVE_MOTIF and MOTIFLIB itself is explicitly placed in MAKE_ENV -- it is
not used with the bsd.port.mk to set any other environment variables
that the package's build system might use.

So, now that you might be able to guess at the true depth to which I've
explored this problem, and now that you might have a better
understanding of the generic nature of it, do you have any different
suggestions?

-- 
							Greg A. Woods

+1 416 218-0098      VE3TCP      <gwoods@acm.org>      <robohack!woods>
Planix, Inc. <woods@planix.com>; Secrets of the Weird <woods@weird.com>