Subject: Re: why not "make includes" before "make do-tools" for build.sh?
To: None <tech-userlevel@NetBSD.ORG>
From: Ben Harris <bjh21@netbsd.org>
List: tech-userlevel
Date: 05/17/2003 23:39:40
In article <m19Gw8E-000B40C@proven.weird.com> you write:
>[ On Thursday, May 8, 2003 at 23:20:36 (+0100), Ben Harris wrote: ]
>> Subject: Re: why not "make includes" before "make do-tools" for build.sh?
>>
>> In article <m19Doxf-000B3kC@proven.weird.com> you write:
>> > If the tools build doesn't use $DESTDIR/include then my suggestion is
>> > obviously pointless, however it seems to me that if the build of the
>> > tools does use $DESTDIR/include then "make includes" must be run before
>> > the (rest of the) tools are built.
>> 
>> It shouldn't (since $DESTDIR/usr/include might not even exist at that
>> point), but your experience suggests that under some circumstances, it does.
>> Someone should work out why, and how to fix that.
>
>Well I have not yet had a chance to repeat my experiment with <paths.h>
>(i.e. the one that prompted me to post initially), however I have just
>found an example of a host tool which definitely should be compiled
>against at least one of the target system's header files:  nbpwd_mkdb.

I think this is a subtly different problem.  There are three places we can
find header files:

(a) /usr/include: the host system's headers
(b) ${NETBSDSRCDIR}/...: the target system's uninstalled headers
(c) ${DESTDIR}/usr/include: the target system's installed headers

Your <tools.h> problem was a host tool using (c), which should never happen. 
This problem is (I think) that a host tool is using the wrong mix of (a) and
(b), and hence ending up with an inappropriate definition of UID_MAX. 
Preventing this is much harder.

>$ ls -l sys/sys/syslimits.h
>-rw-r--r--  1 woods  wheel  3937 Sep 20  2002 sys/sys/syslimits.h
>$ ls -l /home/woods/NetBSD-1.6.x-tools/bin/nbpwd_mkdb
>-r-xr-xr-x  1 woods  local  87911 May 16 14:27 /home/woods/NetBSD-1.6.x-tools/bin/nbpwd_mkdb
>$ fgrep UID_MAX /usr/include/sys/syslimits.h 
>#define UID_MAX            2147483647U  /* max value for a uid_t (2^31-2) */
>$ fgrep UID_MAX sys/sys/syslimits.h                 
>#define UID_MAX       ((~(uid_t)0)-1)   /* max value for a uid_t (2^32-1) */
[...]
>I suspect the namespace pollution issues make solving this a most
>intractable problem to solve in any generic way.  If pwd_mkdb is to
>handle this properly the only viable fix would seem to be to introduce
>variant code to explicitly handle being built as a host build tool, i.e.
>to make sure it uses the build tree's sys/sys/syslimits.h if the limit
>checks are to be preserved for this purpose.

You're right about the namespace issues.  In fact, almost everywhere that we
include a NetBSD header file in a host tool, we're risking this kind of
disaster -- we're just lucky they only bite occasionally.  The fundamental
problem is that lots of our headers expect to be part of the implementation,
so they carefully restrict themselves to the implementation's namespace. 
Obviously this is a bad thing to be doing if you suddenly find yourself
being part of an application, as some of our headers do when building host
tools.

It's all horrid.  I'm going to bed.

-- 
Ben Harris                                                   <bjh21@netbsd.org>
Portmaster, NetBSD/acorn26           <URL:http://www.netbsd.org/Ports/acorn26/>