[ On Saturday, May 17, 2003 at 23:39:40 (+0100), Ben Harris wrote: ]
> Subject: Re: why not "make includes" before "make do-tools" for build.sh?
>
> In article <m19Gw8E-000B40C@proven.weird.com> you write:
> >[ On Thursday, May 8, 2003 at 23:20:36 (+0100), Ben Harris wrote: ]
> >> Subject: Re: why not "make includes" before "make do-tools" for build.sh?
> >>
> >> In article <m19Doxf-000B3kC@proven.weird.com> you write:
> >> > If the tools build doesn't use $DESTDIR/include then my suggestion is
> >> > obviously pointless, however it seems to me that if the build of the
> >> > tools does use $DESTDIR/include then "make includes" must be run before
> >> > the (rest of the) tools are built.
> >> 
> >> It shouldn't (since $DESTDIR/usr/include might not even exist at that
> >> point), but your experience suggests that under some circumstances, it does.
> >> Someone should work out why, and how to fix that.
> >
> >Well I have not yet had a chance to repeat my experiment with <paths.h>
> >(i.e. the one that prompted me to post initially), however I have just
> >found an example of a host tool which definitely should be compiled
> >against at least one of the target system's header files:  nbpwd_mkdb.
> 
> I think this is a subtly different problem.

It may be, though I thought the problem I was seeing with <paths.h> was
also during the tools build, but as I may have mentioned I won't be sure
until I try to repeat it as an experiment.

>  There are three places we can
> find header files:
> 
> (a) /usr/include: the host system's headers
> (b) ${NETBSDSRCDIR}/...: the target system's uninstalled headers
> (c) ${DESTDIR}/usr/include: the target system's installed headers
> 
> Your <tools.h> problem was a host tool using (c), which should never happen. 

<paths.h>, but yes, that's what I thought the problem was then....

> This problem is (I think) that a host tool is using the wrong mix of (a) and
> (b), and hence ending up with an inappropriate definition of UID_MAX. 

Yup, that's this new problem for certain.....

> You're right about the namespace issues.  In fact, almost everywhere that we
> include a NetBSD header file in a host tool, we're risking this kind of
> disaster -- we're just lucky they only bite occasionally.  The fundamental
> problem is that lots of our headers expect to be part of the implementation,
> so they carefully restrict themselves to the implementation's namespace. 
> Obviously this is a bad thing to be doing if you suddenly find yourself
> being part of an application, as some of our headers do when building host
> tools.

Building cross-build tools is indeed "Hard(tm)" to do 100% correctly,
even without trying to be clever at using the headers from the source
tree to provide definitions corresponding to the target system
environment where needed.

> It's all horrid.  I'm going to bed.

The really horrid part is the hack I used to fix this latest problem.  :-)

A more reliable hack, and one that could be used to solve all similar
problems in a generic manner, would be to have "compat_pwd.h" provide
definitions for _TARGET_UID_MAX, _TARGET_GID_MAX, etc. or something like
those, that are carefully kept in sync with the source tree's
definitions, and then to use these special target-specific defines
carefully and deliberately in pw_scan.c when building for the
cross-build host tools.

Index: lib/libc/gen/pw_scan.c
===================================================================
RCS file: /cvs/master/m-NetBSD/main/src/lib/libc/gen/pw_scan.c,v
retrieving revision 1.11
diff -c -r1.11 pw_scan.c
*** lib/libc/gen/pw_scan.c	29 Jan 2002 10:20:30 -0000	1.11
--- lib/libc/gen/pw_scan.c	18 May 2003 01:28:23 -0000
***************
*** 34,42 ****
   */
  
  #if HAVE_CONFIG_H
! #include "config.h"
! #include "compat_pwd.h"
! #else
  #include <sys/cdefs.h>
  #if defined(LIBC_SCCS) && !defined(lint)
  __RCSID("$NetBSD: pw_scan.c,v 1.11 2002/01/29 10:20:30 tv Exp $");
--- 34,58 ----
   */
  
  #if HAVE_CONFIG_H
! 
! /*
!  * XXX including the source tree's sys/syslimits.h is a very bad hack that will
!  * fail on non-*BSD systems where config.h will then include <limits.h> and/or
!  * <sys/syslimits.h> from the host system and those includes won't be protected
!  * from multiple inclusion, causing clashes between host definitions and those
!  * from the source tree's sys/syslimits.h.  It works on *BSD because the host's
!  * <sys/syslimits.h> uses the very same multiple inclusion protection trick
!  * that the source tree's variant does and so this allows our source tree's
!  * definitions to take precedence.  For SunOS-5 it may suffice to pre-define
!  * _LIMITS_H though... (but I haven't tested for lurking clashing/misssing defs)
!  */
! # include "../../sys/sys/syslimits.h"
! # define _LIMITS_H
! # include "config.h"
! # include "compat_pwd.h"
! 
! #else  /* !HAVE_CONFIG_H */
! 
  #include <sys/cdefs.h>
  #if defined(LIBC_SCCS) && !defined(lint)
  __RCSID("$NetBSD: pw_scan.c,v 1.11 2002/01/29 10:20:30 tv Exp $");


The rest of these changes are what I use to allow full use of the 32-bit
integer range for user and group IDs (along with the <sys/syslimits.h>
changes as well of course):

***************
*** 71,81 ****
  	struct passwd *pw;
  	int *flags;
  {
! 	unsigned long id;
  	int root, inflags;
  	char *ep;
  	const char *p, *sh;
  
  	_DIAGASSERT(bp != NULL);
  	_DIAGASSERT(pw != NULL);
  
--- 87,103 ----
  	struct passwd *pw;
  	int *flags;
  {
! 	u_quad_t id;
  	int root, inflags;
  	char *ep;
  	const char *p, *sh;
  
+ #if 0
+ 	assert(sizeof(u_quad_t) == sizeof(unsigned long long int));
+ 	assert(UID_MAX < ULLONG_MAX);
+ 	assert(GID_MAX < ULLONG_MAX);
+ #endif
+ 
  	_DIAGASSERT(bp != NULL);
  	_DIAGASSERT(pw != NULL);
  
***************
*** 94,100 ****
  
  	if (!(p = strsep(&bp, ":")))			/* uid */
  		goto fmt;
! 	id = strtoul(p, &ep, 10);
  	if (root && id) {
  		if (!(inflags & _PASSWORD_NOWARN))
  			warnx("root uid should be 0");
--- 116,122 ----
  
  	if (!(p = strsep(&bp, ":")))			/* uid */
  		goto fmt;
! 	id = strtoull(p, &ep, 10);
  	if (root && id) {
  		if (!(inflags & _PASSWORD_NOWARN))
  			warnx("root uid should be 0");
***************
*** 111,117 ****
  
  	if (!(p = strsep(&bp, ":")))			/* gid */
  		goto fmt;
! 	id = strtoul(p, &ep, 10);
  	if (id > GID_MAX || *ep != '\0') {
  		if (!(inflags & _PASSWORD_NOWARN))
  			warnx("invalid gid '%s'", p);
--- 133,139 ----
  
  	if (!(p = strsep(&bp, ":")))			/* gid */
  		goto fmt;
! 	id = strtoull(p, &ep, 10);
  	if (id > GID_MAX || *ep != '\0') {
  		if (!(inflags & _PASSWORD_NOWARN))
  			warnx("invalid gid '%s'", p);


-- 
								Greg A. Woods

+1 416 218-0098;            <g.a.woods@ieee.org>;           <woods@robohack.ca>
Planix, Inc. <woods@planix.com>; VE3TCP; Secrets of the Weird <woods@weird.com>