Subject: Re: [RFC] framework for platform tools
To: Roland Illig <roland.illig@gmx.de>
From: Johnny Lam <jlam@NetBSD.org>
List: tech-pkg
Date: 04/07/2005 11:14:06
Roland Illig wrote:
> 
> Pros:
> * only the "non-standard" tools need to be mentioned
>   in the ${OPSYS}.mk file.
> * new tools (like diff(1), cvs(1)) can be added without
>   touching {NUMBER_OF_PLATFORM} different files.
> * There's a definite, single list for the list of platform tools
>   (lines 26..46)
> * Each tool has a ${TOOL_FLAGS} variable, which makes the tools
>   consistent. To fully exploit this, the ${TOOL_FLAGS} must be used by
>   the rest of the pkgsrc infrastructure, but it is already done
>   partly.

These are all good reasons, especially the maintenance part.  Plus, we 
simplify porting pkgsrc, which is quite good.  I'm not sure that the 
TOOL_FLAGS variable can be consistently used because it won't be used 
correctly by configure scripts, etc.  I think maybe it'd be better to 
define a TOOL_CMD and TOOL_FLAGS variable and then also a TOOL variable 
that looks like:

	TOOL?=	${TOOL_CMD} ${TOOL_FLAGS}

Then doing:

	CONFIGURE_ENV+=	AWK=${AWK:Q}

will work correctly.  This allows the expected usage of ${AWK} to invoke 
"awk" but gives the user (by "user" I mean the package maintainer) some 
flexibility if he just wants to invoke the tool without the flags.  I 
know there are a few anamolous uses of things like ${GZIP} or ${FILE} 
for some other purpose, but I want us to sweep pkgsrc and get rid of 
those alternative uses (I think they're carryovers from /usr/share/mk).

> Cons:
> * heavier file system usage (about 120 x stat(2))
> 
> Notes:
> * I could not measure any timing differences between the old and the
>   new version, so perhaps the Con argument is bogus.

We stat(2) an enormous number of times in pkgsrc.  I'm not really 
concerned about this, though if we need to optimize it somehow, I'd 
cache the variable values in some file that is checked and included by 
bsd.pkg.mk so that we don't get penalized by doing the same checks again 
for recursively-invoked make processes.  This is probably alleviated a 
great deal in Todd's tv-derecurse branch, so I think we shouldn't go out 
of our way to worry about it too much.

One problem with your implementation is that ":tu" isn't a variable 
modifier that is supported by the bootstrap bmake.  Until we've fixed 
that, and also create some way for pkgsrc to check that the ${MAKE} 
invoked is of the correct vintage, we can't use these extra modifiers. 
I've usually worked around this some sort of extra table lookup to do 
the name-mapping, so something like the following (example code!):

_PLATFORM_TOOLS=	awk basename ... file gzip ...
_PLATFORM_EXECDIRS?=	/bin /sbin /usr/sbin ...

# Map tool name to the variable name (yes, this is going to be verbose).
_PLATFORM_TOOLNAME.awk=		AWK
_PLATFORM_TOOLNAME.basename=	BASENAME
...
_PLATFORM_TOOLNAME.file=	FILE
_PLATFORM_TOOLNAME.gzip=	GZIP
...

# Search the ${_PLATFORM_EXECDIRS} for each tool and create a TOOL_CMD
# variable holding that path, and a TOOL_FLAGS variable to hold optional
# flags for that tool.  Also define a TOOL variable that holds the
# combined values
#
.for _t_ in ${_PLATFORM_TOOLS}
.  for _d_ in ${_PLATFORM_EXECDIRS}
.    if exists(${_d_}/${_t_})
${_PLATFORM_TOOLNAME.${_t_}}_CMD?=	${_d_}/${_t_}
${_PLATFORM_TOOLNAME.${_t_}}_FLAGS?=	# empty
${_PLATFORM_TOOLNAME.${_t_}}?= \
	${${_PLATFORM_TOOLNAME.${_t_}}_CMD} \
	${${_PLATFORM_TOOLNAME.${_t_}}_FLAGS}
.    endif
.  endfor
.endfor

	Cheers,

	-- Johnny Lam <jlam@NetBSD.org>