Subject: pkg/5780: binpkg-list rule is 10x wasteful on some setups
To: None <gnats-bugs@gnats.netbsd.org>
From: None <bgrayson@ece.utexas.edu>
List: netbsd-bugs
Date: 07/17/1998 02:29:30
>Number:         5780
>Category:       pkg
>Synopsis:       binpkg-list rule is wasteful on some archs
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    gnats-admin (GNATS administrator)
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Fri Jul 17 00:35:00 1998
>Last-Modified:
>Originator:     Brian Grayson
>Organization:
	Parallel and Distributed Systems
	Electrical and Computer Engineering
	The University of Texas at Austin
>Release:        July 13, 1998
>Environment:
NetBSD marvin 1.3E NetBSD 1.3E (MARVIN) #40: Thu Jun  4 09:11:48 CDT 1998     bgrayson@marvin:/a/c3p0/home/c3p0/src/sys/arch/i386/compile/MARVIN i386

>Description:
	On x86, doing `make readme' in /usr/pkgsrc runs at a
	dreadful pace.  Top showed that building each README.html
	was requiring 800(!) processes.  A little investigation
	showed that the cause is the rule for binpkg-list, in
	/usr/pkgsrc/mk/bsd.pkg.mk.  Here are the relevant
	excerpts:
PACKAGES?=              ${PKGSRCDIR}/packages
...
# The following assumes that PACKAGES is set like the following:
# PACKAGES= /usr/pkgsrc/packages/`uname -r`/${MACHINE_ARCH}
# i.e. contain OS version and arch name as subdirs
.if !target(binpkg-list)
binpkg-list:
        @cd ${PACKAGES}/../.. ; \
        for i in */* ; \
        do \

	The result:  on an x86 box where PACKAGES isn't
otherwise defined, it is set to /usr/src/pkgsrc/packages (on
my system, where /usr/pkgsrc is a symlink to /usr/src/pkgsrc).  The
for i statement then looks at /usr/src/*/*, and does the -f
test on every file.  Needless to say, there are a whole lot of
subdirectories in /usr/src/*.

	(Is there a reason test/[ is not a /bin/sh builtin?)

	After removing the /../.. above, it only took around 85
processes to create a single README.html.  (I was using
archivers/lha as my test case).  I used something along the lines of:
sh -c 'echo $$';make README.html;sh -c 'echo $$', and compared the
printed pids.  Nothing else was running at the time.

	
>How-To-Repeat:
	
>Fix:
	I am not sure what the right thing to do is.  Perhaps
PACKAGES?= ${PKGSRCDIR}/packages/`uname -r`${MACHINE_ARCH} as suggested?
>Audit-Trail:
>Unformatted: