Subject: Re: Meta package for pkgsrc developer tools
To: NetBSD Packages Technical Discussion List <tech-pkg@NetBSD.org>
From: Greg A. Woods <woods@weird.com>
List: tech-pkg
Date: 08/07/2006 19:42:21
--pgp-sign-Multipart_Mon_Aug__7_19:42:18_2006-1
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

At Sun, 06 Aug 2006 02:51:13 +0200,
Joerg Sonnenberger wrote:
>=20
> On Sat, Aug 05, 2006 at 03:50:43PM -0400, Greg A. Woods wrote:
> > At Sat, 05 Aug 2006 15:18:26 +0200,
> > Joerg Sonnenberger wrote:
> > >=20
> > > On Fri, Aug 04, 2006 at 07:43:35PM -0700, John Nemeth wrote:
> > > > On Dec 25,  1:00pm, Lubomir Sedlacik wrote:
> > > > } On Sat, Aug 05, 2006 at 01:09:36AM +0900, Ryo HAYASAKA wrote:
> > > > } > I think pkgclean and pkgfind are useful tools, too.
> > > > }=20
> > > > } remind me, why do we have such an useless package as pkgclean in =
pkgsrc,
> > > > } again?
> > > > }=20
> > > > } (if you don't want clutter in your pkgsrc tree, mount it read-onl=
y and
> > > > }  set WRKOBJDIR.  and when you feel like it, just wipe WRKOBJDIR c=
lean
> > > > }  with rm.  it's even faster!)
> > > >=20
> > > >      cd /usr/pkgsrc && rm -r */*/work is pretty fast as well.
> > >=20
> > > But not really portable.
> > > 	find /usr/pkgsrc -name work -exec rm -r {} \;
> > > doesn't hit command line limits.
> >=20
> > but that's not as fast or efficient it could/should be.  Use xargs(1)!
>=20
> Ironically, xargs is not necessarily faster. The command above needs
> more execs, but has the advantage that find doesn't visit the work
> directories.

Guess all the metadata for my pkgsrc tree must be in the buffer cache
much of the time!  :-)

Sorry, I didn't actually think about the guts of the work directories
themselves since I haven't actually used pkgsrc that way for half a
decade or so.  :-) I was more worried about the just CVS subdirectories
chewing up unnecessary time and I/Os.


The sad thing is the shell glob example is about two orders of magnitude
faster (at least when you don't have that many work subdirs) than any
form or use of find.

This machine's pkgsrc tree hasn't been touched or looked at since it was
booted (it also doesn't have any work subdirs, just a few work symlinks,
but it does seem to have enough buffer cache to store the whole tree's
metadata):

19:19 [56] $ time sh -c 'echo */*/work > /dev/null'
   11.05s real     0.01s user     0.57s system
19:19 [57] $ time sh -c 'echo */*/work > /dev/null'
    0.04s real     0.00s user     0.03s system
19:20 [58] $ time sh -c 'echo */*/work > /dev/null'
    0.04s real     0.01s user     0.03s system
19:20 [59] $ time sh -c 'echo */*/work > /dev/null'
    0.04s real     0.01s user     0.02s system
19:20 [60] $ time find . -name work -print > /dev/null
   37.83s real     0.39s user     2.69s system
19:21 [61] $ time find . -name work -print > /dev/null
   37.76s real     0.37s user     2.17s system
19:22 [62] $ time find . \( -name CVS -prune \) -o \( -name work -print \) =
> /dev/null
   28.24s real     0.22s user     1.25s system
19:23 [63] $ time find . \( -name CVS -prune \) -o \( -name work -print \) =
> /dev/null
   25.98s real     0.21s user     1.10s system

Thankfully GNU Find isn't really any faster:

19:27 [70] $ time gfind . \( -name CVS -prune \) -o \( -name work -print \)=
 > /dev/null
   26.76s real     0.19s user     1.29s system
19:28 [71] $ time gfind . -name work -print > /dev/null
   37.26s real     0.39s user     2.05s system


> Using maxdepth would have a similiar result.

Hmmm.... "find: -maxdepth: unknown option"  :-)

Perhaps you're thinking about "-prune", as per above?  Or GNU Find?

--=20
						Greg A. Woods

H:+1 416 218-0098 W:+1 416 489-5852 x122 VE3TCP RoboHack <woods@robohack.ca>
Planix, Inc. <woods@planix.com>       Secrets of the Weird <woods@weird.com>

--pgp-sign-Multipart_Mon_Aug__7_19:42:18_2006-1
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 5.0i for non-commercial use
MessageID: zRbIrrzxL39oAx0YT2gEEdoIIPOTHlRa

iQA/AwUBRNfP3GJ7XxTCWceFEQKg6wCfVTpwBiQmDLXc3lZz+UWxwsQzGkkAn39W
9fEjVbjkZdIUp78my31lHw5P
=HtZ1
-----END PGP SIGNATURE-----

--pgp-sign-Multipart_Mon_Aug__7_19:42:18_2006-1--