Wider testing for wrapper performance patch

[Jonathan Perkin <jperkin%joyent.com@localhost>]

Hey all,

This is earlier than I'd normally ask for review of something, but it's 
a huge win and I'm interested in testing on a wide variety of systems to 
see what the performance wins are, and if there are any regressions.

One of the slowest parts of the pkgsrc infrastructure is the wrapper 
phase, where symlinks for all includes, libraries, and ancillary support 
files are created under "${WRKDIR}/.buildlink/".  Each file results in a 
fork of dirname(1) and ln(1).

When you have packages such as devel/boost-headers which is pulled in by 
many packages and alone has over 15,000 files to symlink, the terrible 
performance becomes clear.  For example:

  $ cd x11/kde-workspace4
  $ bmake patch; ptime bmake wrapper

  real     3:20.696394375
  user       50.553556463
  sys      2:23.883823896

I'm currently testing an alternative which uses a small program to batch 
up the symlink creation, and is significantly faster:

  real       19.873100598
  user        8.141441461
  sys        11.740602820

You can grab it and the patch to the infrastructure from here:

  https://github.com/TritonDataCenter/pkgsrc/commit/d119b14357b26ac282c3d5088200cc9986295b6a

It currently lives in the cwrappers package for ease of testing, but I'm 
not sure this is a good long term home, and maybe we want to create a 
new "pkg-mk-tools" package or something that can house a few other 
useful utilities to speed up other areas of the infrastructure.

Cheers,

--
Jonathan Perkin  -  Joyent, Inc.  -  www.joyent.com