Improving 'bmake wrapper' speed

[Jonathan Perkin <jperkin%joyent.com@localhost>]

I recently got fed up with how long the 'wrapper' phase was taking,
especially on packages with a large number of dependencies, so decided
to take a look.

Breaking out DTrace on an otherwise idle system running just the
'wrapper' phase ('patch' had already been done) showed that we are
running a huge number of exec'd processes:

  # dtrace -n 'syscall::exece:return { @num[execname] = count(); }'
    [ full output snipped for brevity ]
    grep                                                             94
    sort                                                            164
    nbsed                                                           241
    mkdir                                                           399
    bash                                                            912
    cat                                                            3893
    ln                                                             7631
    rm                                                             7766
    dirname                                                        7769

Tracking these down, the 'cat's are primarily from
mk/wrapper/gen-transform.sh, and the ln/rm/dirname are from each
symlink done by mk/buildlink3/bsd.buildlink3.mk.

I have a proposed patch which significantly reduces the number of
execs we need to perform:

  http://us-east.manta.joyent.com/jperkin/public/patches/wrapper-perf.diff

The patch reduces the runtime for 'bmake wrapper' in net/kppp on a
SmartOS zone from this:

  real     2:26.094442113
  user       32.463077360
  sys      1:48.647178135

to this:

  real       49.648642097
  user       14.952946135
  sys        33.989975053

with DTrace confirming the change:

  # dtrace -n 'syscall::exece:return { @num[execname] = count(); }'
    [ full output snipped for brevity ]
    grep                                                             94
    cat                                                             106
    sort                                                            164
    nbsed                                                           241
    mkdir                                                           399
    bash                                                            912
    ln                                                             7631

On a Linux system it reduces the runtime from this:

  real    1m3.939s
  user    0m17.437s
  sys     0m43.280s

to this:

  real    0m25.436s
  user    0m8.816s
  sys     0m15.406s

So between a 2.5x and 3x speedup.

I have verified that the WRKDIR is identical after this change, but
will run it through a full bulk build anyway.

Comments?  There are a lot of other 'useless use of cat' cleanups that
can be done, but they do not have such a significant performance
impact.

Regards,

-- 
Jonathan Perkin  -  Joyent, Inc.  -  www.joyent.com