tech-pkg archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Doubling performance of pkgsrc/mk


I got fed up with pkgsrc being slow again, so have spent a bit of time recently improving performance.

Here are some benchmarks for the build of pkgtools/url2pkg (chosen as its "build" time is very short so exposes any delays with the infrastructure):

  | OS       | SmartOS | NetBSD | macOS |
  | Baseline |   12.5s |  10.3s |  3.0s |
  | Patched  |    5.7s |   5.3s |  1.4s |

All of the builds were done on very different hardware so the OPSYS differences aren't interesting, but the patched times show a consistent 2x speedup regardless.

I'm obviously testing this work a lot as it touches a huge amount of very important code, but I'd be intested in feedback early on, as well as reports of testing from anyone who wants to try it out.

The branch is here:

I may occasionally rebase against trunk as well as squashing/improving patches, so expect to sometimes see a forced update.

As with previous performance work the bulk of this is simply thanks to reducing the number of fork+exec, using DTrace to identify areas that can be optimised. On SmartOS the baseline build invoked 2391 processes, the patched build only 877.

The good news is there's still a bunch of things that can be improved, most notably around tools creation which is still very expensive.


Jonathan Perkin   -   -
Open Source Complete Cloud

Home | Main Index | Thread Index | Old Index