Re: Build time measurements

[Andrew Doran <ad%netbsd.org@localhost>]

Hi Andreas.

On Mon, Mar 23, 2020 at 04:11:17PM +0200, Andreas Gustafsson wrote:

> In September and November, I reported some measurements of the amount
> of system time it takes to build a NetBSD-8/amd64 release on different
> versions of -current/amd64.  I have now repeated the measurements with
> a couple of newer versions of -current on the same hardware, and here
> are the results.  The left column is the source date of the -current
> system hosting the build.
> 
>   HP ProLiant DL360 G7, 2 x Xeon L5630, 8 cores, 32 GB, build.sh -j 8
> 
>   2016.09.06.06.27.17    3930.86 real     15737.04 user      4245.26 sys
>   2019.10.18.17.16.50    4461.47 real     16687.37 user      9344.68 sys
>   2020.03.17.22.03.41    4723.81 real     16646.42 user      8928.72 sys
>   2020.03.22.19.56.07    4595.95 real     16592.80 user      8171.56 sys
> 
> I also measured the same versions on a newer machine with more cores:
> 
>   Dell PowerEdge 630, 2 x Xeon E5-2678 v3, 24 cores, 32 GB, build.sh -j 24
> 
>   2016.09.06.06.27.17    3321.55 real      9853.49 user      5156.92 sys
>   2019.10.18.17.16.50    3767.63 real     10376.15 user     16100.99 sys
>   2020.03.17.22.03.41    2910.76 real      9696.10 user     18367.58 sys
>   2020.03.22.19.56.07    2711.14 real      9729.10 user     12068.90 sys

Thank you for doing this, and for bisecting the performance losses over
time (I fixed the vnode regression you found BTW).

There are two options enabled in -current that spoil performance on multi
processor machines: DIAGNOSTIC and acpicpu.  I'm guessing that you had both
enabled during your test runs.  We ship releases without DIAGNOSTIC, and
acpicpu really needs to be fixed.

I did some "build.sh release" runs on a machine vaguely similar in spec to
you second one, a ThinkStation D30 with 2x Xeon E5-2696 v2.

DIAGNOSTIC and acpicpu are disabled in all kernels but they are otherwise
GENERIC.  The 2020-04-??  kernel is HEAD plus the remaining changes from the
ad-namecache branch.

A Linux result is included too for a reference point.  I would have tried
FreeBSD as well, but don't have it installed on this machine yet.

Andrew


HT disabled in BIOS so -j24:

2019-10-23              2445.46 real     17297.90 user     16725.01 sys
2020-01-16              2013.58 real     16772.79 user      7801.39 sys
2020-03-23              1850.98 real     16383.89 user      4777.89 sys
2020-04-??              1791.62 real     16367.51 user      3662.62 sys
Linux 5.4.19            1688.29 real     15682.22 user      1962.81 sys

HT enabled so -j48.  With percentage real time reduction thanks to HT:

2019-10-23      -5%     2583.56 real     24459.69 user     45719.70 sys
2020-01-16      0%      2023.79 real     24495.20 user     20431.37 sys
2020-03-23      5%      1765.39 real     24348.67 user      8856.36 sys
2020-04-??      7%      1672.71 real     24770.28 user      5555.92 sys
Linux 5.4.19    3%      1644.41 real     24425.07 user      2926.18 sys