Subject: Re: Port benchmarks
To: None <netbsd-ports@netbsd.org, port-alpha@netbsd.org>
From: Simon Burge <simonb@netbsd.org>
List: netbsd-ports
Date: 04/01/2000 00:29:22
Manuel Bouyer wrote:

On Mon, Mar 27, 2000 at 02:43:43PM -0800, R. C. Dowdeswell wrote:
> [ instructions for running lmbench ]

Just to make it easier for a comparison, I've mixed in Manuel's results
with my OSF results - NetBSD first, OSF second.

                 L M B E N C H  1 . 9   S U M M A R Y
                 ------------------------------------
                 (Alpha software, do not distribute)

Processor, Processes - times in microseconds - smaller is better
----------------------------------------------------------------
Host                 OS  Mhz null null      open selct sig  sig  fork exec sh  
                             call  I/O stat clos       inst hndl proc proc proc
--------- ------------- ---- ---- ---- ---- ---- ----- ---- ---- ---- ---- ----
ds20        NetBSD 1.4U  498  0.6  2.5   12   16 0.05K  1.2    4 0.7K   3K   5K
alpha-dec     OSF1 T5.0  498  0.5  1.0   76   84 0.13K  0.9    3 0.8K   3K   6K

Context switching - times in microseconds - smaller is better
-------------------------------------------------------------
Host                 OS 2p/0K 2p/16K 2p/64K 8p/16K 8p/64K 16p/16K 16p/64K
                        ctxsw  ctxsw  ctxsw ctxsw  ctxsw   ctxsw   ctxsw
--------- ------------- ----- ------ ------ ------ ------ ------- -------
ds20        NetBSD 1.4U    3     10     48    18     56      22      61
alpha-dec     OSF1 T5.0    3      8     41    16     42      17      45

*Local* Communication latencies in microseconds - smaller is better
-------------------------------------------------------------------
Host                 OS 2p/0K  Pipe AF     UDP  RPC/   TCP  RPC/ TCP
                        ctxsw       UNIX         UDP         TCP conn
--------- ------------- ----- ----- ---- ----- ----- ----- ----- ----
ds20        NetBSD 1.4U     3    16   15    48          49        938
alpha-dec     OSF1 T5.0     3    20   24    46          48

File & VM system latencies in microseconds - smaller is better
--------------------------------------------------------------
Host                 OS   0K File      10K File      Mmap    Prot    Page       
                        Create Delete Create Delete  Latency Fault   Fault 
--------- ------------- ------ ------ ------ ------  ------- -----   ----- 
ds20        NetBSD 1.4U   1408    854   1515   2325  1785192          0.1K
alpha-dec     OSF1 T5.0     63    120    276    182     1348     2    0.0K

*Local* Communication bandwidths in MB/s - bigger is better
-----------------------------------------------------------
Host                OS  Pipe AF    TCP  File   Mmap  Bcopy  Bcopy  Mem   Mem
                             UNIX      reread reread (libc) (hand) read write
--------- ------------- ---- ---- ---- ------ ------ ------ ------ ---- -----
ds20        NetBSD 1.4U   41   45   49    190    406    311    215  406   372
alpha-dec     OSF1 T5.0  411  366   -1    438   1158    527    293 1153   486

Memory latencies in nanoseconds - smaller is better
    (WARNING - may not be correct, check graphs)
---------------------------------------------------
Host                 OS   Mhz  L1 $   L2 $    Main mem    Guesses
--------- -------------   ---  ----   ----    --------    -------
ds20        NetBSD 1.4U   498     5     30         170
alpha-dec     OSF1 T5.0   498     5     30         184


> The '*Local* Communication bandwidths' numbers don't looks rigth compared
> to Simon's results for a ds20/OSF1. I can't believe NetBSD is that bad :)

The other huge discrepancy is the mmap latency, and things seem to
have gotten worse lately in general.  On an 500MHz AlphaPC164, it got
154663us on 1.4.1 and 682128us on 1.4J.  A DS20 should be better than
that you would think.  Similarish results on a pmax - with a 40MHz
R3000, the result was 49203us on 1.4.1 and 241135us on 1.4X.

Simon.