Subject: Re: RiscBSD speed
To: Peter Teichmann <sol@Space.WH1.TU-Dresden.De>
From: Chris G. Demetriou <cgd@pa.dec.com>
List: port-arm32
Date: 06/17/1997 23:56:56
> some days ago I compiled lmbench for RiscBSD and got the following results.
> I added some other computers for comparison: 
> 
>                     L M B E N C H  1 . 0   S U M M A R Y

Be careful of microbenchmarks.  (i'm tempted to say "especially
lmbench" because of previous discussions on NetBSD lists about it, but
it's not really clear to me that it's better or worse than others in
many ways.  8-)


For the software tests, especially for inter-system comparisons,
lmbench is (and other benchmarks are) not particularly useful unless
you understand the innards of the systems involved, because the code
paths they are comparing may be very different.  Inter-system
comparisons between vastly different systems, e.g. NetBSD and Linux or
(probably) one of the commercial systems aren't particularly
meaningful, unless you're familiar with the code paths in the systems
that are causing the numbers to be generated.  Intra-system/port
comparisons, e.g. one type of NetBSD/arm32 box vs another are very
meaningful.  Intra-system comparisons (with different ports),
e.g. comparisons of NetBSD/arm32 and NetBSD/i386 are still meaningful
but again require a bit more careful interpretation.  Comparisons of
ports of similar OSes (e.g. NetBSD and FreeBSD, both on the i386) are
also meaningful with a bit of interpretation.  In other words, you can
get useful information out of the software benchmarks, but you _do_
have to be careful with your interpretation, and apply knowledge of
the systems you're comparing.

I think that the hardware characterization tests, e.g. memory
bandwidth, which don't really rely on the underlying operating
systems, are LMbench's strong point.  They're pretty good at stating
clearly what they claim to be measuring, don't need much
interpretation, and reveal interesting hardware characteristics.





chris