Subject: Re: benchmark
To: Dries Schellekens <email@example.com>
From: Igor Shmukler <firstname.lastname@example.org>
Date: 01/25/2005 19:31:16
If you read this thread of the one from FreeBSD performance carefully, you will notice following.
Nobody EVER argued with the paper or its content.
Actually, as far as I remember original article on OSNews had a normal title too - NetBSD scales
better (or something like this).
So to answer your questions - YES.
BTW, there are problems with the benchmark suite. We used for an internal project but after
changing some of the benchmarks, otherwise results MAY not be correct.
I contacted author to provide details (back when this was originally published), but he did not
follow up. Perhaps spam filter swollowed my email.
The acticle is NOT a rant at all. It simply an analisys of a scalability test-suite.
>> About flaming. I am not a fan of publically humiliating anyone.
>> It's a wrong thing to do. In fact when the person who started this
>> sent an email to FreeBSD performance list I merely pointed out that
>> microbenmark != performance. It does not mean that NetBSD performs
>> worse than FreeBSD. In fact it is possible that NetBSD is better,
>> although I want to see a proof before such a statement could be made.
> Did you read the conclusions of the paper?
>> Microbenchmarks are not always the best indicators to make judgments on
> the overall performance of one operating system over another. However,
> they are useful to infer an understanding of the architectural decisions
> that go into building an operating system. For many applications, the
> results presented in the paper may never affect performance. For others,
> the scalability of the operating system may simply not permit the
> application to run suitably."
> > Although NetBSD 2.0 has outperformed FreeBSD 5.3 in most of the
> benchmarks presented here, FreeBSD 5.3 has made significant developments
> with its symmetric multiprocessor (SMP) architecture, particularly in
> the area of scalability with fine-grained locking. NetBSD 2.0 continues
> to use a single lock to serialize access to kernel mode. Additionally,
> the performance of the thread implementation on multiprocessor systems,
> where thread concurrency can be achieved, would be worth investigating.
> Benchmarks for these areas are the objective of future research."