Subject: Re: benchmark
To: None <tech-perform@netbsd.org>
From: Dmitri Nikulin <setagllib@optusnet.com.au>
List: tech-perform
Date: 02/09/2005 17:00:19
The apology would be because you accused Hubert of lying, which he 
simply could not even possibly have done from his role as a messenger. 
Do you call your browser a liar if you read something on Slashdot you 
don't like?

The paper was done very professionally. It gave clean facts with 
supporting visuals, left no ambiguity with regards to environment, and 
disclaimed its own validity as a measure of real-world performance. 
Hubert's exact wording of "NetBSD... beats FreeBSD" may seem like a 
hasty conclusion, but it's hard to argue that the benchmarks gave a lot 
to suppor that (say what you want, but on a UP i386 machine, NetBSD will 
run loops around FreeBSD 5.3, and non-i386 is hardly worth comparing - 
devs themselves admit FreeBSD 5 has lost a lot of performance ground in 
micro and macro).

Even if Hubert had done the benchmarks himself, even written them 
himself, and then drew conclusions on his own, his establishment as part 
of the NetBSD project (including his often overlooked but very useful 
blog, http://www.feyrer.de/NetBSD/blog.html) is a lot more than any 
credibility you can bring to the table. He's supporting a project which, 
by all rights, DESERVES to be on top because it would take the least 
amount of work to get it there (support a couple of new processors to 
catch up with Linux, add some fine-grained [but does it have to be 
Solaris-like? Isn't that what killed FreeBSD 5 to begin with?] SMP, and 
develop a more convenient update mechanism for pkgsrc, and suddenly 
NetBSD would be a world leader). I can sympathise with his efforts 
entirely. I do my part by speaking out (some call it trolling, that's up 
to them) on Slashdot, and I find a lot of people who agree.

You're quite right that BSDs should work together and not try to 'beat' 
each other, but on the other hand some healthy (even if negative) 
indication of a project's performance gives good incentive to identify 
and remove problems. Prior to people noticing Linux 2.4 was a joke in 
scalability and some areas of performance, it was touted as the be-all 
end-all solution for huge governments and corporations for any need. 
Linux 2.6 is an order of magnitude closer to this, but its stability 
leaves a lot to be desired. If FreeBSD manages to properly clean up all 
of its SMP locking and really perform, it will be a wonderful system. No 
question. But unless it's willing to work on portability, it will be 
left far behind the new architectures that are springing up and taking 
the lead in whatever niche. THAT is an area FreeBSD should work on, and 
it doesn't take a benchmark to demonstrate it either.

I went on a lot of tangents, sorry if you're distracted. Point is: 
Hubert did not lie, and accusing him of lying in such an offensive tone 
is a huge insult.

-D. Nikulin