Subject: Re: muhah
To: Trevor Johnson <email@example.com>
From: Alistair Crooks <firstname.lastname@example.org>
Date: 03/23/2001 09:24:38
On Thu, Mar 22, 2001 at 11:24:41AM -0500, Trevor Johnson wrote:
> > 1. openssl is huge, and I don't want pkgsrc to be reliant upon a huge
> > piece of software as a basic building block. libtool not withstanding.
> You're right, the copy that comes with NetBSD 1.5 for i386 (base.tgz) is
> 231292 bytes. Still, pkgsrc itself is much larger than that.
The openssl source is huge - 1.5S sources:
% du -sk openssl
(and that includes CVS dirs, but excludes Makefile.openssl). You're
seriously telling me I should add well over 10 Megabytes of source to
a single package in pkgsrc? OK, so on to executable sizes, which isn't
% size `which openssl` `which digest`
text data bss dec hex filename
219215 9856 2260 231331 387a3 /usr/bin/openssl
94493 4084 7960 106537 1a029 /usr/pkg/bin/digest
(openssl is linked dynamically, digest statically)
> > 2. openssl produces output in a slightly different format from
> > md5(1). I really don't want to have to pre-process everything with
> > sed or awk or expr.
> Can't the existing md5 utility still handle the MD5 hashes which were
> generated with it? For the SHA-1 and RIPEMD-160 hashes, would you
> consider making the output of your digest utility have the same format as
> the output from OpenSSL?
It's not a huge problem, it's a minor niggle. Yes, we can massage output
with sed, awk or expr. But I don't want to do that.
> > 3. I want a message digest calculation utility that is small and
> > quick, and something that either is present on all Operating Systems
> > on which pkgsrc runs, or is buildable on those operating systems with
> > minimum fuss. openssl does not really fit the bill here.
> Well, pkgsrc is only available for NetBSD, whereas OpenSSL is available
> for NetBSD, Solaris, OpenBSD, FreeBSD, Linux, and Windows (with how much
> fuss I don't know, but it's in the base system for the BSD flavors).
pkgsrc runs on Solaris and Linux as well as NetBSD. For anyone who is
worried, I, personally, have no intention of making pkgsrc work on
Windows. Openssl is only in later versions of the base system on
NetBSD. For previous versions, there is only the package, and there
we run into bootstrap problems again.
> As for performance, for me, OpenSSL is a little faster at calculating all
> three hashes:
> $ cat mozilla-source-0.8.tar.bz2>/dev/null;time digest sha1 mozilla-source-0.8.tar.bz2;cat mozilla-source-0.8.tar.bz2>/dev/null;time openssl dgst -sha1 mozilla-source-0.8.tar.bz2
> SHA1 (mozilla-source-0.8.tar.bz2) = 6e62d9466c3aca13b6020aff532f9245e52a69ad
> real 0m1.150s
> user 0m0.882s
> sys 0m0.266s
> SHA1(mozilla-source-0.8.tar.bz2)= 6e62d9466c3aca13b6020aff532f9245e52a69ad
> real 0m1.785s
> user 0m1.541s
> sys 0m0.242s
> $ cat mozilla-source-0.8.tar.bz2>/dev/null;time digest rmd160 mozilla-source-0.8.tar.bz2;cat mozilla-source-0.8.tar.bz2>/dev/null;time openssl dgst -rmd160 mozilla-source-0.8.tar.bz2
> RMD160 (mozilla-source-0.8.tar.bz2) = 42933b877546311a355893e70e8f9071ba63e275
> real 0m2.940s
> user 0m2.500s
> sys 0m0.431s
> RIPEMD160(mozilla-source-0.8.tar.bz2)= 42933b877546311a355893e70e8f9071ba63e275
> real 0m1.904s
> user 0m1.652s
> sys 0m0.235s
> $ cat mozilla-source-0.8.tar.bz2>/dev/null;time digest md5 mozilla-source-0.8.tar.bz2;cat mozilla-source-0.8.tar.bz2>/dev/null;time openssl dgst -md5 mozilla-source-0.8.tar.bz2
> MD5 (mozilla-source-0.8.tar.bz2) = c2c5e6d1257b9da80ccb6b424974b1c3
> real 0m1.682s
> user 0m1.352s
> sys 0m0.321s
> MD5(mozilla-source-0.8.tar.bz2)= c2c5e6d1257b9da80ccb6b424974b1c3
> real 0m0.964s
> user 0m0.782s
> sys 0m0.181s
Now I'm not a statistician, but the sample size isn't huge, I don't
know what was taking place on your machine while you were taking your
measurements, and, all in all, the results aren't all that conclusive,
are they? I mean, to take your figures above, digest takes 65% of the
time of openssl to calculate sha1 digests.
OTOH, I wouldn't be that surprised if openssl was actually quicker,
since openssl uses assembly code on the more popular architectures.
In the whole scheme of things, though, with all the other processing
that is taking place at that time, the speed of the code produced by
an optimising compiler vs. hand-tuned assembly code is fairly low on
my list of priorities.