On Thu, Mar 22, 2001 at 11:24:41AM -0500, Trevor Johnson wrote:
> > 1.  openssl is huge, and I don't want pkgsrc to be reliant upon a huge
> > piece of software as a basic building block.  libtool not withstanding.
> 
> You're right, the copy that comes with NetBSD 1.5 for i386 (base.tgz) is
> 231292 bytes.  Still, pkgsrc itself is much larger than that.

The openssl source is huge - 1.5S sources:

% du -sk openssl
10949   openssl/
%

(and that includes CVS dirs, but excludes Makefile.openssl).  You're
seriously telling me I should add well over 10 Megabytes of source to
a single package in pkgsrc? OK, so on to executable sizes, which isn't
really relevant:

% size `which openssl` `which digest`
text    data    bss     dec     hex     filename
219215  9856    2260    231331  387a3   /usr/bin/openssl
94493   4084    7960    106537  1a029   /usr/pkg/bin/digest
%

(openssl is linked dynamically, digest statically)
 
> > 2.  openssl produces output in a slightly different format from
> > md5(1).  I really don't want to have to pre-process everything with
> > sed or awk or expr.
> 
> Can't the existing md5 utility still handle the MD5 hashes which were
> generated with it?  For the SHA-1 and RIPEMD-160 hashes, would you
> consider making the output of your digest utility have the same format as
> the output from OpenSSL?

It's not a huge problem, it's a minor niggle. Yes, we can massage output
with sed, awk or expr. But I don't want to do that.
 
> > 3.  I want a message digest calculation utility that is small and
> > quick, and something that either is present on all Operating Systems
> > on which pkgsrc runs, or is buildable on those operating systems with
> > minimum fuss. openssl does not really fit the bill here.
> 
> Well, pkgsrc is only available for NetBSD, whereas OpenSSL is available
> for NetBSD, Solaris, OpenBSD, FreeBSD, Linux, and Windows (with how much
> fuss I don't know, but it's in the base system for the BSD flavors).

pkgsrc runs on Solaris and Linux as well as NetBSD.  For anyone who is
worried, I, personally, have no intention of making pkgsrc work on
Windows.  Openssl is only in later versions of the base system on
NetBSD.  For previous versions, there is only the package, and there
we run into bootstrap problems again.
 
> As for performance, for me, OpenSSL is a little faster at calculating all
> three hashes:
> 
> $ cat mozilla-source-0.8.tar.bz2>/dev/null;time digest sha1 mozilla-source-0.8.tar.bz2;cat mozilla-source-0.8.tar.bz2>/dev/null;time openssl dgst -sha1 mozilla-source-0.8.tar.bz2
> SHA1 (mozilla-source-0.8.tar.bz2) = 6e62d9466c3aca13b6020aff532f9245e52a69ad
> 
> real    0m1.150s
> user    0m0.882s
> sys     0m0.266s
> SHA1(mozilla-source-0.8.tar.bz2)= 6e62d9466c3aca13b6020aff532f9245e52a69ad
> 
> real    0m1.785s
> user    0m1.541s
> sys     0m0.242s
> $ cat mozilla-source-0.8.tar.bz2>/dev/null;time digest rmd160 mozilla-source-0.8.tar.bz2;cat mozilla-source-0.8.tar.bz2>/dev/null;time openssl dgst -rmd160 mozilla-source-0.8.tar.bz2
> RMD160 (mozilla-source-0.8.tar.bz2) = 42933b877546311a355893e70e8f9071ba63e275
> 
> real    0m2.940s
> user    0m2.500s
> sys     0m0.431s
> RIPEMD160(mozilla-source-0.8.tar.bz2)= 42933b877546311a355893e70e8f9071ba63e275
> 
> real    0m1.904s
> user    0m1.652s
> sys     0m0.235s
> $ cat mozilla-source-0.8.tar.bz2>/dev/null;time digest md5 mozilla-source-0.8.tar.bz2;cat mozilla-source-0.8.tar.bz2>/dev/null;time openssl dgst -md5 mozilla-source-0.8.tar.bz2
> MD5 (mozilla-source-0.8.tar.bz2) = c2c5e6d1257b9da80ccb6b424974b1c3
> 
> real    0m1.682s
> user    0m1.352s
> sys     0m0.321s
> MD5(mozilla-source-0.8.tar.bz2)= c2c5e6d1257b9da80ccb6b424974b1c3
> 
> real    0m0.964s
> user    0m0.782s
> sys     0m0.181s

Now I'm not a statistician, but the sample size isn't huge, I don't
know what was taking place on your machine while you were taking your
measurements, and, all in all, the results aren't all that conclusive,
are they? I mean, to take your figures above, digest takes 65% of the
time of openssl to calculate sha1 digests.

OTOH, I wouldn't be that surprised if openssl was actually quicker,
since openssl uses assembly code on the more popular architectures. 
In the whole scheme of things, though, with all the other processing
that is taking place at that time, the speed of the code produced by
an optimising compiler vs.  hand-tuned assembly code is fairly low on
my list of priorities.

Regards,
Alistair