> > system on NetBSD, Solaris, or most Linux distributions), pkgsrc doesn't
> > bootstrap one into place.  If the user installs the wrong C compiler--a
> > buggy, old, or trojaned one (food for thought:
> > http://www.acm.org/classics/sep95/)--he loses.
>
> If the user hasn't installed a C compiler, then they're going to have
> an interesting but unrewarding time building applications from source
> using pkgsrc.

Yes.

> > > > > 2.  openssl produces output in a slightly different format from
> > > > > md5(1).  I really don't want to have to pre-process everything with
> > > > > sed or awk or expr.
> > > >
> > > > Can't the existing md5 utility still handle the MD5 hashes which were
> > > > generated with it?  For the SHA-1 and RIPEMD-160 hashes, would you
> > > > consider making the output of your digest utility have the same format as
> > > > the output from OpenSSL?
> > >
> > > It's not a huge problem, it's a minor niggle. Yes, we can massage output
> > > with sed, awk or expr. But I don't want to do that.
> >
> > You misunderstand.  What I requested is output from "digest sha1 foo" in
> > the format that "openssl dgst -sha1 foo" has, and likewise for "digest
> > rmd160 foo" to have the same format as "openssl dgst -rmd160".  That way,
> > if it ever becomes desirable to use OpenSSL for hashing--for instance, in
> > a future world where pre-1999 versions of NetBSD needn't to be fully
> > supported--such massaging will not be necessary.  I've appended a trivial
> > patch which does this.  For the SHA-1 and RIPEMD-160 hashes, the slightly
> > different output is unnecessary.  MD5 hashes have already been calculated,
> > so I don't propose changing them.
>
> Sorry about the misunderstanding.  But when there are over 3000 files
> which have been prepared using a standard NetBSD utility called
> md5(1), it's a bit of an onerous job to have to reformat those 3000+
> files, just because openssl outputs in a different format.  Yes, we
> can massage the output by using sed, awk or expr, but, frankly, I
> don't want to.

That's quite reasonable.  That's why I didn't propose changing the MD5s.

> Have a look at the logic used to identify checksums
> and checksum algorithms in bsd.pkg.mk, and you'll see why I don't want
> yet another format to parse.  I'm sorry, but I thought I'd put that in
> my original mail.

Well, if y

Your digest utility introduced two new formats.  If my patch were
incorporated in it, then there would be five formats altogether.  However,
the number of formats could be reduced back to three by reformatting the
few "md5" files that have been generated with your digest utility.

> > > > > 3.  I want a message digest calculation utility that is small and
> > > > > quick, and something that either is present on all Operating Systems
> > > > > on which pkgsrc runs, or is buildable on those operating systems with
> > > > > minimum fuss. openssl does not really fit the bill here.
> >
> > > In the whole scheme of things, though, with all the other processing
> > > that is taking place at that time, the speed of the code produced by
> > > an optimising compiler vs.  hand-tuned assembly code is fairly low on
> > > my list of priorities.
> >
> > Thanks for dropping the objection.
>
> Au contraire - I have not dropped any objection.
>
> And I do resent such an implication.
>
> My apologies, however, if you're just being disingenuous.

If I were, it would be I who would owe an apology, wouldn't it?

> What I DID write, however, was:
>
> > Now I'm not a statistician, but the sample size isn't huge, I don't
> > know what was taking place on your machine while you were taking your
> > measurements, and, all in all, the results aren't all that conclusive,
> > are they? I mean, to take your figures above, digest takes 65% of the
> > time of openssl to calculate sha1 digests.
> >
> > OTOH, I wouldn't be that surprised if openssl was actually quicker,
> > since openssl uses assembly code on the more popular architectures.
> >
> > In the whole scheme of things, though, with all the other processing
> > that is taking place at that time, the speed of the code produced by
> > an optimising compiler vs.  hand-tuned assembly code is fairly low on
> > my list of priorities.

Yes, that's what you wrote.  I've quoted it all now.  As I understood your
original statement that you wanted something "quick" and OpenSSL "does not
fit the bill", I took it to mean that in your testing, OpenSSL was too
slow for practical use.  Now you seem to be saying that my testing is
statistically invalid and inconclusive, that you haven't done any
conclusive testing of your own, OpenSSL may be faster in sum^H^Home cases,
speed is a low priority.  Nonetheless you still object that OpenSSL is too
slow.  Is that a fair summary?  If not, kindly rephrase.  I don't mean to
be unfair.

If performance is still crucial for you, would you please describe what
benchmark or criterion would satisfy you?

> Perhaps you'd like to go away

:-)

> and cut down openssl so that it's < 200K in source form, is quick to
> compile,

For people who already have it, such as those who have it in the base
system, these are non-issues.  Eventually, it's likely that the great
majority of users will be in this category.

> is statically linked,

Although I read your earlier statement about this ("we really need a
statically-linked utility, if ever we're going to install a system from
packages of any kind"), I didn't follow the reasoning.  Perhaps what you
had in mind was a situation where the hashing utility were dynamically
linked and the user replaced the libraries on which it depended, without
replacing it at the same time.  Clearly that would be broken.  It's not
clear to me that statically linking the hashing utility would be the only
way to avoid that problem.

> has no patches to be applied,

Is the reasoning here that if there are patches, the hashes for those
patches must be tested, and since we're preparing the hashing utility, we
encounter a chicken-and-egg problem?  Since this doesn't apply for people
who have OpenSSL in the base system, I assume you're talking about people
who install it from pkgsrc.  Checking hashes can be done to detect
accidental file corruption, or malicious changes.  For detecting
accidental damage, MD5s are adequate (overkill, really).  If there were
malicious changes, a malefactor who could change the patch files could
just as easily change the "md5" files.  Therefore the SHA-1 and RIPEMD-160
hashes provide no extra protection to speak of.  Therefore, the md5
utility could be used to avoid the chicken-and-egg problem.

> and provides md5(1)-compatible output

The whole purpose of the exercise is to replace, or at least augment, MD5
with longer hashes.  If MD5 is abandoned, then this becomes a non-issue.

> and is distributed with a BSD licence,

Strictly speaking, it's not under a BSD license, because the copyright
does not belong to the Regents of the University of California.  However,
the authors' description of it as "BSD-style" seems accurate enough, given
the mostly identical terms (IANAL). I've appended it for you.

> then we'll consider switching over to using openssl as part of pkgsrc.

Thanks!
-- 
Trevor Johnson
http://jpj.net/~trevor/gpgkey.txt



  LICENSE ISSUES
  ==============

  The OpenSSL toolkit stays under a dual license, i.e. both the conditions of
  the OpenSSL License and the original SSLeay license apply to the toolkit.
  See below for the actual license texts. Actually both licenses are BSD-style
  Open Source licenses. In case of any license issues related to OpenSSL
  please contact openssl-core@openssl.org.

  OpenSSL License
  ---------------

/* ====================================================================
 * Copyright (c) 1998-2001 The OpenSSL Project.  All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 *
 * 1. Redistributions of source code must retain the above copyright
 *    notice, this list of conditions and the following disclaimer.
 *
 * 2. Redistributions in binary form must reproduce the above copyright
 *    notice, this list of conditions and the following disclaimer in
 *    the documentation and/or other materials provided with the
 *    distribution.
 *
 * 3. All advertising materials mentioning features or use of this
 *    software must display the following acknowledgment:
 *    "This product includes software developed by the OpenSSL Project
 *    for use in the OpenSSL Toolkit. (http://www.openssl.org/)"
 *
 * 4. The names "OpenSSL Toolkit" and "OpenSSL Project" must not be used to
 *    endorse or promote products derived from this software without
 *    prior written permission. For written permission, please contact
 *    openssl-core@openssl.org.
 *
 * 5. Products derived from this software may not be called "OpenSSL"
 *    nor may "OpenSSL" appear in their names without prior written
 *    permission of the OpenSSL Project.
 *
 * 6. Redistributions of any form whatsoever must retain the following
 *    acknowledgment:
 *    "This product includes software developed by the OpenSSL Project
 *    for use in the OpenSSL Toolkit (http://www.openssl.org/)"
 *
 * THIS SOFTWARE IS PROVIDED BY THE OpenSSL PROJECT ``AS IS'' AND ANY
 * EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
 * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE OpenSSL PROJECT OR
 * ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
 * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
 * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
 * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
 * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
 * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
 * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
 * OF THE POSSIBILITY OF SUCH DAMAGE.
 * ====================================================================
 *
 * This product includes cryptographic software written by Eric Young
 * (eay@cryptsoft.com).  This product includes software written by Tim
 * Hudson (tjh@cryptsoft.com).
 *
 */

 Original SSLeay License
 -----------------------

/* Copyright (C) 1995-1998 Eric Young (eay@cryptsoft.com)
 * All rights reserved.
 *
 * This package is an SSL implementation written
 * by Eric Young (eay@cryptsoft.com).
 * The implementation was written so as to conform with Netscapes SSL.
 *
 * This library is free for commercial and non-commercial use as long as
 * the following conditions are aheared to.  The following conditions
 * apply to all code found in this distribution, be it the RC4, RSA,
 * lhash, DES, etc., code; not just the SSL code.  The SSL documentation
 * included with this distribution is covered by the same copyright terms
 * except that the holder is Tim Hudson (tjh@cryptsoft.com).
 *
 * Copyright remains Eric Young's, and as such any Copyright notices in
 * the code are not to be removed.
 * If this package is used in a product, Eric Young should be given attribution
 * as the author of the parts of the library used.
 * This can be in the form of a textual message at program startup or
 * in documentation (online or textual) provided with the package.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 * 1. Redistributions of source code must retain the copyright
 *    notice, this list of conditions and the following disclaimer.
 * 2. Redistributions in binary form must reproduce the above copyright
 *    notice, this list of conditions and the following disclaimer in the
 *    documentation and/or other materials provided with the distribution.
 * 3. All advertising materials mentioning features or use of this software
 *    must display the following acknowledgement:
 *    "This product includes cryptographic software written by
 *     Eric Young (eay@cryptsoft.com)"
 *    The word 'cryptographic' can be left out if the rouines from the library
 *    being used are not cryptographic related :-).
 * 4. If you include any Windows specific code (or a derivative thereof) from
 *    the apps directory (application code) you must include an acknowledgement:
 *    "This product includes software written by Tim Hudson (tjh@cryptsoft.com)"
 *
 * THIS SOFTWARE IS PROVIDED BY ERIC YOUNG ``AS IS'' AND
 * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 * ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
 * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
 * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
 * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
 * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
 * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
 * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 * SUCH DAMAGE.
 *
 * The licence and distribution terms for any publically available version or
 * derivative of this code cannot be changed.  i.e. this code cannot simply be
 * copied and put under another distribution licence
 * [including the GNU Public Licence.]
 */