Subject: Re: digest
To: Todd Vierling <tv@wasabisystems.com>
From: Alistair Crooks <agc@pkgsrc.org>
List: tech-pkg
Date: 03/23/2001 11:31:48
On Wed, Mar 21, 2001 at 10:07:57AM -0500, Todd Vierling wrote:
> On Wed, 21 Mar 2001, Alistair Crooks wrote:
> 
> : So, let's assume for a moment that we ditch digest, and move to a
> : modified cksum(1) utility.
> 
> I'd like to reiterate my suggestion to make this particular bit of
> functionality part of pkg_admin(1), not cksum(1), which makes points 1, 2,
> and 5 here complete non-issues:
> 
> : 1.  we need to make a package for cksum(1).
> 
> : 2.  it needs to grow a version flag, since we want to know whether we
> : have a cksum binary which supports the sha1 algorithm, at a minimum.
> 
> : 5.  On other OSes, such as Solaris, Linux etc, you'll need to make
> : sure whether you can use the existing cksum binary, or install your
> : own under ${LOCALBASE}.

Points 1, 2, and 5 were addressing the unsuitability of using cksum(1) to
provide the digests. It is not really fitting that you should use those
reasons to champion the calculation of digests in pkg_admin. But, since
you brought it up, I'll address it here.

The problem with sinking digest calculation into the pkg_install tools
is that of bootstrapping. Since we don't currently carry the pkg_install
source in pkgsrc, we need to fetch it from somewhere.  So the distfile
needs to be checked for authenticity and integrity.  We usually use an
archive based on the source in basesrc, and apply any patches for
make(1) syntax funnies, or missing targets etc - it's by far the
easiest way for us.  So, until pkg_admin is built with digest
calculation on board, we have to trust to luck with the patches.  You
seem to indicate below that that is not a problem for you.  It is for
me.

It may shock, bewilder and surprise some of you to know that I actually
thought of this when I was working out how to support different message
digests in the future. It will certainly alarm some of you to know that
I added a separate pkg_digest part to the pkg_install tools, and then
wondered how to bootstrap this cleanly. I am still wondering.

I was rightly criticised for the "chicken and egg" problem which
occurred when digest came along, and the pkg_install tools were bumped
straight away. I have no desire to repeat any of that.
 
> And to address other specific points:
> 
> : We do need to know the version, since I believe the sha1 code in libc
> : until very recently was buggy on Alphas.  Oh, and the ident strings
> : don't show the version of code in libc.so that's dynamically linked in?
> : Bummer.  How do we know whether we have the correct sha1 code on Alphas?
> 
> Deliberately add the sha1 code to pkg_admin, as libc probably won't have the
> code on other OS's, either.  Hacking zoularis to add the code is a rather
> hokey (and IMHO not acceptable) workaround....

What you have described is how it is done in digest.  No-one (except
you) suggested hacking Zoularis.
 
> : 3.  This package that we create for cksum - it can't have any patches,
> : can it, since how would we check (a) the distfile, and (b) the patches
> : if we don't have an existing message digest calculation method.  Use
> : the existing md5 sums that we have?  Yes, but that means that cksum
> : has different handling in its checksum and patch-sum targets from
> : every other package on the system.  So we don't use checksums at all
> : for the cksum package?  Not too happy about this one, since we'd have
> : to fetch a distfile without using a checksum to make sure that
> : no-one's tampered with it. How about bundling the source with pkgsrc?
> : Yes, good idea, that would work, let's go with that.
> 
> The source doesn't need to be bundled with pkgsrc.  Add detection in
> pkg_install for a needed pkg_admin(1) binary that can checksum it, possibly
> falling back to md5(1).  If no such program exists, print a warning to the
> screen about pkg_install not being verified by checksum.  Problem solved.

No.  The problem is not "solved".  You are now adding custom checks
and different code to the pkg_install package (unacceptable to me), or
you are ignoring the problem by displaying a warning.  Ignoring a
problem doesn't make it go away. Most definitely not "problem solved".
 
> : 4.  We really need a statically-linked utility, if ever we're going to
> : install a system using packages of any kind.
> 
> This is irrelevant to pkgsrc.  System install utilities belong in basesrc,
> and are linked to the install media using crunchgen.

Thanks - I wondered what was happening when I built those 1.5S CDs last week
and installed them on two servers here, but I am now enlightened.

And the issue of static vs. dynamic linking, and installation of system
packages, is certainly not irrelevant to pkgsrc.

Regards,
Alistair