On Tue, 26 Jul 2005, Curt Sampson wrote:

> There are two issues I see with this, though I don't know if they're
> important enough that we want to give up "single-pass" functionality.

Aha, "single-pass" (extract-in-place) is the issue you've been inferring
here.  I forgot that there was work done on that -- seems my perception of
pkg_add is still a bit back in the stone age.  :)

>     1. If we extract, then nuke when we discover an error, we wipe out
>     any files that existed before we started the extraction.

If tempfile-then-rename(2) is used, maybe not.  Perhaps the tarball can be
extracted in a special subdirectory of ${PREFIX}, to be moved into place by
rename(2) once the members are all extracted and verified?  That would be
similar to a staged install, but skips the extra copy step.

This does have partition-spanning issues, unfortunately, if someone has
${LOCALBASE} (or other ${PREFIX} depending on the package) mounted partially
on one partition, and partially on another.  <sigh>  It's difficult to
satisfy all parties here....

We could bend over backwards and treat the "extras" as files to move a la
classical staging installs.  Or offer the user the option to do
rename-in-place (with caveat that ${PREFIX} must be one partition) or staged
install.

I'm not sure what approach would work best.

>     2. Creating a signed archive can't be done with an append operation
>     on an existing archive. Though now that I think about it, since you
>     could just write a new signed archive (except, ironically enough,
>     not if the input is streamed to you), this is probably no big deal
>     at all.

Personally, I get warm fuzzies if the signing process is forced to happen
at package tar-up time, because that reduces the compromise time window.
But that's anecdotal and not a technical argument.

In reality, it's not hard to sign an archive that uses stream-based
compression after the fact without extracting to the filesystem.  It just
requires a bit more CPU to do two decompression passes (one to create
signatures/hashes, and one to stream data to be recompressed) and one
compression pass (to rebuild the tarball with the signature up front).

What is good about having the signatures/hashes available up front in the
tarball is the ability to have quick-abort functionality on a failed
signature.

>     1. In a "generic signed archive," the first file in the archive is a
>     list of hashes for all of the following files, in order.[...]

>     2. It is an error for files to be in the archive but not in the
>     signed list of hashes.
>
>     3. The first file will have a defined name, as well as an easily
>     recognisable format.[...]

That sounds exactly like what I was imagining.

> It seems to me that this could work for tar, cpio, pax, ZIP, whatever.
> Is there anything it's missing that pkgsrc would need? (I don't see the
> need for hashes for files created by scripts in the archive, since those
> scripts will be verified with hashes.)

If pkg_add understands the archive format internally (thus eliminating any
"hidden data" attacks caused by invoking an external tar or pax or
what-have-you), then just signing/hashing every physical file in the archive
is directly analogous to signing the tarball externally.

-- 
-- Todd Vierling <tv@duh.org> <tv@pobox.com> <todd@vierling.name>