> This was discussed a lot recently. Some suggested we should just provide 
> complete files (such as included in syspkgs) and not binary diffs (like 
> FreeBSD's binary updates).

My preference is for complete syspkgs.  Cutting down the amount of data to
be transferred seems a worthy aim, but I thought that making a syspkg the
smallest unit that is transferred is more manageable (but see below).

> Do you have further details on how you'd like this to be done?
> 
> As mentioned in the FreeBSD binary updates whitepaper, my Puget Sound 
> Technology system just created MD5 checksums for all files of resulting 
> install and then compared after patching. Then I manually reviewed the 
> changed files to determine if there was any noise. (FreeBSD automates 
> this.)

Some sort of checksum per syspkg is what I was thinking.  However, it
might be easier to do one per file and then convert the list of changed
files into a list of changed syspkgs.  I was hoping that this could be
automated - it could then be added to the build process for an update
release.

> I tarred up the new files and appended to a self-extracting sh script 
> that backed up previous files, can roll back changes, list files, and 
> give info about the update.

I was thinking that the syspkgs would be available to download individually.
However, it probably makes more sense to have them in one file for release
updates (e.g. 3.0 to 3.0.1).  Security fixes could also be done as a single
download containing all the updated syspkgs.  I was thinking that each new
version would contain all the changed syspkgs from the base version, so
that it's possible to go from 3.0 or 3.0.1 to 3.0.2.  For the 3.0.1 to
3.0.2 update, one would download more than is necessary, but at least this
would be manageable on the production (releng) side.

I wasn't thinking of putting the installation mechanism in the update file.
I was thinking that a client (e.g. an extension to sysinst) would fetch the
current list of available updates for the current system and then download
and install these updated syspkgs.  The client would handle backing up the
existing syspkgs and installing the updated ones, as well as roll back and
commit (remove backup).

I haven't considered in detail the format of the "database" listing the
available releases/updates and security updates.  The client needs to be
able to determine which updates are available and which updates supercede
others.  For example, the client has 3.0.1 installed and updates are
available for 3.0.2 and 3.1 from the same base system (3.0), as well as
security patches for 3.0 and 3.0.1.  The client needs to know which choices
to present to the user - maybe it just presents them all.  How do we handle
versioning for security patches?

> I think we don't need to do binary diffs. But just use syspkgs. Tell the 
> users that new syspkgs are available (even in database, like 
> pkg_summary(5), or in security announcements) and let the user upgrade 
> with pkg_add. (By the way, I have a pkg_add that overwrites files instead 
> of deleting existing package and removing files first; I have used it 
> hundreds of times for over a year on various NetBSD and Linux systems but 
> not recently; most of it is in pkgsrc-wip.)

There seems to be a lot of overlap in the way that I was envisaging this
working with how I'd like pkgsrc package updates to be done.  However, I
decided against extending the project to include pkgsrc, as I thought that
the scope would then be too broad.  Plus, I haven't really been following
pkgsrc and pkgsrc-wip recently.

Do you think that there is still merit in having this as an SoC project?
It seems that there is a fair amount of work available that goes some of
the way to achieving the goals I had in mind.  pkg_add could be the tool
that is used to update system packages as well as pkgsrc packages - thinking
about this, it makes sense from a user perspective.  Possibly the project
would be better as a pkgsrc or part pkgsrc one?

Thanks,

J

-- 
  My other computer also runs NetBSD    /        Sailing at Newbiggin
        http://www.netbsd.org/        /   http://www.newbigginsailingclub.org/