Re: Introducing the patchadd binary patch toolchain

[Giles Lean <giles.lean%pobox.com@localhost>]

With the reservation that I am not sure that binary patches are
a Good Thing overall -- and granting that all alternatives have
problems too -- some comments and questions.

How do you plan/do you plan to identify patched binaries?  Will
their ident(1) strings be changed when they're patched?

Tonnerre Lombard <tonnerre%NetBSD.org@localhost> wrote:

>  - Arbitrarily named bsdiff(1) patches lifting the file from old to
>  new.

I would think about including the whole new file; that makes it simpler
to get the new files out, e.g. for experimenting to see if they fix a
problem.

>    * NAME: The name of the patch, frequently the file name. A form for
>      this would have to be agreed on; I would suggest the pullup
>      request ID (e.g. NAME=4123). Can be free form though, and does not
>      have to be the same as the file name.

Hmm.  There's a bicycle shed there.  I'd recommend choosing some
guidelines (any guidelines) about how patches are named.

I think I would include some name space suggestions: if the NetBSD
project releases official patches, it would be nice to recognise them
as being distinct from local patches, and from patches released by
third parties.

>    * VERSION: The version of the patch. A patch can be re-released in a
>      new version if appropriate, and would be suggested as an upgrade
>      by patch_tool.

I've seen patches with >50 replacements or updates on commercial
Unixes, FYI.

>    * DEPENDS: The name of a different patch file to depend on. This
>      line may appear as many times as required, but please only add one
>      dependent patch per line.

Patch dependencies are a can of worms, including patches that are
applicable to only some ports, patches that require different
dependencies on different ports, new revisions of patches that
alter the original dependency list, and more.

What about patches that are co-requisites (i.e. that must be installed
together) or patches which have alternate sets of patches which will
satisfy their dependencies (a case to be avoided if possible!).

Experience relates that tools will be wanted and needed to calculate
these dependencies, so being strict about this part of the file format
is IMHO important.

>    * CONFLICTS: The name of a different patch which cannot coexist with
>      this patch. The same rules as for DEPENDS apply.

It's (obviously) highly desirable never to have conflicting patches.
I would recommend the guideline of /never/ releasing a patch with a
known conflict, but reserving this field for problems discovered
post-release.

Which brings a new question: can the patch information be updated
without having to update the version of the patch?  e.g. if a patch
is put out but is missing a dependency, it might be desirable to
update the patch information, but not require users to install a
new version

>  - A file called +COMMENT containing a longer description (optional).

I'd recommend a guideline of including a URL that links to a description
of the patch which can be updated as needed.

> The tar file is not distributed as-is but as a bzip2 compressed OpenSSL
> binary file (.tbz) with detached signature (.tbz.sig). The command
> 
>  openssl smime -sign -binary -outform PEM -signer signingkey.pem -in
>     patchname.tbz -out patchname.tbz.sig
> 
> shall be used to produce the detached signature.

Be good to have all files that are released by the project
signed/checksummed in a consistent manner.

> The patches shall be kept in a directory called
> ftp://ftp.netbsd.org/${path}/${arch}/${vers}/, e.g. 
> ftp://ftp.netbsd.org/pub/NetBSD/misc/tonnerre/binpatches/amd64/5.0/

What about patches that are architecture independent?  Perhaps a small
enough case to ignore.  (I would /definitely/ avoid allowing patches
to apply to more than one OS version -- e.g. no 5.0 and 5.1 patches,
please -- so perhaps architecture is just another flavour of the
same issue.)

Will build.sh be able to be used to build patches?

> The directory shall contain a file named patchindex. This file starts
> with a version statement:
> 
> Version 0.1
> 
> to indicate that the patch file format version 0.1 is used. This
> statement is followed by a list of all patch metadata for each patch,
> starting with the Patch keyword and ending with the EndPatch keyword.
> Multiple consecutive words in a value have to be quoted. For example:
> 
> Patch 4123
> ABI=NetBSD
> OS_VERSION=4.0
> [...]
> COMMENT="A patch to add a funny hat to stargazer"
> EndPatch
> 
> The patch index is then amended with an OpenSSL s/mime inline text
> signature executed with the patch signing key.

What suffix will this file have?
Are end of line conversions a problem? (I think so.)
What character set will these files use -- UTF-8?

> This index file is used solely for the patch_tool utility. Even without
> this patch index, people can still safely run patch_add 4123 to add the
> patch.
> 
> 
> I think this might be a good point to start from.

Good luck.  Patching is a nightmare.

Cheers,

Giles