Re: Introducing the patchadd binary patch toolchain

[matthew sporleder <msporleder%gmail.com@localhost>]

On Wed, Apr 29, 2009 at 3:45 PM, Elad Efrat <elad%netbsd.org@localhost> wrote:
> On Wed, Apr 29, 2009 at 10:12 PM, matthew sporleder
> <msporleder%gmail.com@localhost> wrote:
>
>> Comparing destdir's is a pretty good way to get differences between
>> two builds.  I did it successfully to generate patches in my own
>> updating tool.
>
> What are the management costs of always keeping an "old" and a "new"
> build? how does it work? each time you run a build something puts the
> current build into "old" and the new build into "new?
>

It was as easy as changing the target destdir on my next build.  Not a
huge deal.

>> I found that the biggest gotcha in the whole process is kernel changes
>> for two reasons:
>> 1) there are a lot of custom kernels out there (maybe there is where
>> binary patching is superior to file replacement?)
>
> I'm sorry, but I don't understand. Are you suggesting that it might be
> easier to patch because people might have different binaries?
>

I was asking if binary patching allowed you to replace just parts of a
binary kernel because replacing a whole kernel file is just the wrong
thing to do without a way to validate that a user definitely wants
your new GENERIC.  For example, would binary patching let you look at
/netbsd and ask "is LFS in here ? patch xyz : goto nextpatch"

I actually think the move to /stand/ modules + lighter kernels makes
this more possible than ever before (even if you add drivers to your
kernel, replacing a subsystem in /stand/ is probably still safe), I
don't have a good solution in mind.

> There was an effort on making sure compiling is "deterministic" in the
> sense that you could compare binaries and see no change. Assuming that
> happened, how reliable is patching going to be even in that case? what
> are you going to do about the people with kernels the patch simply
> doesn't apply to because the context differs too much (and that's not
> that hard for binary diffing)?
>
> One solution would be to agree that because we can't cover all cases
> with custom kernels, or have the means to "measure" to what extent a
> custom kernel can be patched (because this is subject to the custom
> features used and the patch itself), we ship complete replacements
> (say, GENERIC) people can use in the mean time. You could say, "but
> what about those where who *run* GENERIC? we can ship patches to them"
> -- and then we need to discuss whether we truly prefer patches over
> simple replacements, or want them in addition.
>

I agree.  Kernels available at sysinstall time should get an mtree
associated with them so they can be verified as being what came from
sysinstall.

> And in either case, a tool that can give you the choice to cover
> everything (produce both) or use everything (apply patches/updates)
> is, to me, superior to a tool that was "designed" for only half the
> functionality.

Since my tool was a nasty/slow collection of awk and shell, I won't
comment on the best way to distribute anything.  My primary goal was
to explore using mtree files as a system package format that could
easily be used in the build.sh structure.  My conclusion was that it
worked but couldn't handle dependencies without extra meta-files that
I wasn't willing to invent.  If I went down that path, I would rather
replace mtree with pkgsrc's format and go from there.