Re: Package naming when no backward compatibility at all?

[Greg Troxel <gdt%lexort.com@localhost>]

"J. Lewis Muir" <jlmuir%imca-cat.org@localhost> writes:

> What I was trying to say here is that by packaging multiple versions
> with major versions in the name (but for Python and Ruby and maybe
> others, it's major and minor in the name), it's basically saying that
> the expectation is that the package will always be that major version
> and will be backward compatible within that major version.

Yes, and further that we accept that upstream is like this, and that the
package is sufficiently popular as a dependency that we accept that
there will be upstreams that do not have releases that work with the
most recent version, and that it's better for users to accomodate this,
even though it is time-consuming for pkgsrc people.

> I see, so if one package changes, let's say package foo changes from
> version 7.0.8 to 7.0.9, then I could bump the PKGREVISION number of
> all packages that depend on foo just to be safe, even though their
> version did not change.  I understand this is referred to as a revbump
> in the pkgsrc community.  (And I know there's a revbump.py tool in
> pkgtools/revbump that could help automate this.  The revbump.py(1) man
> page says that it also increases BUILDLINK_ABI_DEPENDS, but I don't
> know whether it handles my exact match approach below.  I would guess
> it doesn't, but even so, I could still use it and then change the
> buildlink3.mk file to use the exact match.)

The rule is to increase PKGREVISION of all depending packages if (and
only if) there *is* an ABI break.   Sometimes that is "there seems like
there might, so err on the side of caution", which leads to unnecessary
rebuilds.

An ABI break can be any of:
  - symbol withdrawn in a shlib
  - calling args changed in a function in a shlib
  - behavior changed for any function, other than bug fixes
  - any program/script which could reasonably be used by another program
    changing behavior (in a non-bugfix way)

Whether or not ABI changes happen in various kinds of releases is an
upstream ASE issue, and is not relevant to the revbump process.

> In package foo's buildlink3.mk file, I think I'd use "=" for the
> BUILDLINK_A{P,B}I_DEPENDS.foo assignment, not "+=", and set an exact
> match for the dependency, not a package wild card.  For example, I think
> I'd want to use
>
>   BUILDLINK_API_DEPENDS.foo=     foo-7.0.9
>   BUILDLINK_ABI_DEPENDS.foo=     foo-7.0.9
>
> not
>
>   BUILDLINK_API_DEPENDS.foo+=    foo>=7.0.9
>   BUILDLINK_ABI_DEPENDS.foo+=    foo>=7.0.9
>
> This way, I'd catch at build time any API changes, and I think this
> would make a binary package upgrade work too.

This is contrary to pkgsrc norms.

You haven't yet explained whether *every single* release from this
upstream has an ABI break, or what fraction of them.  (You haven't named
it either, so nobody else could look.)

> I think there's one last case that I'd need to handle, which is when
> a package, say foo-using-app, needs an executable from package foo to
> run.  If I'm understanding correctly, that case is not handled by the
> buildlink3 framework because the buildlink3 framework is for headers
> and libraries, not executables.  In this case, package foo-using-app
> doesn't include package foo's buildlink3.mk, I don't think.  Instead
> it appends to DEPENDS in its Makefile, and I think it would need an
> explicit version instead of a package wild card, so
>
>   DEPENDS+=    foo-7.0.9:../../wip/foo
>
> not
>
>   DEPENDS+=    foo-[0-9]*:../../wip/foo
>
> Am I right about this?

Yes, except it should be >= because the normal expectation is that when
there's a new release, things will often/usually still work.

> I think I'm going to end up with the equivalent of <major> in the
> package name, which will correspond to a release series of the
> software in question, just because I think there will be a need to
> support multiple release series at the same time.  This should remain
> manageable, though, because there will probably be at most three release
> series supported by upstream at any given time.  This is just the naming
> scheme for having different major versions of a package installed at the
> same time like, for example, lang/ruby{31,32,33}.

Whether this is ok in pkgsrc proper depends on how useful this is vs the
clutter; we don't really have any norms.  It's fine it wip, I'd say.

>> Also:
>> 
>>   "does not promise not to change the API" is different from "the API is
>>   broken often enough that this is a real problem in practice".  Make sure
>>   you are solving a real problem, not engaging in massive amounts of work
>>   to solve a theortical problem.   (To me, that's what it feels like you
>>   are doing.)
>
> Upstream definitely has broken the API, so it's not just theoretical,
> but how often, I'm not sure.

API changes happen in many packages.  By theoretical, I meant changes
that are so frequent that the normal approach becomes painful.   In this
upstream's case, I'd expect that after an API break, that either

  the rest of the programs build anyway, because they have stopped using
  that piece of the API

  there is a new release of the other programs soon or at the same time
  that works with the new API

  there was already a release that can cope with old or new API

  there will be soon a new release

and basically you should locally stage the update and rebuild everything
else, and when it's all ok, commit it, and if it's not ok pester
upstream.

> Re solving a real problem and avoiding a
> massive amount of work, I hear you and agree.  At the same time, though,
> if I do go to the trouble of packaging this stuff, I'm hoping to package
> in a way that is as correct as possible within reason and that will
> result in binary package updates that work right.

To get binary updates to work, you need to revbump when there is an ABI
break.  That's it, just like any other package.  If upstream NEWS
doesn't mantion ABI breaks, then that's a further ASE failure and it's
even harder -- but it doesn't change what you have to do.

When there's a revbump, then any binary package set used e.g. by pkgin,
should build new versions of all, and update to them together.

IMHO, you should not worry about "if someone replaces one package and
doesn't change a depeending one (even though it was revbumped), things
can break".  It is not possible to encode in a package the upper bound
of a version of a depending package -- one can't predict the future.