Re: Package naming when no backward compatibility at all?

["J. Lewis Muir" <jlmuir%imca-cat.org@localhost>]

On 10/18, Greg Troxel wrote:
> "J. Lewis Muir" <jlmuir%imca-cat.org@localhost> writes:
> > Since one strategy in pkgsrc seems to be to put the version in the
> > package name (e.g., lang/python27, lang/python313, security/gnupg, and
> > security/gnupg2), I initially thought that I could do the same, but then
> > I realized that since a backward incompatible change could be made for
> > any version change, even a patch-level version change, I think I'd have
> > to put the entire version in the package name (e.g., for foo version
> > 7.0.8.1, I'd have foo7081).  But then I'd have a bunch of versioned
> > packages.  Maybe that's the best I could do, but I'm not sure, which is
> > why I'm asking.
> 
> I don't follow this.  In the general case, we have one package for one
> upstream, name it without version numbers, and update it as releases
> come out.
> 
> As a workaround for upstreams that don't do a good enough job of API
> backwards compatibility, we package multiple versions with major
> versions in the name, so that depending packages can depend on them.
 
All of your comments are very helpful; thank you!

What I was trying to say here is that by packaging multiple versions
with major versions in the name (but for Python and Ruby and maybe
others, it's major and minor in the name), it's basically saying that
the expectation is that the package will always be that major version
and will be backward compatible within that major version.  And in the
case of the software ecosystem that I'm asking about packaging, since
upstream doesn't promise backward compatibility for the major, minor,
nor patch version, I was extending the concept to then put the major,
minor, and patch version in the package name since it would essentially
only be backward compatible with that exact version.

> Here, it's really hard to know how it's going to go.   Given that there
> are a bunch of related packages, it seems likely that when one package
> has an API break, then there will be new versions available of things
> that depend on them.  Thus, there's no need to keep multiple versions.

I see, so if one package changes, let's say package foo changes from
version 7.0.8 to 7.0.9, then I could bump the PKGREVISION number of
all packages that depend on foo just to be safe, even though their
version did not change.  I understand this is referred to as a revbump
in the pkgsrc community.  (And I know there's a revbump.py tool in
pkgtools/revbump that could help automate this.  The revbump.py(1) man
page says that it also increases BUILDLINK_ABI_DEPENDS, but I don't
know whether it handles my exact match approach below.  I would guess
it doesn't, but even so, I could still use it and then change the
buildlink3.mk file to use the exact match.)

In package foo's buildlink3.mk file, I think I'd use "=" for the
BUILDLINK_A{P,B}I_DEPENDS.foo assignment, not "+=", and set an exact
match for the dependency, not a package wild card.  For example, I think
I'd want to use

  BUILDLINK_API_DEPENDS.foo=     foo-7.0.9
  BUILDLINK_ABI_DEPENDS.foo=     foo-7.0.9

not

  BUILDLINK_API_DEPENDS.foo+=    foo>=7.0.9
  BUILDLINK_ABI_DEPENDS.foo+=    foo>=7.0.9

This way, I'd catch at build time any API changes, and I think this
would make a binary package upgrade work too.

I think there's one last case that I'd need to handle, which is when
a package, say foo-using-app, needs an executable from package foo to
run.  If I'm understanding correctly, that case is not handled by the
buildlink3 framework because the buildlink3 framework is for headers
and libraries, not executables.  In this case, package foo-using-app
doesn't include package foo's buildlink3.mk, I don't think.  Instead
it appends to DEPENDS in its Makefile, and I think it would need an
explicit version instead of a package wild card, so

  DEPENDS+=    foo-7.0.9:../../wip/foo

not

  DEPENDS+=    foo-[0-9]*:../../wip/foo

Am I right about this?

> Also, having multiple versions is clutter and messy, and for pythonXY,
> that's ok because of the broad benefit.  I would be opposed to having a
> dozen versions of 5 programs in pkgsrc becuase upstream makes
> unreasonable choices.  I would probably even be opposed to that in wip.

Yeah, that would be a mess.

> > Maybe another approach would be to include the <major> or <major><minor>
> > part of the version in the package name, but then declare a specific
> > version of that package in all packages that depend on it.  I'm not sure
> > this would even be possible, I'd have to study more, but I thought I'd
> > ask before spending too much time investigating this approach if it's
> > doomed to fail, or if there's a much saner approach that someone else
> > might suggest.
> 
> Just package each one, with no versions in the name, and see how it
> goes.  If updating something breaks everything else, wait a bit.  Ask
> upstream how to deal.  Look at Debian, and other packaging systems to
> see what they are doing.

I think I'm going to end up with the equivalent of <major> in the
package name, which will correspond to a release series of the
software in question, just because I think there will be a need to
support multiple release series at the same time.  This should remain
manageable, though, because there will probably be at most three release
series supported by upstream at any given time.  This is just the naming
scheme for having different major versions of a package installed at the
same time like, for example, lang/ruby{31,32,33}.
 
> Also:
> 
>   "does not promise not to change the API" is different from "the API is
>   broken often enough that this is a real problem in practice".  Make sure
>   you are solving a real problem, not engaging in massive amounts of work
>   to solve a theortical problem.   (To me, that's what it feels like you
>   are doing.)

Upstream definitely has broken the API, so it's not just theoretical,
but how often, I'm not sure.  Re solving a real problem and avoiding a
massive amount of work, I hear you and agree.  At the same time, though,
if I do go to the trouble of packaging this stuff, I'm hoping to package
in a way that is as correct as possible within reason and that will
result in binary package updates that work right.

>   Think about a strategy of deferring updates unless the whole system
>   builds (make a meta-package if there isn't one top package).  This
>   will result in doing updates top-down, and that's ok.  It will be far
>   less work than multi-version contortions.

OK, thanks for the suggestion.

Regards,

Lewis