Re: 2008Q1 -> current: downgrade

[David Holland <dholland-pkgtech%netbsd.org@localhost>]

On Sun, Jul 06, 2008 at 10:37:24PM +0300, Aleksey Cheusov wrote:
 > > On Thu, May 22, 2008 at 11:51:12AM +0300, Aleksey Cheusov wrote:
 > >> [downgrades]
 > >>
 > >> pkgsrc versions fetched by 'cvs up -D<DATE>'
 > 
 > > The overwhelming majority of what you've found are actually upgrades
 > > documented perfectly well in CHANGES-*.
 >
 > Not exactly. CHANGES-* says "updated". "updated" means either upgraded
 > (version is increased) or downgraded (version is decreased).

"Update" implies "upgrade" in the sense you're using it - it means
"bring this package to a newer version". We don't always use the word
"upgrade" because "upgrade" implies increased functionality, and as we
know new versions of software frequently don't deliver that. :-)

That's a matter of translation though.

 > Thus, there is no information about downgrades in CHANGES-* files.
 > No automatic collecting an information about downgrades - no control
 > over them. This is what pkg/39080 is about.

There aren't supposed to be downgrades, in general, and when there are
they should be noted as such in CHANGES.

 > > They show up as downgrades to
 > > you and/or pkgsrc-dewey because of disagreements about the semantics
 > > of version numbering, most commonly:
 > 
 > >  o  Dated snapshots normally come before numbered releases, but you're
 > > describing the transition as a "downgrade".
 >
 > This is not MY problem. This is how dewey actually works.  Versions in
 > X.Y.Z format are always lower than that in YYYYMMDD format.  This is
 > how dewey is documented in pkg_info(8)

This is something dewey needs to be taught about, however.

 > >  o  Disagreement over whether 1.19 > 1.2 (the "dewey" approach) or 
 > > 1.2 > 1.19 (the "decimal number" approach).
 > Of course I use dewey approach ;-)

Right. But not all upstream packages do, and while sometimes the
people who import packages correct for this, sometimes they don't, and
this results in the apparent downgrades you're seeing, which are not
downgrades.

 > >  o  Disagreement over whether an alpha/beta version like 1.1b3 comes
 > > before the release version 1.1; it should and I think pkgsrc-dewey
 > > recognizes this if it's presented correctly, but either some old
 > > revisions don't present it correctly or your script did the wrong
 > > thing.
 >
 > According to pkg_info(8) 1.1a3/1.1b3 is not alpha/beta versions
 > of the package.

Then some old revisions don't present it correctly. At least one of
the packages I checked had a distname with "alpha1" in it that was
imported into pkgsrc as "a1".

Again, sometimes people who import packages adjust for version
numbering semantics properly and sometimes they don't.

 > >  o  In one case, the use of hex digits in version numbers.
 > ??? There is nothing about hex numbers in pkg_info(8).

Yes? Well, there was a hex digit ('b', as I recall) in someone's
upstream version number, and it went into the pkgsrc version that
way.

Again, sometimes people who import packages adjust for version
numbering semantics properly and sometimes they don't.

Ideally, they always would. Documenting the proper steps to take in
the pkgsrc guide would be a good start. (I'm pretty sure there's
nothing at all there right now.)

Some of the glitches/mistakes you've discovered may have been made
before dewey was systematized, too.

 > All alphabetic characters are valid in dewey version
 > in any position. 1z1 is equal to 1.26.1 and 9abc9 is equal
 > to 9.1.2.3.9. This is how it is documented in pkg_info(8).
 > alpha/beta/rc and some other letter sequences are treated especially.

Yes, that's fine, but you're missing the point.

 > > There are also a couple of cases where mistakes transcribing version
 > > numbers into pkgsrc resulted in apparent downgrades, either
 > > immediately or downstream.
 >
 > ???

Yes? There were a couple cases where people transcribed version
numbers into pkgsrc wrong. For example, in graphics/spcaview,
version 0.60 should have been 0.6a, so when 0.6b came next your script
complained about it. 

What do you mean by "???"?

 > > Of the eight (yes, only 8) cases remaining from those you posted:
 > Much more than 8 ;-)

No. There were exactly eight after excluding all the cases above.

 > > It is not entirely clear to me what other points you may be trying to
 > > make with this list (or by filing it in a PR). Could you please
 > > explain?
 > 
 > First, your numbers (two, three and eight) are totally wrong.

No, they are not totally wrong.

 > Second, I have already showed (in my previous emails) real life
 > examples why PKG_EPOCH is helpful and in which cases.

Yes, there are some cases where it might be helpful. But you haven't
shown that these cases are common enough to make it worth the trouble
of developing the infrastructure.

 > Fourth, do not forget we are discussing two different things:
 > 1) CHANGES-* doesn't provide an information about downgrades.
 > 2) PKG_EPOCH.
 > They are close but different. 1) is about why unseen downgrades happen,
 > 2) and is one of the way how these downgrades can be fixed.

No, we're discussing a third thing, which is what version number
semantics are observed in the wild and how this relates to the issue
of "downgrades" (both real downgrades, as in moving to an older
version, and apparent downgrades, where moving to a new version
confuses pkgsrc-dewey).

This is connected to both of the issues you cite but not the same as
either.

The data that's been posted indicates that
  (a) real downgrades don't happen very often: 3 cases in four years;
  (b) real epoch changes don't happen very often: 2-3 cases in four years;
  (c) glitches and outright mistakes in version number handling
      are more common than either: 6 cases in four years;
  (d) problems with version number semantics not matching pkgsrc-dewey
      are vastly more common than anything else: most of the cases you
      posted.

This indicates that

  (a) putting a lot of effort in to handle real downgrades specially
      is not worthwhile;
  (b) putting a lot of effort in to handle version epoch changes isn't
      worthwhile since there are other ways to cope;
  (c) it *is* probably worthwhile to run some kind of crosschecking
      script that complains about things that look like downgrades to
      pkgsrc-dewey, since the vast majority of them are errors;
  (d) it is probably worthwhile to find some way to teach dewey about
      YYYYMMDD version numbers;
  (e) the documentation of how to handle version numbers in pkgsrc is
      inadequate.

 > YET ANOTHER case/example why PKG_EPOCH may be useful.

My purpose in here is not to argue that it isn't useful. I'm just
questioning the assumption that it's urgently necessary, and
questioning the data you brought to support you arguments.

-- 
David A. Holland
dholland%netbsd.org@localhost