Dieter Baron <dillo@danbala.tuwien.ac.at> writes:

>   I think you misunderstood my proposal.  I'll try to explain in more
> detail:

I probably did, but we will have to be very careful about defining these
if we hope for the variables to be set correctly.  Let me see if I can
restate your proposal in a way which I (and perhaps no one else :) find
clearer.

>   Besides a version number, each package can set the variable
> INCOMPATIBLE_BEFORE which states that the current version of the
> package is incompatible with versions before this.  If this variable
> is not set, it is presumed to be backwards compatible to all prior
> versions.

Perhaps we call this EARLIEST_COMPATIBLE_ABI to emphasize that it should
contain the earliest version with which the current version is ABI
compatible.  As an example, if package foo which contains only libfoo
had previous versions 1.3, 1.3.1, 1.3.2 had shlib version 5.0, 5.1, 5.1,
versions 1.4, 1.4.1, and 1.4.2 had shlib versions 6.0, 6.0, 6.1, then in
1.4 we would set EARLIEST_COMPATIBLE_ABI to 1.4, and in 1.4.2 we would
also set EARLIEST_COMPATIBLE_ABI to 1.4 to indicate that

1) if a package is built against e..g 1.3.2 then neither 1.4 nor 1.4.2
may  be used to satisfy dependecies

2) if a package is installed with 1.3.2 as a dependency, then updating
libfoo to 1.4.2 (via make replace, or pkg_add -u) is unsafe and will
break things.

>   Now when checking wether a dependency is usable, we check that the
> dependency pattern is fulfilled (as we do now), but in addition we
> check that the depending package does not accept a version earlier
> than the dependency's INCOMPATIBLE_BEFORE, as that would mean it was
> built against an incompatible earlier version.

I don't see where "depending package does not accept" comes from.  The
package is recorded as built against 1.3.2, and when considering 1.4.2
we see EARLIEST_COMPATIBLE_ABI is 1.4, and since that's > 1.3.2, it's
not usable.  I see the depending package as simply recording what it was
built against, and the rule is that the version of libfoo has to be >=
the recorded version, and the EARLIEST_COMPATIBLE_ABI for libfoo has to
be <= the recorded version.

>   So, to give an example: we have four packages:
>
>   package aaa: DEPENDS += bbb>=1.7
>                DEPENDS += ccc>=1.4.2
>                DEPENDS += ddd>=2.2
>   package bbb: PKGNAME = bbb-1.8
>   package ccc: PKGNAME = ccc-1.5.6nb2
>                INCOMPATIBLE_BEFORE = 1.5.6nb1
>   package ddd: PKGNAME = ddd-2.3
>                INCOMPATIBLE_BEFORE = 2.0
>
>   When installing aaa, each of its dependencies is checked:
>
>   bbb: bbb-1.8 satisfies bbb>=1.7; since bbb doesn't have
>        INCOMPATIBLE_BEFORE set, we are done: bbb is usable.

So INCOMPATIBLE_BEFORE or EARLIEST_COMPATIBLE_ABI defaults to 0, or some
version that is less than any valid version.

>   ccc: ccc-1.5.6nb2 satisfies ccc>=1.4.2; but ccc is incompatible with
>        versions before 1.5.6nb1, and aaa can use 1.4.2, so ccc is *not*
>        usable.
>
>   ddd: ddd-2.3 satisfies ddd>=2.2; ddd is compatible back to 2.0, and
>        aaa can use 2.2, so ddd is usable.

So yes, I think that we agree on what's to be checked.

> [storing the abi version for each dependency]
>> It's only a few bytes, and I suspect this is going to be hard enough to
>> get truly right that choosing simpler and easier over space-efficient
>> will turn out better.
>
>   Agreed.  However, I don't think your approach is simpler.

This way gives up on minor versions, but I think that's ok.  It's
basically never safe to use a lower version, and we lose almost nothing
by changing "basically never" to "never".

I think what I was suggesting is slightly conceptually simpler, but in
fact the difficulty will be correctly setting the INCOMPATIBLE_BEFORE /
EARLIEST_COMPATIBLE_ABI variables, and your approach makes that a bit
easier to get right and is therefore preferable.

>> With this, if ABI values are equal during make replace or pkg_add -u,
>> then unsafe_depends doesn't need to be set.
>
>   [Assuming ABI value is basically the same as INCOMPATIBLE_BEFORE.]
>
>   If the ABI values are not the same, the package is known to break
> compatibility (at least for some dependencies).  In that case, pkg_add
> -u should refuse to replace the package.  That is the whole point of
> the exercise.

I see it differently.  It's still useful to be able to do pkg_add -u,
and then to update the next package, so that one doesn't end up without
pacakges installed, even if some are broken.  pkg_chk can remove
packages and fail and then one can lose the list and end up hosed.  At
least it's happened to me.

With package_rolling-replace, it does 'make replace', which is
technically unsafe with respect to ABI changes, and sometimes is
actually unsafe.  It marks the depending packages dirty, and then does
make replace on those, to clean up any potential unsafeness.  I'm
propposing that we change the way make replace tags depending packages
to have a "known unsafe" mark (for when it is known that there is an ABI
change) and a "known safe, but we changed it" mark (for when it is known
that there was not an ABI change, but we changed packages, and the
paranoid realize that there could actually have been a change).

Then, I want pkg_add -u to set the same marks, with the same rules,
because it's really the exact same operation as make replace, just with
binaries.

If you want pkg_add -u to need some sort of forcing flag in the case of
a known ABI change, that's fine with me.  The binary version of
pkg_rolling-replace will just always use the flag.

>> So I can see your proposal as working, even though it doesn't have a way
>> to record the equivalent of increases in shlib minor version.
>
>   Why would you want to record a shlib minor version bump. What would
> you do with that information?

To be able to use 1.4.1 to satisfy a dependency for a package built
against 1.4.2 if that's known to be safe.  But as I said above I think
giving up on this is fine; in practice this is almost never safe and it
would be a huge headache to try to maintain the data.

>>  Above I
>> am essentially proposing that we presume that this changes with every
>> version change, or maybe every upstream change but not nb change.  I am
>> slightly nervous about these broad presumptions for which I'm sure we'll
>> find exceptions.
>
>   That would mean that you can replace a package only with the same
> version.  That doesn't sound useful.

No, I meant to allow a later version, just not going backwards unless
minors were the same.

> (Also, an nb bump could break
> backwards compatibility.)

It really shouldn't (due to the nature of nb bumps), but I suppose it
could.  With EARLIEST_COMPATIBLE_ABI, we don't have to treat nb bumps
specially, so that'sgood.

So if you think I mean the same  thing as you, I think we're converged.