Re: postgresql packages, PG_SUBPREFIX and CONFLICTS

[Aleksej Saushev <asau%inbox.ru@localhost>]

Aleksey Cheusov <cheusov%tut.by@localhost> writes:

> Unfortunately discussion in this thread came to a standstill.  Instead of
> continuing repeating the same useless questions/answers again and again I
> decided to accumulate significant parts in one place together with some
> background information, just to make sure that everyone reading this
> understands what I'm talking about and to avoid misunderstanding.  This
> email will probably not very short, sorry.
>
> ============================================================
> Part 1. Why information about conflicting packages is important
> (regardless of how this information is stored and where).
>
> - When pkg_add is given a package having files that belong to
>   already installed package, it exits with error.
>
> - When pkg_add is given a package that CONFLICTS with installed
>   packages, it fails with error.
>
> - Being a low level package management utilities pkg_* are good enough for
>   managing a few number of installed packages but it's not appropriate tool
>   for managing hundreds of installed packages.  This is why pkg_chk, pkgin
>   and some others were created.  This is why I created pkgtools/nih that
>   works on top of pkg_install and uses pkg_summary(5).
>
> - For many years we have pkg_summary(5) that contains all meta-information
>   about packages in a "repository" including information about conflicting
>   package.  According to *existing* documentation CONFLICTS entry is
>   responsible for this.  Take a note that pkg_summary(5) does *NOT* include
>   PLIST entries, so CONFLICTS is the only place in pkg_summary(5) where
>   conflicting packages are registered.
>
> - One of the main goal of nih is to avoid breakages in the middle of
>   installation process. To achive this goal
>   "nih refresh; nih install list_of_packages" make the
>   following steps:
>       a) Download and unpack pkg_summary.gz and SHA512.gz.
>       b) Analyse installed packages and pkg_summary(5) and build an update
>         plan which is a list of packages user will have installed
>         after the installation. All nonlisted packages will be removed.
>         In order to build update plan DEPENDS and CONFLICTS fields
>         are used in order to make sure update plan is consistent.
>         Take a not that
>         pkg_summary(5) is the only remote file which is used for building
>         the update plan. Binaries to be installed are not available yet.
>       c) Proceed (yes/no)? Yes!
>       d) Download all required binaries and make sure
>          their checksums correspond to SHA512.
>       e) All packages from update plan are analysed for common files,
>          that is, real conflicts. This step would not be necessary if
>          either *all* conflicting packages had appropriate CONFLICTS
>          entries in pkg_summary(5) or pkg_summary(5) had a PLIST entries.
>          If conflicting packages
>          were found "nih install" immediately exits with error. Such a
>          failure means that update plan was in fact inconsistent.
>          Believe it or not such failures happen to me rather often.
>          This step is needed in order to avoid pkg_add failures in step f), 
> i.e.
>          breakage in the middle of update (the goal!).

This step is always necessary, unless you're going to neglect that user
can build and install packages with non-default options. Practice shows
that many people turn some on or off.

>       f) Install/removed packages in a correct order, that is run
>          pkg_add, pkg_delete. It's very bad if something fails here.
>          This is what I'm trying to avoid.
>
> I don't know exactly but I think pkgin does something similar.
> I'm not sure about step e), though.
>
> I hope the above explanation answers the question why pkg_add failures don't
> sasisfy needs of binary package management and why "it is too late..."
>
> Conclusion: information about conflicting packages should be available
> before downloading binaries, for example, in pkg_summary or nearby, but not
> inside the package.

This conclusion is wrong, and you've been told many times why.
You cannot eliminate step "e", and another twenty minutes are negligible
time when compared with time spent on update itself.

> ============================================================
> Part 2. How to represent information about conflicting packages.
>
> v1. Enrich pkg_summary(5) with PLIST. Bulk build tools can easily be adapted
>    to do so.  In this case pkgin, nih, etc.  will be able to use this
>    information (in addition to CONFLICTS for common conflicting config files,
>    generated by INSTALL script etc similar) for building a consistent update
>    plan.
>
>     Pros: PLIST generated by bulk build is always up-to-date and is
>       sufficient for package management.
>
>     Cons: pkg_summary(5) will be about 100Mb in size, or about 10Mb of
>       .bz2. Existing package managers need to be modified to use
>       PLIST.
>
> v2. Generate CONFLICTS entries automatically. Bulk build tools are able to do
>    this.
>
>     Pros: It's better than nothing. It's always up-to-date.
>     pkg_summary(5) is kept small.
>
>     Cons: This approach doesn't work because bulk build tools are
>     limited to packages available at build time.  Only CONFLICTS like
>     pkg-x.y.z may be generated, which is not enough for detecting
>     conflicts between *installed* packages and packages from the
>     repository. Autogenerated conflicts like pkg-[0-9]* are totally
>     wrong. Another problem is that registering conflicts automatically
>     hides the problem.  If new conflicts appear we have no any chance
>     to detect them and (probably) fix.
>
> v3. Manual CONFLICTS in Makefiles and therefore in pkg_summary(5)
>    (what we currently do).
>
>     Pros: pkg_summary(5) is kept small. Existing package managers work as
>     is.  Provided that *all* conflicts are registered (that is known and
>     seen) every new conflict appeared in pkgsrc can easily be analysed for
>     bugs in packaging as soon as it appears (it happens rather often that
>     conflicts are unintentional and should be fixed, I already showed a few
>     examples in this thread and there are others).
>
>     Cons: Joerg complained manual CONFLICTS are fragile and it's hard
>     to keep them up-to-date.  In this thread I proposed a solution for
>     this -- distbb bulk build reports. In order to make this work, a
>     number of unregistered conflicts should be significantly reduced
>     from ~800 to 0. Obviously this requires some amount of work and
>     time.  *I* will do this.  Then everything will be trivial.

v4. Manual conflicts in separate file for those who insist on
maintaining this sort of information.

> ============================================================
> Part 3. What to do.
>
> Unless anyone demonstrate a better idea I consider v3 a better plan
> and continue registering CONFLICTS manually. Also, *I* will track
> all unexpected conflicts in the future and keep them up-to-date
> with a help of distbb.

If your tool does need this information, you can maintain it separatly.

> (optional) provide pkg_summary_xxx = pkg_summary + PLIST (distbb and
> pbulk again).  Two applications: see above; pkg_summary_xxx can be
> used searching by filename.


-- 
HE CE3OH...