Re: postgresql packages, PG_SUBPREFIX and CONFLICTS

[Aleksey Cheusov <cheusov%tut.by@localhost>]

Unfortunately discussion in this thread came to a standstill.  Instead of
continuing repeating the same useless questions/answers again and again I
decided to accumulate significant parts in one place together with some
background information, just to make sure that everyone reading this
understands what I'm talking about and to avoid misunderstanding.  This
email will probably not very short, sorry.

============================================================
Part 1. Why information about conflicting packages is important
(regardless of how this information is stored and where).

- When pkg_add is given a package having files that belong to
  already installed package, it exits with error.

- When pkg_add is given a package that CONFLICTS with installed
  packages, it fails with error.

- Being a low level package management utilities pkg_* are good enough for
  managing a few number of installed packages but it's not appropriate tool
  for managing hundreds of installed packages.  This is why pkg_chk, pkgin
  and some others were created.  This is why I created pkgtools/nih that
  works on top of pkg_install and uses pkg_summary(5).

- For many years we have pkg_summary(5) that contains all meta-information
  about packages in a "repository" including information about conflicting
  package.  According to *existing* documentation CONFLICTS entry is
  responsible for this.  Take a note that pkg_summary(5) does *NOT* include
  PLIST entries, so CONFLICTS is the only place in pkg_summary(5) where
  conflicting packages are registered.

- One of the main goal of nih is to avoid breakages in the middle of
  installation process. To achive this goal
  "nih refresh; nih install list_of_packages" make the
  following steps:
      a) Download and unpack pkg_summary.gz and SHA512.gz.
      b) Analyse installed packages and pkg_summary(5) and build an update
        plan which is a list of packages user will have installed
        after the installation. All nonlisted packages will be removed.
        In order to build update plan DEPENDS and CONFLICTS fields
        are used in order to make sure update plan is consistent.
        Take a not that
        pkg_summary(5) is the only remote file which is used for building
        the update plan. Binaries to be installed are not available yet.
      c) Proceed (yes/no)? Yes!
      d) Download all required binaries and make sure
         their checksums correspond to SHA512.
      e) All packages from update plan are analysed for common files,
         that is, real conflicts. This step would not be necessary if
         either *all* conflicting packages had appropriate CONFLICTS
         entries in pkg_summary(5) or pkg_summary(5) had a PLIST entries.
         If conflicting packages
         were found "nih install" immediately exits with error. Such a
         failure means that update plan was in fact inconsistent.
         Believe it or not such failures happen to me rather often.
         This step is needed in order to avoid pkg_add failures in step f), i.e.
         breakage in the middle of update (the goal!).
      f) Install/removed packages in a correct order, that is run
         pkg_add, pkg_delete. It's very bad if something fails here.
         This is what I'm trying to avoid.

I don't know exactly but I think pkgin does something similar.
I'm not sure about step e), though.

I hope the above explanation answers the question why pkg_add failures don't
sasisfy needs of binary package management and why "it is too late..."

Conclusion: information about conflicting packages should be available
before downloading binaries, for example, in pkg_summary or nearby, but not
inside the package.

============================================================
Part 2. How to represent information about conflicting packages.

v1. Enrich pkg_summary(5) with PLIST. Bulk build tools can easily be adapted
   to do so.  In this case pkgin, nih, etc.  will be able to use this
   information (in addition to CONFLICTS for common conflicting config files,
   generated by INSTALL script etc similar) for building a consistent update
   plan.

    Pros: PLIST generated by bulk build is always up-to-date and is
      sufficient for package management.

    Cons: pkg_summary(5) will be about 100Mb in size, or about 10Mb of
      .bz2. Existing package managers need to be modified to use
      PLIST.

v2. Generate CONFLICTS entries automatically. Bulk build tools are able to do
   this.

    Pros: It's better than nothing. It's always up-to-date.
    pkg_summary(5) is kept small.

    Cons: This approach doesn't work because bulk build tools are
    limited to packages available at build time.  Only CONFLICTS like
    pkg-x.y.z may be generated, which is not enough for detecting
    conflicts between *installed* packages and packages from the
    repository. Autogenerated conflicts like pkg-[0-9]* are totally
    wrong. Another problem is that registering conflicts automatically
    hides the problem.  If new conflicts appear we have no any chance
    to detect them and (probably) fix.

v3. Manual CONFLICTS in Makefiles and therefore in pkg_summary(5)
   (what we currently do).

    Pros: pkg_summary(5) is kept small. Existing package managers work as
    is.  Provided that *all* conflicts are registered (that is known and
    seen) every new conflict appeared in pkgsrc can easily be analysed for
    bugs in packaging as soon as it appears (it happens rather often that
    conflicts are unintentional and should be fixed, I already showed a few
    examples in this thread and there are others).

    Cons: Joerg complained manual CONFLICTS are fragile and it's hard
    to keep them up-to-date.  In this thread I proposed a solution for
    this -- distbb bulk build reports. In order to make this work, a
    number of unregistered conflicts should be significantly reduced
    from ~800 to 0. Obviously this requires some amount of work and
    time.  *I* will do this.  Then everything will be trivial.

============================================================
Part 3. What to do.

Unless anyone demonstrate a better idea I consider v3 a better plan
and continue registering CONFLICTS manually. Also, *I* will track
all unexpected conflicts in the future and keep them up-to-date
with a help of distbb.

(optional) provide pkg_summary_xxx = pkg_summary + PLIST (distbb and
pbulk again).  Two applications: see above; pkg_summary_xxx can be
used searching by filename.