Improvement on reducing dependencies

["Johnny C. Lam" <jlam%NetBSD.org@localhost>]

# I'm resending this mail again without the large attachment.
# I apologize if you are receiving this message twice.

pkgsrc does a step called "reducing dependencies" as part of
creating a package.  This step involves trying to remove
extraneous dependencies from the dependency list before it is
embedded into the package.  For example, in the following
example, the first two dependencies are not needed:

    DEPENDS+= pkg>=1.0:../../category/pkg
    DEPENDS+= pkg>=2.0:../../category/pkg
    DEPENDS+= pkg>=3.0:../../category/pkg

Conceptually, each pattern represents a set of valid packages,
and the reduction step is taking the intersection of those sets.

I've enhanced the script that performs this reduction to
understand all patterns of the form:

    pkg>=1.0[<2.0]

The relational operators can be either open or closed ("or equal
to").  With the new script, the following dependency list is
reduced to a single dependency:

    DEPENDS+= pkg>=1.0:../../category/pkg
    DEPENDS+= pkg>=2.0:../../category/pkg
    DEPENDS+= pkg>=3.0:../../category/pkg
    DEPENDS+= pkg<=4.0:../../category/pkg
    DEPENDS+= pkg<4.0:../../category/pkg

This is reduced to:

    DEPENDS+= pkg>=3.0<4.0:../../category/pkg

I've tested the new scripts across almost every package in pkgsrc
by running "make show-depends" using both the old script and the
new script.  The following is the list of improved patterns
produced by the new reduction script:

    MesaLib7>=7.11.2<10:../../graphics/MesaLib7
    apache>=2.2.31nb2<2.3:../../www/apache22
    apache>=2.4.18nb1<2.5:../../www/apache24
    eog>=2.32.1nb24<3:../../graphics/eog
    evince>=2.32.0nb54<2.33:../../print/evince
    evince>=2.32.0nb54<3:../../print/evince
    gedit>=2.30.4nb27<3:../../editors/gedit
    gegl>=0.2.0nb22<0.3.0:../../graphics/gegl0.2
    gtksourceview2>=2.10.5nb28<3:../../x11/gtksourceview2
    gtksourceview>=1.8.5nb36<2:../../x11/gtksourceview
    libgdata>=0.6.6nb22<0.16:../../net/libgdata0.6
    libgee0.6>=0.6.5nb1<0.8:../../devel/libgee0.6
    lua51>=5.1.5<5.2:../../lang/lua51
    mono>=2.10.9nb21<3:../../lang/mono2
    mysql-client>=5.1.72nb6<5.5:../../databases/mysql51-client
    mysql-client>=5.5.48nb1<5.6:../../databases/mysql55-client
    mysql-client>=5.6.29nb1<5.7:../../databases/mysql56-client
    pari>=2.3.5nb5<2.4:../../math/pari23
    perl>=5.26.0<5.28.0:../../lang/perl5
    qwt>=5.2.3nb1<6:../../x11/qwt-qt4
    xulrunner192>=1.9.2.28nb36<2.0:../../devel/xulrunner192

Other redundant patterns that remained when using the old script
are now removed, and all other patterns that aren't understood
are still passed through without changes.  The full diffs for
"make show-depends" output across all packages are available at:

    http://www.NetBSD.org/~jlam/show-depends.log

The new script also fixes a bug in the way that version strings
were matched in the following case if ${LOWER} included a
non-numeric string, e.g., "1.0beta3":

    DEPENDS+= pkg>=${LOWER}<${UPPER}:../../category/pkg

This bug exists in the 2017Q3 release branch for packages that
include x11/fltk/buildlink3.mk, which has this line:

    BUILDLINK_API_DEPENDS.fltk+= fltk>=1.1.5rc1<1.3

For example, running "make show-depend" in audio/csound5 causes
the following to be emitted to standard error:

    sh: cannot open 1.3: no such file

I think a pullup of the new reduce-depends.awk script to the
2017Q3 branch is warranted.

Regards,
-- 
Johnny C. Lam
jlam%NetBSD.org@localhost