tech-pkg archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: multi-variant packages and bulk builds



On Sun, Jul 20, 2008 at 03:33:56PM +0300, Aleksey Cheusov wrote:
> In pkgsrc there are a number of packages (category/package) that can
> be built with different PKGNAME depending on a value of some special
> variables. For example, www/ap2-* packages, */py-* and many others.
> 
> AFAIU in order to buils these packages pbulk uses hardcoded list of
> special variables in mk/pbulk/pbulk-index.mk. For multi-variant
> packages my distbb currently builds only package, the default one, but
> I don't thing the way pbulk works is the right way.
> 
> Instead, I propose to introduce one special variable, e.g. VARIANTS,
> and to support it in all multi-variant packages.
> 
> Example:
>   cd /usr/pkgsrc/www/py-clearsilver
>   make show-var VARNAME=VARIANTS
>   > PYTHON_VERSION_DEFAULT=15,20,21,22,23,25
>   cd /usr/pkgsrc/www/ap2-auth-external
>   make show-var VARNAME=VARIANTS
>   > PKG_APACHE_DEFAULT=apache2,apache22
>   cd /usr/pkgsrc/www/ap2-python
>   > PKG_APACHE_DEFAULT=apache2,apache22 PYTHON_VERSION_DEFAULT=23,24,25
> 
> Using cartesian product of all supported values for all listed
> variables, bulk build software can easily build all variants all such
> packages. I hope the idea is clear.
> 
> Of course the code that sets VARIANTS variable should use appropriate
> xxx_ACCEPTED variables.
> 
> One of the main benefit of this approch is that pkgsrc itself and bulk
> build programs become more isolated and independant from each other
> because pkgsrc API becomes clearer and easier.
> 
> Any thoughts?

You are mixing two different problems:

1.  Knowing exactly all the parameters that influence the build of a
    given package.

    The {PYTHON,PHP,APACHE}_ACCEPTED variables are used to know if you
    can build a package in a given environment.  While there may be a
    point in having a framework to handle multiple versions of packages,
    the goal is not to make a list of all possible combinations.

    Either the user controls his environment and sets the appropriate
    variables to set the specific version of such pieces of software she
    wants, or lets pkgsrc decide, using the _DEFAULT values.

    Back in pkgviews days, you could have multiple versions of a package
    in the same installation, but that approach ultimately failed
    because in the end it wasn't that much simpler than having several
    LOCALBASEs.

    What I feel is missing currently is a way for the user to retrieve
    the information about version dependencies.  A lot of effort was put
    in the options framework with great success, and my humble opinion
    is that such information ("I will build with apache22, per your
    environment" or "I will build with php5, per pkgsrc's default")
    belongs in "make show-options", even though this is slightly
    different from a regular option.

2.  Providing binary packages for all the world and its mother.

    Here I think both pbulk (iow, Joerg) and you are wrong.  pkgsrc's
    flexibility is both its greatest asset and its curse.  From the
    administrator's point of view, the options framework and the
    support for different versions of PHP, Apache and Python is an
    amazing feature.

    However, it is also part of the reason why our binary package
    distribution (actually, the lack of a proper one) is pkgsrc's
    greatest weakness.  Distributing the results of a bulk build is
    easy, but maintaining a binary distribution over the time is where
    we're not to the level of competitors.

    A binary packages distribution has to be seen as a whole, with a
    set of default options and settings that are consistent across all
    packages.  Red Hat, Debian and whoever else provide you with a
    static binary distribution, because that's the only way they can
    sanely maintain and support it.

    That said, there are different uses of bulk package building:  I
    covered the user-oriented one, for which I oppose the idea of
    building multiple versions of a package, with the exception of the
    case where they don't conflict, in which case they should have their
    own package directory anyway.  As for packages depending on any
    version of another package, they should only be built for the pkgsrc
    default version.

    The other use of bulk building is for testing purposes, in which
    case your VARIANTS idea could have merit.  The problem is, if you
    want to test all building possibilities of a package, you also have
    to test all subsets of the options list from the options framework.

    I'll leave as an exercise to the reader to compute how many times
    mplayer and mencoder would be built if we made pbulk or any other
    bulk building system do all possible options combinations.

    The point is, if you really want to formalise that part, you'd
    better go all the way, and I really wish you good luck with that.
    As for me, I think pbulk is already exceeding its expectations in
    allowing such possibilities, and while pbulk can do whatever it
    wants in its world, there is no point is pushing that feature into
    pkgsrc.

-- 
Quentin Garnier - cube%cubidou.net@localhost - cube%NetBSD.org@localhost
"See the look on my face from staying too long in one place
[...] every time the morning breaks I know I'm closer to falling"
KT Tunstall, Saving My Face, Drastic Fantastic, 2007.

Attachment: pgpYrVUCpzFdY.pgp
Description: PGP signature



Home | Main Index | Thread Index | Old Index