Subject: Re: Builk build vs. p5-Test-Harness
To: Krister Walfridsson <cato@df.lth.se>
From: Dan McMahill <dmcmahill@NetBSD.org>
List: tech-pkg
Date: 06/22/2004 21:03:45
On Mon, Jun 21, 2004 at 01:16:04PM +0200, Krister Walfridsson wrote:
> The problem
> -----------
> The bulk build refuse to build some packages because it registers
> spurious dependencies.
> 
> The problem stems from lines of the form:
> 
>    DEPENDS+= {perl{,-thread}>=5.8.3,p5-Test-Harness-[0-9]*}:../../devel/p5-Test-Harness
> 
> This makes the bulk build register ../../devel/p5-Test-Harness as a
> dependency, even though it is the perl package that will be used in
> the build.  This is normally not a problem, since the right thing
> happens when the package is built.  But the bulk build does
> recursively mark all depending packages as broken when a package
> fails to build (which in the current pkgsrc gives us 51 failures
> when devel/p5-Test-Harness fails to build...)
 

So the issue is that p5-Test-Harness is needed if you were using
perl-5.6 but if you have perl-5.8, you do not need p5-Test-Harness?


> The proposed solution
> ---------------------
> The current bulk build process of building a package is roughly as:
> 
>   for each dependency
>     install dependency
>   build
>   if "build failed"
>     mark packages depending on this package as broken
> 
> I suggest changing this to:
> 
>   for each dependency
>     if "dependency is broken"
>       fail
>     install dependency
>   build
> 

I much much rather somehow improve how the bulk code deals with
these dependencies which depend on the order in which packages
are installed.

The current approach of marking all pkgs which depend on a 
failing package as broken was added because it really really
speeds up the build in some cases.  What led me to implement
that was the following case:

libfoo fails.

kdeblah depends on lots and lots of pkgs.  The build system
goes along installing all these different dependencies (each
time with more make calls) and finally after installing 50
dependency pkgs, it finds that the last one needed (libfoo)
is broken.  Now you deinstall lots, and repeat the exercise
many more times.

Depending on the package which failed, the bulk build can
potentially spend lots and lots of time installing dependencies
for packages which are doomed to fail.  

If something like perl, libtool, or libpng fails, the
reduction in time is dramatic.

It seems to me that the root problem is that we potentially
have more than one valid dependency tree for a given package.
Its some what goofy in that the dependency tree you get will
depend on the order in which you happened to build packages.
I'd like to see the bulk build system somehow decide which
it will be up front.  Then the problem you're having here
should go away.

-Dan


--