Module Name:	pkgsrc
Committed By:	dmcmahill
Date:		Thu Feb  1 08:47:30 UTC 2001

Modified Files:
	pkgsrc/mk/bulk: bsd.bulk-pkg.mk build post-build

Log Message:
This commit incorporates several speed improvements which have been tested
over several bulk builds on an alpha:

- At the start of a bulk build, 4 files are created to allow fast lookup of
  various dependency tree things.  These files are
  .index       ==  maps package name (foo-2.3) to directory (bar/foo)
  .dependstree == contains the complete pkgsrc dependency tree in a tsort(1)
                  compatible format.
  .depends     == contains 1 line per package and lists all the build depends
                  for the package.
  .supports    == contains 1 line per package and lists all packages which
                  depend on this package.

- When a package fails to build, the list of all packages which depend upon
  it is read from .supports.  Each of those packages is immediately marked
  as broken.  This prevents us from trying to build those packages which can
  have significant overhead if lots of other depends are installed before the
  system notices the broken one.

  In addition, the post-build postprocessing will now indicate that a package
  is 'truely broken' (ie, bad PLIST, failed compilation) vs. a package which
  is broken because it depends on a failed package.  This assists in determining
  where to focus our efforts in fixing broken packages.

- In the old approach, all packages are removed after each one is built.  The
  purpose was to a) conserve disk space, b) verify that all required dependencies
  are in fact listed, and c) prevent conflicts.  The problem was that often times
  several packages in a row have similar depends.  For example we might want to
  build several perl packages in a row.  In the old approach, we would install perl,
  build the pkg, deinstall perl and continue with the next one.

  In the new approach, when it is decided that a pkg is out of date and should be
  rebuilt, the .depends file is used to obtain a list of pkgs we require.  Then
  and pkgs which are no longer needed are removed.  This helps to minimize the total
  number of pkg_add/pkg_delete's in a bulk build.

- Since the order of the build is controlled by the depends tree, all depends for a
  given package will have been built by the time the pkg in question needs them.
  If any of the depends failed to build, then the pkg which needs the failed one will
  have been marked as broken.  Given this, the complete depends list for a pkg is
  read from .depends and the depends installed via pkg_add rather than relying on
  recursive make calls to install the depends.

- while here, fix a few minor bugs
  x - don't leave .make log files around when the build succeeds
  x - make sure we refer to the correct report file in the email
  x - use '.order' for the build order file instead of '.l'
  x - use 'grep -c' instead of 'grep | wc -l'


To generate a diff of this commit:
cvs rdiff -r1.16 -r1.17 pkgsrc/mk/bulk/bsd.bulk-pkg.mk
cvs rdiff -r1.7 -r1.8 pkgsrc/mk/bulk/build
cvs rdiff -r1.8 -r1.9 pkgsrc/mk/bulk/post-build

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.