Subject: Re: CVS commit: pkgsrc/mk/bulk
To: Alistair Crooks <agc@pkgsrc.org>
From: Roland Illig <rillig@NetBSD.org>
List: pkgsrc-bulk
Date: 11/20/2005 12:42:26
Alistair Crooks wrote:
> On Sat, Nov 19, 2005 at 09:05:00PM +0100, Roland Illig wrote:
> 
>>Alistair G. Crooks wrote:
>>
>>>Module Name:	pkgsrc
>>>Committed By:	agc
>>>Date:		Sat Nov 19 19:21:47 UTC 2005
>>>
>>>Modified Files:
>>>	pkgsrc/mk/bulk: bsd.bulk-pkg.mk
>>>
>>>Log Message:
>>>Allow the bulk builds to work by adding security/audit-packages as a
>>>BULK_PREREQ package.
>>
>>This slows down the pkgsrc build on my machine by approximately 8 
>>seconds per package. Do you really want that?
> 
> 
> Bulk builds from scratch run for 5 days on a 3.6 GHz P4, 7 days on a
> 2.8 GHz P4.
> 
> Now *I* am not the one who made the changes which forced
> audit-packages to be installed, or SKIP_AUDIT_PACKAGES to be defined
> to "yes" before a bulk build run.

I'm sorry of not having noticed that earlier. Of course you were not 
involved, it had just looked to me as if. :(

> Altogether, there has been far too much upheaval over the last few
> days which was ill-considered, not wholly tested, and the drivers
> for which are still unclear to me.

If you mean most changes from me I can tell you what the drivers to them 
are:

I don't have confidence in shell programs that don't run in "set -e" 
mode. Ususally there is no error checking at all, not to mention error 
handling. So whenever something unexpected happens, the programs 
continue to run, leading to undefined behavior. With the "set -e" mode 
they just die. That's why I'm changing the programs into "set -e" mode.

The other thing is that I'm still a beginner regarding bulk builds. I'm 
trying to completely understand how they work. But some things make 
understanding harder than necessary. For example variable names and file 
names. The BROKENFILE variable is used for two completely different 
purposes, yet it's only one variable. The top level BROKENFILE is called 
".broken.html" by default, yet it does not contain a single HTML tag at 
all. Don't you find this confusing, too?

I'm trying to make the bulk build framework so consistent that it would 
be a pleasure for me to document it in the pkgsrc guide as a system that 
is well thought-out, consistent, robust, reliable, etc. It isn't yet. It 
just happens to work. I, however, want it to work because it is designed 
to work, not out of coincidence.

Some days ago I had written to Thomas in a private mail that I really 
don't like to document things which are awful to describe. At least not 
if I can change them. It's fun trying out how the various awk(1) 
implementations behave when fed with too much input (in the 
regress/tools package), as I know I cannot change them. But the bulk 
build framework is something I can change, and that's why I do it.

Roland