Subject: Looking for some assistance with bulk scripts
To: None <>
From: Florian Heigl <>
List: tech-pkg
Date: 06/05/2007 17:46:34

I tried a first bulk build on HP-UX back in April, with not too much
success, but I will have time to resume on that during the next weeks.
I should add that, even with X support missing back then I build
around 800 packages simply by trying to build everything in sight. :>

I remember a few little issues:

The usage of bulk cache (.dependstree and .index):
1. the tsort call bails because the tsort on HP-UX does not know a -q
flag, and I've not found the actual call in the bulk scripts so I
could remove -q; from what i understood it simply means 'quiet'
2. if i could remove it (i know from manually testing) the tsort will
output a comment like 'tsort: odd data' I do not really understand,
but i suppose the return code won't be 0 which might add some trouble.
The easiest way would be using a NetBSD source tsort, but I don't
really know where to find it.
3. if possible i want to preserve those .files as every run of
creating the cache takes roughly 16 hours. (it's about 12000 bmake
calls after all), but from what i saw in the documentation i can
either not use the cache (even slower in the long run) or regenerate
it (see before)
4. personally I think it would be helpful if the pkgsrc cvs would
contain a daily generated set of these cache files. sure there are
purposes, where this won't be sufficient, but if generating the cache
takes almost a day on a moderately sized host there is not much to
lose in my opinion.

I really need ANY advice to cut down the times per test to something
like 30 minutes :)
I've done all I can to tune the underlying system, but there's not
really anything left I could do.

- I tried to work with a subset of SITE_PKGS (or something like that)
for testing, i.e. to say the buildchain works once quite stable when
www/apache can be built.
when testing, it appeared that dependencies like bzip would be
noticed, but not built early on in the bulk phase.
for that reason i had added them to BULK_PREREQS, in that case they
were built on time but removed later on and not added back.
This was a bit weird and i'd plead for someone experienced with bulk
builds on non-NetBSD OSses look at my logs.

- also i'm looking for the correct way to integrate ccache from pkgsrc
as soon as possible.
- Later on I also would like to add distcc from pkgsrc into the
buildchain as I can't add more raw cpus but have two more systems with
a gigabit backend just for buildpurposes.

Maybe some of the Tru64/Solaris people still have some notes from
their bulk builds I could look at?
On NetBSD or even Linux most of these issues won't show due to the
more 'expected' set of utilities in the base operating system.


'Sie brauchen sich um Ihre Zukunft keine Gedanken zu machen'