Re: build.sh: set creation is serialized

[Alan Barrett <apb%cequrux.com@localhost>]

Please don't misunderstand me: I am not arguing that the set
serialisation should remain as it is.  I think sets should be built in
parallel by default, with an option to build them in series for people
who find that useful.

On Thu, 01 May 2008, James Chacon wrote:
> Sure...This is no different than if someone was building with any
> other slow medium for disk. Should we slow down all parts of the build
> which might not scale to everyone's hardware?

The problem is in NetBSD itself, in the way it shares disk I/O across
multiple competing processes.  No matter how slow the hardware is, the
same work needs to be done: read thousands of files, create several
tarballs (8 to 14, depending on whether or not X is built), compress
them, and write them to disk.  Whether that's done in 8 to 14 parallel
tasks or 8 to 14 serialised tasks makes no difference to the amount
of real work done.  But, due to bugs in NetBSD (at least in versions
of NetBSD that were in use when this workaround was committed to the
Makefile), it could make a huge difference to the run time.

> Remember, using -j was a conscious choice. If it doesn't scale for
> your hardware, don't use it.

You have said this sort of thing several times, so I felt the need to
respond this time.  The "build sets in series" hack is not a workaround
for slow hardware, it's a workaround for some really bad NetBSD
behaviour (which might or might not have improved over the past year).

On my current laptop (less than 2 years old, Intel "Centrino Duo"
dual-core CPU, 1GB memory, running NetBSD/i386-4.99.60), building
sets in parallel on two CPUs is faster than building in series on one
CPU.  Using only one CPU, build time is about the same in series or
in parallel (building in parallel may be a few seconds slower, but
the difference is close to the noise floor).  I don't have similar
measurements from last year's version of NetBSD-current.  However, the
poor disk I/O scheduling remains a huge problem in NetBSD-4.99.60.

Interactive performance whild building sets, whether in series or in
parallel, is terrible:  I can read mail and use vi in an xterm, but
when I click stuff in a graphical web browser I have to wait *tens of
seconds* for the browser to open a new tab or window, or even just to
switch between tabs; sometimes the mouse freezes for 30 seconds; all
the time, my window manager's CPU usage monitor shows one CPU mostly
busy and the other CPU mostly idle.  Building sets in parallel uses both
CPUs, and completes faster, but is worse for interactive performance
then building them in series.  If I disable one of the CPUs and build
sets in parallel ("-j4") on one CPU, then it takes about the same time
as building in series on one CPU (maybe a few seconds longer, averaged
over several runs), provided the machine is otherwise idle.  Building in
parallel, even on two CPUs, feels worse for interactive performance than
building in series.

--apb (Alan Barrett)