Subject: buildlink framework has problems with parallel make
To: None <tech-pkg@netbsd.org>
From: Georg Schwarz <georg.schwarz@freenet.de>
List: tech-pkg
Date: 04/16/2005 21:41:30
I am using pkgsrc on a Dual-Pentium machine running Linux. I have added
to my mk.conf the following line:

MAKE_FLAGS+=-j2

So far this seemed to work fine with most packages (it has only an
effect when a packages uses GNU make, but many do) except for a few ones
whose Makefiles are buggy.
Under heavy load I have just come across something that looks like a
general issue of the buildlink wrapper framework with parallel make
jobs:

...
/usr/src/pkgsrc/pkgsrc/wip/linphone/work/.wrapper/bin/gcc:
/usr/src/pkgsrc/pkgsrc/wip/linphone/work/.wrapper/tmp/cache: line 59:
syntax error near unexpected token `-DHA'
/usr/src/pkgsrc/pkgsrc/wip/linphone/work/.wrapper/bin/gcc:
/usr/src/pkgsrc/pkgsrc/wip/linphone/work/.wrapper/tmp/cache: line 59:
`-DHA'
....

The respective line now reads:

-DHAVE_CONFIG_H) arg=-DHAVE_CONFIG_H; split_arg=no; cachehit=yes ;;

interestingly the next line is exactly identical.
It looks like two processes were in parallel coming across the argument
-DHAVE_CONFIG_H.
When one process was looking it up, the first one had already started
writing to the cache file, and the second one just read -DHA (the rest
was not yet written/flushed/whatever).

The above error messages do not cause buildlink to terminate. I am not
sure if they do have any effect such as the scripts skipping some vital
step.
If not, I think have douplicate line in the cache is not nice but should
not cause any harm, should it? Then the above message should rather be
phrased as a warning. Would that be possible?
Is it ensured that two concurrent build jobs do write their findings in
the cache in separate lines and do not mess up things by having their
lines mixing up in one line?
 

-- 
Georg Schwarz    http://home.pages.de/~schwarz/
 georg.schwarz@freenet.de     +49 178 8545053