kern/42484: tmpfs problem: make: Cannot open `.' (Permission denied)

[Thomas Klausner <wiz%NetBSD.org@localhost>]

>Number:         42484
>Category:       kern
>Synopsis:       tmpfs problem: make: Cannot open `.' (Permission denied)
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Dec 20 15:00:00 +0000 2009
>Originator:     Thomas Klausner
>Release:        NetBSD 5.99.22
>Organization:
Curiosity is the very basis of education and if you tell me that
curiosity killed the cat, I say only that the cat died nobly.
- Arnold Edinborough
>Environment:
        
        
System: NetBSD yt.nih.at 5.99.22 NetBSD 5.99.22 (YT) #29: Sat Dec 12 01:20:02 
CET 2009 wiz%yt.nih.at@localhost:/archive/cvs/src/sys/arch/amd64/compile/obj/YT 
amd64
Architecture: x86_64
Machine: amd64
>Description:
I'm regularly running bulk builds of about 1200 packages completely in
a tmpfs on 5.99.22/amd64.

tmpfs on /home/wiz/sandbox type tmpfs (local)
/bin on /home/wiz/sandbox/bin type null (read-only, local)
/sbin on /home/wiz/sandbox/sbin type null (read-only, local)
/lib on /home/wiz/sandbox/lib type null (read-only, local)
/libexec on /home/wiz/sandbox/libexec type null (read-only, local)
/usr/X11R7 on /home/wiz/sandbox/usr/X11R7 type null (read-only, local)
/usr/bin on /home/wiz/sandbox/usr/bin type null (read-only, local)
/usr/games on /home/wiz/sandbox/usr/games type null (read-only, local)
/usr/include on /home/wiz/sandbox/usr/include type null (read-only, local)
/usr/lib on /home/wiz/sandbox/usr/lib type null (read-only, local)
/usr/libdata on /home/wiz/sandbox/usr/libdata type null (read-only, local)
/usr/libexec on /home/wiz/sandbox/usr/libexec type null (read-only, local)
/usr/share on /home/wiz/sandbox/usr/share type null (read-only, local)
/usr/sbin on /home/wiz/sandbox/usr/sbin type null (read-only, local)
/var/mail on /home/wiz/sandbox/var/mail type null (read-only, local)
/archive/cvs/src on /home/wiz/sandbox/usr/src type null (read-only, local)
/archive/cvs/pkgsrc on /home/wiz/sandbox/usr/pkgsrc type null (local)
/archive/cvs/xsrc on /home/wiz/sandbox/usr/xsrc type null (read-only, local)
/disk/1/archive/packages/5.99.22 on /home/wiz/sandbox/packages type null (local)
/disk/1/archive/distfiles on /home/wiz/sandbox/distfiles type null (local)

Sometimes single packages or a few packages immediately after another
fail with "make: Cannot open `.' (Permission denied)". Usually, during
configure, but sometimes during the build as today in kdenetwork4:

[ 72%] Generating smsadd.cpp
[ 72%] Built target iris_kopete
[ 72%] Building CXX object 
kopete/protocols/bonjour/CMakeFiles/kopete_bonjour.dir/bonjouraddcontactpage.o
[ 72%] Generating smsadd.moc.cpp
[ 72%] Building CXX object
+kget/transfer-plugins/bittorrent/CMakeFiles/kget_bittorrentfactory.dir/libbtcore/torrent/torrentcontrol.o
[ -3%] Building CXX object 
kopete/protocols/testbed/CMakeFiles/kopete_testbed.dir/testbedaccount.o
Linking CXX shared module ../../../lib/kopete_wlm.so
Scanning dependencies of target kopete_sms
[ -3%] Building CXX object 
kopete/protocols/groupwise/CMakeFiles/kopete_groupwise.dir/gwconnector.o
make: Cannot open `.' (Permission denied)
--- kopete/protocols/sms/CMakeFiles/kopete_sms.dir/all ---
*** [kopete/protocols/sms/CMakeFiles/kopete_sms.dir/all] Error code 1
A failure has been detected in another branch of the parallel make

I think this might be a bug in tmpfs, since I've never seen this when
I was using ffs as an underlying file system for the sandbox.

The problem is, that, while the problem happens about in every second
or third bulk build I run, I have no sure way to reproduce it. Perhaps
it's a timing or locking related problem.

It usually causes a few packages in a row to fail, but then the build
continues and finishes with most packages built.

On stdout, I see:

[1120/1122] Starting build of  trac-0.11.5
cd: can't cd to /scratch/www/trac/work
*** Error code 2

Stop.
make: stopped in /usr/pkgsrc/www/trac
[1120/1122] Failed to build    trac-0.11.5


It offen happens during the build of openoffice3. The last time it
happened, the first error in the openoffice3 build log was:


--- NOOPTFILES OVER ---
dmake:  Error: --
`/scratch/misc/openoffice3/work/OOO310_m19/solver/310/unxbsdx3
.pro/inc/310minor.mk' not found, and can't be made
Running processes: 15
making .dpslo
Running processes: 16
/scratch/misc/openoffice3/work/OOO310_m19/starmath/source
dmake:  Error: --
`/scratch/misc/openoffice3/work/OOO310_m19/solver/310/unxbsdx3
.pro/inc/310minor.mk' not found, and can't be made
Running processes: 15
Running processes: 16
/scratch/misc/openoffice3/work/OOO310_m19/sw/source/core/SwNumberTree
if test -e ../../../unxbsdx3.pro/slo/slideshowviewimpl.o ; then touch
../../../u
nxbsdx3.pro/slo/slideshowviewimpl.obj ; fi
making .dpslo
/scratch/misc/openoffice3/work/OOO310_m19/solver/310/unxbsdx3.pro/bin/makedepend
 @/tmp/mk18527i > ../../../unxbsdx3.pro/misc/s_slideshowviewimpl.dpcc


The setup is as follows:
tmpfs sandbox containing all files for the bulk build.
12GB RAM in the machine, 24GB swap, so it should be sufficient.
Sometimes oo3 build works even in this setup. There are complete
bulk build runs where it doesn't occur.

There are enough inodes available:
tmpfs          34408948    1312212   33096736   3%    10564 162937777 0% 
/home/wiz/sandbox

>How-To-Repeat:
Run a limited bulk build as described above and put in some big
packages like openoffice3.
>Fix:
        

>Unformatted: