NFS: open(...O_CREAT...) fails, yet file is created

[Edgar Fuß <ef%math.uni-bonn.de@localhost>]

I noticed a strange behaviour of NetBSD as an NFS server:

If you open(..., O_CREAT|O_EXCL, S_ISGID) in a directory mounted over NFS and 
belonging to a group you're not a member of, the open() fails with EPERM. 
However, the file is created nevertheless. On a local fs, the call just fails 
with no file being created.

I'm unsure as whether this behaviour (creating the file) is correct. At least, 
it's somewhat surprising.
I guess it may be a result of the whole open() operation resulting in several 
NFS RPC, some of which succeed and some fail.

I've tried to boil down this as far as possible: O_EXCL is necessarry and I 
couldn't find another reason for the open() to fail resulting in this 
behaviour. It seems to appear only with the combination of implicit group 
membership assignement, the issuer not belonging to this group and S_ISGID.

For us, it causes Adobe Reader 9.0 to fail in strage ways ("the document has 
been changed") for files with Reader Extensions enabled. There seems to be a 
bug in Adobe Reader causing it to open 
~/.adobe/Acrobat/9.0/AdobeIDataSync/CDataSync_AddressBook with some 
uninitialized mode which happens to have S_ISGID set. Our user's home 
directories (on a NetBSD NFS server) have their group set to a "home" group 
with no members.

Any comments?