Subject: readdir ops of linux emul and nfs
To: None <tech-kern@netbsd.org>
From: enami tsugutomo <enami@but-b.or.jp>
List: tech-kern
Date: 10/13/2003 22:20:54
On the japanese local mailing lists, there was a report that some
directory operation fails if it is linux emulation and on nfs file
system.

For example,

    # /emul/linux/sbin/ldconfig -r /emul/linux/
    # /emul/linux/sbin/ldconfig -r /emul/linux/ -p

gives too few entries.

This is because,

	(1) It looks like linux binary expects d_off to fits signed
            32bit value.  And it signal an error if the value didin't
            back to extend to 64bit value.  Since I just read this
            code roughly, I may be wrong though (glibc is hard to
            read).

	(2) If NFS server is NetBSD, it encodes directory cookie as
            XDR hyper.

	(3) If NFS client is NetBSD, it partially decodes directory
            cookie if server is NetBSD.  I.e, only swap high and low 4
            bytes set of XDR hyper.

	(4) As a result, linux binary sees byte swapped 32bit direcory
            cookie, and if MSB of the swapped 32bit value is set, the
            biary thinks it overflows.

The attached diff fixes NetBSD server (and some other unixens like
tru64 or solaris) - NetBSD client case (the patch effectively decodes
full XDR hyper).  But I'm not sure how the other server encodes
directory cookies.  And I'm not sure if decoding this cookie in client
side is really good thing.

Thoughts?

enami.

#
# possibily, *_hyper macros can be used, but it's just style matter.
#
Index: xdr_subs.h
===================================================================
RCS file: /cvsroot/src/sys/nfs/xdr_subs.h,v
retrieving revision 1.14
diff -u -r1.14 xdr_subs.h
--- xdr_subs.h	7 Aug 2003 16:33:58 -0000	1.14
+++ xdr_subs.h	13 Oct 2003 12:51:50 -0000
@@ -57,12 +57,13 @@
  * are just here to attempt to keep information within 32 bits. And
  * make things look better. See nfs_cookieheuristic.
  */
-#define fxdr_cookie3(v) (((off_t)((v)[0]) << 32) | ((off_t) (v)[1]))
+#define fxdr_cookie3(v) \
+	((fxdr_unsigned(off_t, (v)[0]) << 32) | fxdr_unsigned(off_t, (v)[1]))
 #define fxdr_swapcookie3(v) (((off_t)((v)[1]) << 32) | ((off_t) (v)[0]))
 
 #define txdr_cookie3(f, v) { \
-	(v)[1] = (u_int32_t)((f) & 0xffffffffLL); \
-	(v)[0] = (u_int32_t)((f) >> 32); \
+	(v)[1] = txdr_unsigned((f) & 0xffffffffLL); \
+	(v)[0] = txdr_unsigned((f) >> 32); \
 }
 #define txdr_swapcookie3(f, v) { \
 	(v)[0] = (u_int32_t)((f) & 0xffffffffLL); \