Re: NTFS Mounting does not work reliably on current versions of NetBSD

["William D. Jones" <thor0505%comcast.net@localhost>]

I've done a bit of searching through the system logs, and I think I've found 
the culprit, by enabling perfused's debugging facilities:
"
Aug  3 06:00:00 rpi-ptrain newsyslog[1937]: log file turned over
Aug  3 06:00:00 rpi-ptrain syslogd[187]: restart
Aug  3 06:08:01 rpi-ptrain perfused: perfused_open_sock: setsockopt 
SO_SNDBUF to
540672 failed: No buffer space available
Aug  3 06:08:01 rpi-ptrain perfused: perfused_open_sock: setsockopt 
SO_RCVBUF to
540672 failed: No buffer space available
Aug  3 06:08:21 rpi-ptrain perfused: perfused_writeframe: send returned -1, 
errn
o = 40: Message too long
Aug  3 06:08:43 rpi-ptrain syslogd[187]: last message repeated 4548 times
Aug  3 06:10:43 rpi-ptrain syslogd[187]: last message repeated 25823 times
Aug  3 06:19:02 rpi-ptrain syslogd[187]: last message repeated 107306 times
Aug  3 06:19:02 rpi-ptrain perfused: perfused_open_sock: setsockopt 
SO_SNDBUF to
540672 failed: No buffer space available
Aug  3 06:19:02 rpi-ptrain perfused: perfused_open_sock: setsockopt 
SO_RCVBUF to
540672 failed: No buffer space available
Aug  3 06:19:02 rpi-ptrain perfused: perfused_writeframe: send returned -1, 
errn
o = 40: Message too long
Aug  3 06:19:43 rpi-ptrain syslogd[187]: last message repeated 8765 times
Aug  3 06:21:13 rpi-ptrain syslogd[177]: restart"

The short answer appears to be: The default TCP/IP buffer sizes for the RPI 
kernel are too low. According to the following forum post: 
https://forums.freebsd.org/viewtopic.php?&t=1528 and some Googling, the 
relevant kernel option/NetBSD analogue that should be changed is 
NMBCLUSTERS, as described here: 
http://www.netbsd.org/docs/kernel/#mclpool-limit

Incidentally, transmission-daemon fails to work for the same reason (no 
buffer space available), so I wonder if in general the Pi's buffer sizes 
should be increased? I can think of numerous uses besides my own experiences 
that would exhaust the defaults. Any suggestions/feedback before I modify 
any kernel configurations?

-----Original Message----- 
From: thor0505%comcast.net@localhost
Sent: Saturday, July 19, 2014 5:59 PM
To: current-users%netbsd.org@localhost
Subject: NTFS Mounting does not work reliably on current versions of NetBSD

Hello all (are we getting tired of me posting yet?),

This is a long post, but it requires some background:

Although I'm only capable of testing this on the Raspberry Pi for now, I 
can't duplicate this problem on Linux on the Pi. Therefore, I'll write to 
the mailing list for now to get some advice, and if this turns out to be a 
fuse/ntfs-3g bug, I'll submit a patch.

I need to mount a NTFS external drive from my Raspberry Pi. For those who 
have not been following my previous posts ('pkg_add packages for 
evbearmv6hf-el: "Cannot execute ELF binary"'), the Raspberry Pi cannot 
currently execute NetBSD packages from evbarm pkgsrc repositories, so I need 
to compile it manually. There were two options I had- compile directly from 
source, or use the pkgsrc to automate the process/apply patches.

This leads to the first issue... The pkgsrc version of fuse-ntfs-3g will not 
compile properly- specifically, due to the applied wrappers from -lfuse 
to -lrefuse. From my understanding, NetBSD provides its own fuse 
implementation- librefuse- which is essentially a wrapper for the original 
libpuffs implementation that solved the same problem as fuse. In 2011, a 
mailing list post brought to attention that then-current versions of 
lib-ntfs-3g required functions that librefuse did not supply, and Adam 
Hamsik provided a patch to add the missing functions. As of the February 15, 
2014 version of ntfs-3g, this patch is no longer sufficient:
gcc -DHAVE_CONFIG_H -I. -I.. -I/usr/pkg/include -DFUSE_USE_VERSION=26 -D_FILE_OFFSET_BITS=64 
-I/usr/local/include/fuse -I../include/ntfs-3g -O2 -I/usr/pkg/include -Wall 
-MT lowntfs_3g-lowntfs-3g.o -MD -MP -MF 
.deps/lowntfs_3g-lowntfs-3g.Tpo -c -o lowntfs_3g-lowntfs-3g.o `test -f 
'lowntfs-3g.c' || echo './'`lowntfs-3g.c
lowntfs-3g.c:31:27: fatal error: fuse_lowlevel.h: No such file or directory
#include lowlevel.h>
^
compilation terminated.
*** Error code 1

Stop.
make: stopped in 
/usr/pkgsrc/filesystems/fuse-ntfs-3g/work/ntfs-3g_ntfsprogs-2014.2.15/src
*** Error code 1

Stop.
make[1]: stopped in /usr/pkgsrc/filesystems/fuse-ntfs-3g
*** Error code 1

Stop.
make: stopped in /usr/pkgsrc/filesystems/fuse-ntfs-3g

I'm unaware if even more headers are required which are missing.

Therefore, I decided to compile both libfuse and ntfs-3g manually, linking 
against my new libfuse instead of librefuse. As long as perfused is ran 
beforehand (ntfs-3g will fail ENOBUFS otherwise), I am capable of mounting 
my external hard drive, accessing, reading and writing files! However, if I 
attempt to do large reads or writes, such as checking out a CVS repository, 
all disk activity ceases and the data transfer never finishes. BOTH CVS 
verbose and ntfs-3g with the debug option indicate that the transfer has 
halted- i.e. there is NO diagnostic output at all.

CTRL+C will get me back to the shell prompt, but ls will no longer return 
any directory entries on the mounted drive (indicating the drive is still 
not responding). Trying to unmount the drive will freeze the prompt, and 
while using SSH to reboot may work successfully, the reboot may freeze while 
waiting for the drive to respond, when it never does! The below shell script 
is what I use to show the behavior (run as root):
#!/bin/sh
NTFS3G_EXE=/bin/ntfs-3g
MNT_SRC=/dev/sd0e
MNT_TGT=/mnt/USBdisk
TEST_DIR=$MNT_TGT/test_dir

perfused
$NTFS3G_EXE $MNT_SRC $MNT_TGT
cd $MNT_TGT
mkdir -p $TEST_DIR
cd $TEST_DIR
env CVS_RSH=ssh cvs -z2 -d anoncvs%anoncvs.NetBSD.org@localhost:/cvsroot checkout -P 
pkgsrc
cd ~
umount $MNT_SRC #Incidentally, "device is busy" always the first umount. 
Second/third umount are okay.

To duplicate my environment, ./configure fuse-2.9.3 with "--exec-prefix=/ 
(default /usr/local --prefix is okay)", and make and make install. Then, 
./configure ntfs-3g_ntfsprogs-2014.2.15 (no options needed, default 
/usr/local --prefix is okay), and make and make install. ntfs-3g should be 
in /bin.


Is there any possibility this behavior is caused by linking against a fresh 
libfuse instead of linking against the librefuse provided with the NetBSD 
source? I'm considering recompiling the kernel with debugging options and 
using a kernel debugger for this- the ability for me to mount an NTFS drive 
is rather important and rather difficult to work around currently.

As always, thanks for the help!


Sincerely,

--
William D. Jones

--
William D. Jones
Rowan University | ECE | 2012
Member IEEE
Member Tau Beta Pi
thor0505%comcast.net@localhost
Message sent using 'Windows Live Mail' client.