Re: wip/glusterfs to pkgsrc filesystems/glusterfs

[Alexander Schreiber <als%thangorodrim.ch@localhost>]

On Wed, Mar 01, 2023 at 07:54:59PM -0500, Greg Troxel wrote:
> Alexander Schreiber <als%thangorodrim.ch@localhost> writes:
> 
> > I've successfully tested (built & run) this on all 64bit platforms I have
> > available for testing: aarch64, amd64, sparc64. It cannot run (but can be
> > built) on any 32bit platform, since the code tries to atomically update
> > a 64bit value eventually (userspace-rcu), which is not something that is
> > going to work on 32bit (unless you got a weird one). New 32bit platforms
> > are unlikely to spring up, but pkgsrc help already mentions e.g. riscv64.
> >
> > I would prefer a cleaner ONLY_FOR_64BIT (or similar), but that doesn't
> > appear to exist.
> 
> Is this new in glusterfs, that it doesn't work on i386, or just new that
> you figured it out and are documenting it?  For distributed/remote
> filesystems, I tend to feel its important for them to run everywhere
> since in my view that's a major part of suitability -- but I realize
> that if upstream is 64-bit only, it's next to impossible to tilt against
> that.

Support for 32bit was effectively dropped by glusterfs a while ago,
there is https://github.com/gluster/glusterfs/issues/702 from 2019 that
starts proposing this. I know that glusterfs 8.2 (current pkgsrc) works
on NetBSD/i386, but somewhere between 8.2 and 10.3 (the one I'm working
on for pkgsrc), 32bit support effectively disappeared. You can still
_build_ it on 32bit, but it will die with SIGILL very early after
startup due to
https://github.com/gluster/glusterfs/issues/3911#issuecomment-1451711686
(found via trawling through backtraces and then through left behind
WRKOBJDIR contents due to the way userspace-rcu is integrated at
build time).

There are some sporadic discussions about "hey, why does this crash on
32bit?", but I do not expect anyone to actually put any work into
making it work on !64bit again, partially because that would involve
some ugly hackery (a bunch of internal things went from 32 to 64 bit
and they do atomic updates on those).

Even so, there were reports (from older versions) that 32bit glusterfs
appears noticeably slower than 64bit.

My workaround for 32bit machines is to mount the glusterfs filesystem
from a 64bit machine via sshfs from the 32bit machine. Not fast either,
but at last it works at all (yes, I still have some 32bit hardware in
active use with both NetBSD and Linux).

Kind regards,
            Alex.
-- 
"Opportunity is missed by most people because it is dressed in overalls and
 looks like work."                                      -- Thomas A. Edison