Re: kern/48449: y2038 shortcomings of ffsv1

[Robert Elz <kre%munnari.OZ.AU@localhost>]

The following reply was made to PR kern/48449; it has been noted by GNATS.

From: Robert Elz <kre%munnari.OZ.AU@localhost>
To: gnats-bugs%netbsd.org@localhost
Cc: 
Subject: Re: kern/48449: y2038 shortcomings of ffsv1
Date: Sat, 12 Jun 2021 18:42:01 +0700

     Date:        Fri, 11 Jun 2021 22:35:01 +0000 (UTC)
     From:        David Holland <dholland%netbsd.org@localhost>
     Message-ID:  <20210611223501.5DC9C1A923B%mollari.NetBSD.org@localhost>
 
   |  Your argument seems to be that ffsv1 should be deprecated entirely;
 
 Almost entirely.   Removed from the kernel (at about the end of this
 century -- none of us will ever see it happen... (probably))
 
   |  that's fine if we can pull it off, but given that among other things
   |  there are machines with ffs-reading firmware that don't understand
   |  ffsv2, probably isn't realistic.
 
 As I said, we can keep support for FFSv1 for this kind of use via FUSE,
 or otherwise (makefs would be sufficient too, for this only the firmware
 needs to be able to read the thing) with user level conversion that can
 be tailored to the needs, and do whatever is best to do, given the limitations.
 
 But trying to keep FFSv1 alive in general is folly -- remember that its
 limited time range is one of its lesser problems, it also has limited
 uid/gid ranges, and more importantly, limited block number ranges - it
 isn't quite the case yet that you cannot buy a disk small enough for FFSv1
 to work on it, but we're approaching that point.   And while partitioning
 can avoid that problem, only if the partitioning scheme works for the reason
 that you're needing FFSv1 in the first place - I am fairly sure nothing that
 has ffs reading firmware has GPT support in there, and disklabels (and MBR,
 but that tends not to be relevant) are also block number size limited, so
 they don't help either.   FFSv1 is simply doomed.
 
 We cope with keeping this stuff working (assuming that any of it will still
 exist to work in the next century - emulators can easily get given "updated
 firmware" in this area, so they can boot other ways than the genuine article).
 in the same way we keep support for a.out which this kind of firmware also
 tends to expect, not by keeping it as a first class citizen, but by making
 tools which support it in a somewhat hackish way.
 
   |  It's not hard to do this and do it in a way that's adequately
   |  recoverable/restartable. It's nowhere near as complicated as
   |  resize_ffs.
 
 ["this" is changing the time offset in a FFSv1 filesystem]
 
 It terms of implementation complexity, no, that's easy.   The problem is
 that this conversion requires touching every (allocated) inode in the
 filesystem, and that takes time (human measurable time) and so must be
 done in a way that's restartable.   The only way that really works for
 this is a "converted" bit in each inode (trying to do the inodes in
 ascending sequence and recording where we're up to has non-atomic update
 problems, so isn't safe).  The issue here is that FFSv1 inodes really
 have no suitable free bits (there are free bits in the flags word, but
 those aren't really suitable for this - though it could be forced in I
 guess).
 
 Incidentally there's another unbelievably gross hack that could be done
 to extend the time range of FFSv1 (from 136 years or whatever, to > 500
 years) ... each of the time fields is really 2 32 bit values.  One for the
 (old) time_t (seconds) which is the one that is causing all of these
 problems, the other is the nanoseconds field (seconds+nanoseconds == the
 timestamp).  One property of nanoseconds is that their range of values
 is 0..999999999 (they're never negative in this use) and 999999999
 only takes 30 bits to represent.   That means we have two currently
 always zero bits associated with each time field in every inode (the
 "last write time" fields in the superblock and cylinder groups don't
 have this I think, but there are other spare bits in those things
 which could be co-opted).
 
 We could use those 2 bits to give us a 34 bits of seconds value instead
 of 32 (whether that's 34 bits unsigned, or signed, would be a question for
 whoever decided this approach wasn't too absurdly ugly and decided to
 try it.)
 
   |  Maybe. It isn't clear that large timespans are necessary in practice,
   |  especially for system volumes, where ordinarily nothing is older than
   |  the original OS install.
 
 Once again, nothing cares about FFSv1 for this kind of purpose.
 Boot filesystems for those architectures that need it can be just
 that, they don't need to have dual role as the root, and they don't
 need to be (kernel supported) mountable.
 
 The only system volumes that matter are the ones that already exist,
 and those have times old enough to limit what we can possibly do with
 modifiable time bases.  If one were to assume that no NetBSD FFSv1 can
 be older than 1990, we could use unsigned times and allow up to 2125
 or something, rather than the 2105 we'd get with the 1970 origin.  Is
 that extra 20 years really worth the effort?  The latest we can possibly
 move forward would be a base of 2022 (on the assumption that we tell people
 to stop using FFSv1 now, so any (real usable, not firmware boot) FFSv1
 will have been created 2021 or earlier, and so have dates 2021 (or earlier)
 in the filesystem - which means the upper limit on (unsigned) 32 bit
 time support is 2158 (and almost certainly quite a lot earlier).
 
 Of course, all of this kind of analysis ignores the fact that when files
 are restored to a filesystem, their (older) dates come along with them,
 which is how my (I think 3 year old now) laptop has old e-mail messages
 (and other files) with dates from the mid 1990's.
 
 Those are the times that matter - and no, you cannot arbitrarily change
 the mod times of files that exist.   (Not that this matters to me, of course,
 I'm not using FFSv1, why would I?)
 
   | (And it doesn't matter if unpacking some
   |  200-year-old 3rd-party source tarball to build it gives it all
   |  timestamps of INT_MIN.)
 
 It does if that source tarball contains pre-built files which will now
 not look younger than their sources, and require 200-year-old (no longer
 supported) 3rd-party applications to regenerate them.  Of course, hacks
 can be done, but why would anyone be inflicting FFSv1 upon themselves
 when they don't need it?
 
   |  If one is going to change the on-disk format at all, it's best to not
   |  do it halfassedly. In the computer biz doing things halfassedly is
   |  almost always a mistake, especially in the long term.
 
 Sure, but there's no long term here, all of this is just to keep FFSv1
 workable just barely long enough that no-one cares any more.   After that
 ancient filesystems can be accessed via user level tools (unpacking a
 (max sized) 2TB FFSv1 into a tmpfs using a "dump/restore" or similar
 procedure will be nothing ... the tmpfs wouldn't even notice that 2TB
 used (a fraction of a percent of its available space probably).
 
   |  The biggest costs of either of these approaches (or any other) are (a)
   |  negotiating the on-disk format change and
 
 That's this discussion, right?
 
   | (b) identifying all the code
   |  paths in the system that need adjustment; these cost pretty much the
   |  same no matter what the change is.
 
 Yes.
 
 But those are only the significant costs if the result is so good that
 there are never any problems.  Otherwise, over time, dealing with the
 various issues becomes orders of magnitude more than these.
 
 Since I see no way (certainly none that has been suggested, including mine)
 which has any potential of being issue free, that is the cost that really
 matters, and the only way I see to be rid of that is to be rid of FFSv1
 (as a first class object that anyone can simply use).
 
 kre
 
 ps: when considering 2038 issues, while dealing with 32 bit times in
 FFSv1 is a problem, a much bigger problem approaching in 2038 is what
 happens with old applications linked with libs that use the 32 bit time_t
 interfaces everywhere.   That one we cannot simply wiggle our noses at and
 magically cause things to alter - those applications know the range of
 a (32 bit) time_t, and aren't going to alter themselves.