Re: kern/48449: y2038 shortcomings of ffsv1

[Robert Elz <kre%munnari.OZ.AU@localhost>]

The following reply was made to PR kern/48449; it has been noted by GNATS.

From: Robert Elz <kre%munnari.OZ.AU@localhost>
To: gnats-bugs%netbsd.org@localhost
Cc: 
Subject: Re: kern/48449: y2038 shortcomings of ffsv1
Date: Thu, 10 Jun 2021 18:25:27 +0700

     Date:        Thu, 10 Jun 2021 05:30:02 +0000 (UTC)
     From:        David Holland <dholland-bugs%netbsd.org@localhost>
     Message-ID:  <20210610053002.E2EEC1A923B%mollari.NetBSD.org@localhost>
 
   |  Or, much more simply, store a 64-bit epoch in the superblock (this
   |  lets you pick it at newfs time) and add the 32-bit times to it. This
   |  requires adding a superblock field but we can probably afford that.
 
 No, that's is almost the worst of all possible solutions (even including
 just doing nothing).   It is simple and easy to implement in the steady
 state, but the transition mechanism is horrid.  If you really meant *only*
 at newfs time, then it is a waste of effort, anyone doing a newfs can set -O2
 and the issue doesn't arise.   The problem is with all the existing FFSv1
 filesystems, not new ones (and doing a dump/restore to handle the problem
 is way too invasive, and risky).
 
 While it is possible to set up a method whereby the offset can be added to
 (or changed in, which is the same thing) an existing filesystem, doing it
 is far more complex than is needed, as maintaining constant timestamps
 means that if the offset alters, so must every timestamp in every inode,
 and as that can't be done instantaneously, there's the possibility that the
 filesystem might get into a state where some inodes have been converted,
 and others not, which means inodes would need a state bit (old or new offset)
 and the superblock would need two new offset fields.   All a giant mess.
 
 Further, no matter what is done, a FFSv1 cannot store a range of timestamps
 that spans more than 140 years (approx), so any solution needs to live with
 that - which really means that we need to make FFSv1 vanish by 2100 or so.
 That is aside from perhaps access to some ancient filesystems in read only
 (or at least no time update) mode.
 
 Creating an environment where we can live with new FFSv1 filesystems created
 hundreds of years from now, because they can have a base offset at or near
 that time (and so would work fine) should not be an objective - assuming that
 FFS filesystems (in any form) will survive that long in common usage is
 something of a stretch, planning to make it possible is absurd - even if we
 do still want them then, the extra overhead of FFSv2 (or successors) will be
 irrelevant (it already is with today's storage sizes, compared with when it
 was created, and it was OK then).
 
 The "paste in the top half of the current time" is one solution that is even
 worse than this suggestion, as it is really just the same thing with a
 dynamically varying offset, and no flexibility at all.
 
 A very simple solution which will give us time to phase out (most, see below
 for the workaround) FFSv1 completely (within another 80/90 years from now) is
 to add a single extra bit to the superblock, which, when set, indicates that
 all times stored in the filesystem are unsigned.   That alters the range of
 times from 1907..2038 to 1970..2100 (give or take) instantly.
 
 To that we add a tool that scans the filesystem (or an option to fsck more
 likely) which determines if it is safe to set the bit (ie: there are no
 times with the top bit set anywhere) which should be true if no-one has
 been cheating (no-one really ever created/modified/accessed a FFS filesystem
 file before 1970 or after 2037) and if so, simply set the bit and that
 filesystem becomes safe to continue using until 2100 (after which, because
 of the range issue above, nothing can really extend it by more than a few
 years).  The kernel just needs one change to deal with this - when converting
 FFSv1 timestamps to time_t, check the bit in the superblock (copied to
 the mount struct) and if set, zero extend rather than sign extend the value.
 Nothing needs to change going the other way (except possibly defeating some
 compiler sign/range warnings, but that's probably already done).
 
 Simple, safe, and works.   Closer to 2100 we can drop support in the
 kernel for mounting FFSv1 filesystem at all (see below).
 
 For filesystems with times before 1970 or after 2037 (ie: anything with the
 top bit currently set) we can offer to fix them (set the value to something
 else of the user's choosing) and until that is fixed, hopefully sometime
 in the next 15 years, simply leave the bit clear, and the filesystem works
 the same (for that period) as it does now.
 
 In 2038 (or the years just before) we change the kernel to refuse to mount
 FFSv1 filesystems with the bit clear (see below) at least in rw mode (ro
 mode mounts can continue as long as the kernel retains FFSv1 support).
 
 There's a slightly more general method we could use if there are users
 with a legitimate reason for files with timestamps that predate 1970, but
 only perhaps for a decade or two or three (ie: files from older, pre-unix,
 systems, but created in the computer age ... seems doubtful to me, but
 perhaps someone has such things).
 
 For that, we just need to recognise that while the 140 year range is all
 that FFSv1 can possibly deal with, nothing says that the 0 value must
 occur in the centre of that range.   We could allow timestamps that range
 from 1950..2080 (approx) for example (and the range can vary per filesystem).
 This requires a (32 bits is enough) field in the superblock, not to store
 the origin, but to store the minimum (or max, makes no real difference) value
 of the range, and then we simply convert between those values and time_t.
 Once again time_t -> FFS time is a no-op conversion (just truncate the
 value to the low 32 bits) but this time FFS->time_t requires a comparison,
 if the (unsigned) FFS value is less than N (the unsigned version of the
 minimum (maybe maximum, they turn out to be almost the same thing) value
 of the supported range, for that filesystem) then treat the value as signed,
 sign extend it to become a time_t.   If it is greater (or equal), then zero
 extend instead. Or some arithmetic just like that (the precise formula can
 come later, if we need this, but it is that simple).   This becomes the same
 as the "one bit" version when the range is set to be 1970..2100 (or whatever)
 ie: all values become zero extended as the other side of the test can never
 be true.   Again, when the current time passes the end of the range, the
 kernel refuses to (rw) mount that filesystem any more.
 
 To deal with old filesystems into the distant future (this is the "below"
 referred to above) we need a user level (FUSE I assume) FFSv1 filesystem
 converter which can be configured via mount options to convert timestamps
 however is appropriate (there could even be a range of such things for
 different types of conversion, especially when allowing writes, and deciding
 how to store timestamps that are out of the range of what the filesystem
 could normally handle ... since performance doesn't matter any more, and
 special cases are definitely allowed, all kinds of possibilities exist
 for what to do).   Fortunately we have some decades before this will
 really be required (and by then, it might not be).
 
 Just please, no "variable time" nonsense in the filesystems, an inode
 with a timestamp that currently converts as Thu Jun 10 11:01:00 UTC 2021
 must always convert as exactly that (over the range, 1970-2038),
 never anything else, or not from the kernel anyway (what a user level hack
 might do is up to it, and if the user is content, that's good enough).
 
 kre