Re: kern/40474: Kernel panic after remounting raid root with softdep

[David Holland <dholland-bugs%netbsd.org@localhost>]

The following reply was made to PR kern/40474; it has been noted by GNATS.

From: David Holland <dholland-bugs%netbsd.org@localhost>
To: kivinen%iki.fi@localhost
Cc: kern-bug-people%netbsd.org@localhost, gnats-admin%netbsd.org@localhost,
        netbsd-bugs%netbsd.org@localhost, gnats-bugs%netbsd.org@localhost
Subject: Re: kern/40474: Kernel panic after remounting raid root with
        softdep
Date: Sun, 25 Jan 2009 21:07:55 +0000

 On Sun, Jan 25, 2009 at 11:05:00AM +0000, kivinen%iki.fi@localhost wrote:
  >     The system has mirrored raid as root, and when I boot it up
  >     and then run mount -o softdep -u / the command works, but after
  >     that next time I start "vi" the machine panics. The mount
  >     command does not show that the softdep took effect (i.e
  >     it still list options as local), and if I can still 
  >     run some other programs from the root filesystem. The 
  >     file edited with vi does not matter, and crash happens
  >     even if no file is given. If this is tried on the second 
  >     (non root disk) then this does not happen, even if the 
  >     vi binary is copied to the other disk (i.e. mount the
  >     another disk without softdep, remount it with -u -o softdep
  >     and then try to run vi from there do work). 
  > 
  >     I assume this might be related to somehow to the way 
  >     vi creates temp files or something.
 
 I'm pretty sure turning softdep on after mount is a known problem, and
 that there's an existing PR about it, but I can't find one. Anyway, I
 doubt it's got much to do with either vi or raid.
 
 I think I know what's going on, maybe, but I don't have time right now
 to look into it in any detail, and unfortunately I don't think anyone
 else is interested in fixing softdep issues... :(
 
 The immediate problem appears to be that it's allocating the same
 inode twice. I think this is probably happening either because of a
 general known problem with mount updates not flushing things properly
 (e.g. PR 30525)... or because softdep uses different rules for
 handling free object bitmaps from baseline ffs, and after the mode
 transition not all filesystem state is arranged the way softdep
 expects.
 
 Did the fsck after the crash show any problems? And, if you reboot
 again without having used softdep, boot to single user, and force
 running a fsck, does *that* show any problems? (In fact, it's probably
 a good idea to do such a fsck on general principles; fsck behaves
 differently if it thinks softdep was in use, so crashing in this
 fashion might have confused it.)
 
 
 If you have a test machine to crash on, the following information
 might be useful, maybe:
 
    - If you sync after the remount, does the crash still occur?
      (I expect it will, but it would be interesting if it didn't.)
 
    - Can you reproduce the crash without vi but instead by manually
      creating a scratch file in (probably) /var/tmp or
      /var/tmp/vi.recover?
 
    - If you boot single-user, remount / read-write, sync, remount /
      again to turn on softdep, sync, and then run vi, does it crash?
      If not, does omitting any of the syncs change the behavior?
 
    - If not, and you do the same, but create a scratch file in /tmp or
      /var/tmp before turning on softdep, does it then crash? If not,
      does creating a lot of scratch files instead of just one trigger
      the crash?
 
    - It might also be interesting to deliberately hit the power switch
      at various points, because this allows comparing what's been
      written to disk with what's being kept in memory, and any
      discrepancy is interesting. However, this requires mucking about
      with fsdb and I'm not sure exactly what to look for either.
 
 (None of this is worth crashing anything other than a test machine to
 find out, though.)
 
 -- 
 David A. Holland
 dholland%netbsd.org@localhost