ffs corruption

To: tech-kern%netbsd.org@localhost
Subject: ffs corruption
From: Emmanuel Dreyfus <manu%netbsd.org@localhost>
Date: Wed, 24 May 2017 19:29:44 +0000
Hello

I experience recurent ffs corruption on a NetBSD-7.1/i386 machine
running XEN3PAE_DOMU kernel.

The VM has two filesystems and it keeps corrupting them. It will
complain about unallocated inodes an bad file descriptors, and
a fsck will remove a bunch of files.

Other VM on the machine have no problems, which suggests it is not a disk
problem.

Below is a typical fsck at the end of which /us/pkg/sbin is gone. Does
that rings a bell for someone?

# fsck -fy /
** /dev/rxbd0a
** File system is already clean
** Last Mounted on /
** Root file system
** Phase 1 - Check Blocks and Sizes
PARTIALLY ALLOCATED INODE I=829440
CLEAR? yes

PARTIALLY ALLOCATED INODE I=829441
CLEAR? yes

** Phase 2 - Check Pathnames
UNALLOCATED  I=829440  OWNER=0 MODE=0
SIZE=0 MTIME=Jan  1 01:00 1970  
NAME=/kern

REMOVE? yes

UNALLOCATED  I=829441  OWNER=0 MODE=0
SIZE=0 MTIME=Jan  1 01:00 1970  
NAME=/usr/X11R6/lib/X11/fonts/Type1

REMOVE? yes

UNALLOCATED  I=829442  OWNER=65539 MODE=0
SIZE=0 MTIME=Jan  1 01:00 1970  
NAME=/usr/pkg/sbin

REMOVE? yes

** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
LINK COUNT DIR I=2  OWNER=0 MODE=40755
SIZE=1024 MTIME=May 12 09:52 2017   COUNT 28 SHOULD BE 27
ADJUST? yes

LINK COUNT DIR I=760352  OWNER=0 MODE=40755
SIZE=512 MTIME=Feb  6 18:27 2010   COUNT 13 SHOULD BE 12
ADJUST? yes

UNREF FILE  I=829444  OWNER=0 MODE=100555
SIZE=2687 MTIME=Apr  4 21:03 2015  
RECONNECT? yes

UNREF FILE  I=829445  OWNER=0 MODE=100555
SIZE=620 MTIME=Apr  4 21:03 2015  
RECONNECT? yes

UNREF FILE  I=829446  OWNER=0 MODE=100755
SIZE=192527 MTIME=Apr  4 21:03 2015  
RECONNECT? yes

UNREF FILE  I=829447  OWNER=0 MODE=100755
SIZE=188415 MTIME=Apr  4 21:03 2015  
RECONNECT? yes

UNREF FILE  I=829448  OWNER=0 MODE=100755
SIZE=156116 MTIME=Apr  4 21:03 2015  
RECONNECT? yes

UNREF FILE  I=829449  OWNER=0 MODE=100755
SIZE=152010 MTIME=Apr  4 21:03 2015  
RECONNECT? yes

UNREF FILE  I=829450  OWNER=0 MODE=100755
SIZE=178169 MTIME=Apr  4 21:03 2015  
RECONNECT? yes

UNREF FILE  I=829470  OWNER=0 MODE=100755
SIZE=39101 MTIME=May 22 05:58 2017  
RECONNECT? yes

UNREF FILE  I=832309  OWNER=0 MODE=100755
SIZE=4186 MTIME=Apr  6 02:42 2015  
RECONNECT? yes

UNREF FILE  I=832438  OWNER=0 MODE=100755
SIZE=11932 MTIME=Apr  6 00:17 2015  
RECONNECT? yes

UNREF FILE  I=832439  OWNER=0 MODE=100755
SIZE=12336 MTIME=Apr  6 00:17 2015  
RECONNECT? yes

UNREF FILE  I=832440  OWNER=0 MODE=100755
SIZE=10472 MTIME=Apr  6 00:17 2015  
RECONNECT? yes

UNREF FILE  I=832538  OWNER=0 MODE=100755
SIZE=3410 MTIME=Apr 12 20:53 2015  
RECONNECT? yes

UNREF FILE  I=832539  OWNER=0 MODE=100755
SIZE=6179 MTIME=Apr 13 02:55 2015  
RECONNECT? yes

UNREF FILE  I=832540  OWNER=0 MODE=120755
SIZE=25 MTIME=Apr 13 02:55 2015  
RECONNECT? yes

UNREF FILE  I=832541  OWNER=0 MODE=100644
SIZE=1053 MTIME=Apr 12 20:53 2015  
RECONNECT? yes

UNREF FILE  I=832542  OWNER=0 MODE=100755
SIZE=9015 MTIME=Apr 13 02:55 2015  
RECONNECT? yes

UNREF FILE  I=832543  OWNER=0 MODE=100755
SIZE=32906 MTIME=Apr 13 02:55 2015  
RECONNECT? yes

UNREF FILE  I=832544  OWNER=0 MODE=100755
SIZE=568512 MTIME=Apr 13 02:55 2015  
RECONNECT? yes

UNREF FILE  I=832545  OWNER=0 MODE=100755
SIZE=17130 MTIME=Apr 13 02:55 2015  
RECONNECT? yes

UNREF FILE  I=832999  OWNER=0 MODE=100755
SIZE=36212 MTIME=Apr 19 00:39 2015  
RECONNECT? yes

UNREF FILE  I=833001  OWNER=0 MODE=102755
SIZE=125912 MTIME=Apr 19 01:37 2015  
RECONNECT? yes

UNREF FILE  I=833057  OWNER=0 MODE=100755
SIZE=73975 MTIME=Jul 24 15:09 2015  
RECONNECT? yes

UNREF FILE  I=833121  OWNER=0 MODE=100755
SIZE=19604 MTIME=Apr 10 13:24 2015  
RECONNECT? yes

UNREF FILE  I=833122  OWNER=0 MODE=100755
SIZE=6116 MTIME=Apr 10 13:24 2015  
RECONNECT? yes

UNREF FILE  I=833123  OWNER=0 MODE=100755
SIZE=4400 MTIME=Apr 10 13:24 2015  
RECONNECT? yes

UNREF FILE  I=833124  OWNER=0 MODE=100755
SIZE=4352 MTIME=Apr 10 13:24 2015  
RECONNECT? yes

UNREF FILE  I=833646  OWNER=0 MODE=100755
SIZE=105460 MTIME=Apr 10 17:07 2015  
RECONNECT? yes

UNREF FILE  I=834002  OWNER=0 MODE=100755
SIZE=7790 MTIME=Apr  8 15:10 2015  
RECONNECT? yes

UNREF FILE  I=834003  OWNER=0 MODE=100755
SIZE=7911 MTIME=Apr  8 15:10 2015  
RECONNECT? yes

UNREF FILE  I=834004  OWNER=0 MODE=100755
SIZE=50149 MTIME=Apr  8 15:10 2015  
RECONNECT? yes

UNREF FILE  I=834005  OWNER=0 MODE=100755
SIZE=18117 MTIME=Apr  8 15:10 2015  
RECONNECT? yes

UNREF FILE  I=834006  OWNER=0 MODE=100755
SIZE=18118 MTIME=Apr  8 15:10 2015  
RECONNECT? yes

UNREF FILE  I=834217  OWNER=0 MODE=100755
SIZE=6627 MTIME=Jan 16 15:19 2017  
RECONNECT? yes

UNREF FILE  I=834219  OWNER=0 MODE=120755
SIZE=69 MTIME=Jan 16 15:20 2017  
RECONNECT? yes

UNREF FILE  I=834220  OWNER=0 MODE=100755
SIZE=90322 MTIME=Jan 16 15:19 2017  
RECONNECT? yes

UNREF FILE  I=834221  OWNER=0 MODE=100755
SIZE=1113452 MTIME=Jan 16 15:19 2017  
RECONNECT? yes

UNREF FILE  I=834227  OWNER=0 MODE=120755
SIZE=10 MTIME=Jan 16 15:19 2017  
RECONNECT? yes

UNREF FILE  I=834229  OWNER=0 MODE=120755
SIZE=10 MTIME=Jan 16 15:19 2017  
RECONNECT? yes

UNREF FILE  I=835243  OWNER=0 MODE=100755
SIZE=450274 MTIME=Jan 16 15:19 2017  
RECONNECT? yes

UNREF FILE  I=835244  OWNER=0 MODE=100755
SIZE=13894 MTIME=Jan 16 15:20 2017  
RECONNECT? yes

UNREF FILE  I=835406  OWNER=0 MODE=100755
SIZE=48797582 MTIME=Dec 28 03:07 2016  
RECONNECT? yes

UNREF FILE  I=835407  OWNER=0 MODE=100755
SIZE=25345 MTIME=Jan 16 15:19 2017  
RECONNECT? yes

UNREF FILE  I=835408  OWNER=0 MODE=100755
SIZE=51794 MTIME=Jan 16 15:20 2017  
RECONNECT? yes

LINK COUNT DIR I=837583  OWNER=0 MODE=40755
SIZE=512 MTIME=Aug 11 14:18 2015   COUNT 14 SHOULD BE 13
ADJUST? yes

UNREF FILE  I=840550  OWNER=0 MODE=100444
SIZE=21991 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=840554  OWNER=0 MODE=100444
SIZE=75363 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=840555  OWNER=0 MODE=100444
SIZE=21529 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=840556  OWNER=0 MODE=100444
SIZE=72834 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=840558  OWNER=0 MODE=100444
SIZE=21960 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=840560  OWNER=0 MODE=100444
SIZE=74892 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=840873  OWNER=0 MODE=100444
SIZE=22144 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=840924  OWNER=0 MODE=100444
SIZE=72354 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841538  OWNER=0 MODE=100444
SIZE=15395 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841544  OWNER=0 MODE=100444
SIZE=40766 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841554  OWNER=0 MODE=100444
SIZE=15407 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841556  OWNER=0 MODE=100444
SIZE=39511 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841557  OWNER=0 MODE=100444
SIZE=15397 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841558  OWNER=0 MODE=100444
SIZE=40008 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841559  OWNER=0 MODE=100444
SIZE=15414 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841560  OWNER=0 MODE=100444
SIZE=39871 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841561  OWNER=0 MODE=100444
SIZE=21626 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841562  OWNER=0 MODE=100444
SIZE=33799 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841563  OWNER=0 MODE=100444
SIZE=22099 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841564  OWNER=0 MODE=100444
SIZE=35229 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841565  OWNER=0 MODE=100444
SIZE=19764 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841566  OWNER=0 MODE=100444
SIZE=34869 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841567  OWNER=0 MODE=100444
SIZE=19927 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841568  OWNER=0 MODE=100444
SIZE=35118 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841569  OWNER=0 MODE=100444
SIZE=29887 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841570  OWNER=0 MODE=100444
SIZE=126064 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841571  OWNER=0 MODE=100444
SIZE=29815 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841572  OWNER=0 MODE=100444
SIZE=126868 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841573  OWNER=0 MODE=100444
SIZE=29979 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841574  OWNER=0 MODE=100444
SIZE=125201 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841575  OWNER=0 MODE=100444
SIZE=30032 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841576  OWNER=0 MODE=100444
SIZE=122124 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841577  OWNER=0 MODE=100444
SIZE=54914 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841578  OWNER=0 MODE=100444
SIZE=14987 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841579  OWNER=0 MODE=100444
SIZE=77488 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841580  OWNER=0 MODE=100444
SIZE=14971 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841581  OWNER=0 MODE=100444
SIZE=78101 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841582  OWNER=0 MODE=100444
SIZE=15096 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841583  OWNER=0 MODE=100444
SIZE=77941 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841584  OWNER=0 MODE=100444
SIZE=15047 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841585  OWNER=0 MODE=100444
SIZE=78500 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841586  OWNER=0 MODE=100444
SIZE=31048 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841587  OWNER=0 MODE=100444
SIZE=70121 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841588  OWNER=0 MODE=100444
SIZE=31403 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841589  OWNER=0 MODE=100444
SIZE=72496 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841590  OWNER=0 MODE=100444
SIZE=31418 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841591  OWNER=0 MODE=100444
SIZE=72332 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841592  OWNER=0 MODE=100444
SIZE=31443 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841593  OWNER=0 MODE=100444
SIZE=71902 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841594  OWNER=0 MODE=100444
SIZE=31441 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841595  OWNER=0 MODE=100444
SIZE=93860 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841596  OWNER=0 MODE=100444
SIZE=31422 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841597  OWNER=0 MODE=100444
SIZE=92799 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841598  OWNER=0 MODE=100444
SIZE=31696 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

NO SPACE LEFT IN /lost+found
EXPAND? yes

UNREF FILE  I=841599  OWNER=0 MODE=100444
SIZE=102798 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841600  OWNER=0 MODE=100444
SIZE=31672 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841601  OWNER=0 MODE=100444
SIZE=94877 MTIME=Feb 28 17:53 2003  
RECONNECT? yes

UNREF FILE  I=841602  OWNER=0 MODE=100444
SIZE=6867 MTIME=Jul  7 08:52 2008  
RECONNECT? yes

UNREF FILE  I=841603  OWNER=0 MODE=100444
SIZE=6867 MTIME=Jul  7 08:52 2008  
RECONNECT? yes

UNREF FILE  I=841604  OWNER=0 MODE=100444
SIZE=2090 MTIME=Jul  7 08:51 2008  
RECONNECT? yes

UNREF FILE  I=841605  OWNER=0 MODE=100444
SIZE=14330 MTIME=Jul  7 08:52 2008  
RECONNECT? yes

** Phase 5 - Check Cyl groups
FREE BLK COUNT(S) WRONG IN SUPERBLK
SALVAGE? yes

SUMMARY INFORMATION BAD
SALVAGE? yes

BLK(S) MISSING IN BIT MAPS
SALVAGE? yes

250092 files, 3336939 used, 1566620 free (128300 frags, 179790 blocks, 2.6% fragmentation)

***** FILE SYSTEM WAS MODIFIED *****

-- 
Emmanuel Dreyfus
manu%netbsd.org@localhost
Follow-Ups:
- Re: ffs corruption
  - From: Emmanuel Dreyfus
Prev by Date: Re: Adding ruminit(4)
Next by Date: Re: Adding ruminit(4)
Previous by Thread: Adding ruminit(4)
Next by Thread: Re: ffs corruption
Indexes:
Home | Main Index | Thread Index | Old Index