Subject: Re: Data corruption issues, probably involving ffs2 and >1Tb
To: Daniel Carosone <dan@geek.com.au>
From: Brian Buhrow <buhrow@lothlorien.nfbcal.org>
List: current-users
Date: 01/22/2007 16:48:46
	Hello.  I've setup a test, using the shell script below, which I
expect to take a few days to complete.  I'll let you know the results when
they're in.
expdct 
	If you see anything wrong with this shell script in terms of its
methodology, let me know.
-thanks
-Brian


Usage: 
cd to top of tree to be tested.
/path/to/script >& /var/tmp/scrip.log&
...
wait
...

#!/bin/sh
#$Id: test-ffsv2.sh,v 1.1 2007/01/23 00:42:32 buhrow Exp $
#NAME: Brian Buhrow
#DATE: January 22, 2007
#The purpose of this shell script is to determine if ffsV2 as implemented
#on NetBSD-3.x systems, is corrupting files somehow.  This could test files
#on any filesystem, it just happens that FFSV2 may be a problem, so this is
#the impetus to write this script.

#Now, let's define constants and do work.

PATH=/bin:/usr/bin:/sbin:/usr/sbin; export PATH
MAXRUNS=1002

#The basic methodology of this script is to create a list of files to
#check, capture an MD5 checksum for each file, and then repeatedly run each
#file through MD5 again and see if its output differs from the original
#captured output.  Because we use find to get a list of files to check, 
#Be sure you run this on filesystem trees where you know how many files
#you'll be scanning.  Otherwise, you might scan more than you bargained on.

fillist=`find . -type f -print |tr '\012' ' '`

#Now, we'll do our runs.  The first run is comprised of collecting the MD5
#signatures, but other than that, there's nothing special about it.
currun=1

while [ $currun -lt $MAXRUNS ]; do
	for curfil in $fillist; do
		hashname=`echo $curfil |sed 's+/+-+g' |sed 's+^\.-++'`
		hashname="/var/tmp/$hashname"
		if [ ! -s "$hashname" ]; then
			destfile=$hashname
			initrun="true"
		else
			destfile="/var/tmp/testmd5.$$"
			initrun="false"
		fi
		/bin/rm -f $destfile
		md5 $curfil > $destfile
		if [ X"$initrun" != X"true" ]; then
			diff  -u $hashname $destfile
			if [ $? -ne 0 ]; then
				echo "Differences encountered for $curfile!!!"
			fi
			echo "$curfil($currun)"
		fi
	done
	currun=`echo $currun |awk '{ count = $1 + 1;print count }'`
done

echo "All finished!!!"
exit 0