--pgp-sign-Multipart_Tue_Mar_27_15:27:52_2007-1
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

At Tue, 27 Mar 2007 00:17:25 -0500 (CDT), Johan A. van Zanten wrote:
Subject: Re: Summer of code ideas
>=20
>  Yep.  I thought that one of the benefits of soft updates' ordered writes
> is that in the worst case system crash scenario, the most one would loose
> would be 20 seconds of metadata.  However, if when you say "something goes
> wrong," you are referring to a bug in the soft updates code that does
> something not in the design, then the consequences could be quite
> gruesome, as with any bug of that type in any sort of file system code.

Well, 20 seconds of metadata on a large filesystem on a very large
memory and very busy server can mean vast swaths of metadata are now
scrambled beyond belief.  I.e. with soft-updates you now loose not only
all the dirty buffers in the buffer cache, but also _all_ the metadata
that's been dirtied recently.  That 20 seconds, BTW, is 20 seconds of
solid write activity too, not just the last 20 seconds of elapsed wall
clock time.

I've seen filesystems with, say, 100,000 files end up with a 10-20% loss
of files and all the rest end up in lost+found, but only after running
"fsck -y" a half-dozen times, the first 4 runs of which go extremely
slow due to the massive amount of output they generate.

If you want to do an experiment to see just how bad it gets, just try it.

If you can scrounge/borrow the hardware, take a big-memory server, make
a big filesystem on a bunch of scratch disks with RAIDframe RAID-5 just
for added fun, unpack as many source trees as will fit, then randomly
remove and rename and edit 25% of those files with a bunch of test
scripts running in parallel, and then unmount the filesystem and time
how long it takes to write everything out.  Then for the real test start
over and do it all again, but this time after "systat bufcache" shows
that most of available RAM is used up for metadata an file data, pull
the power plugs.  Then try to clean up the mess.

--=20
						Greg A. Woods

H:+1 416 218-0098 W:+1 416 489-5852 x122 VE3TCP RoboHack <woods@robohack.ca>
Planix, Inc. <woods@planix.com>       Secrets of the Weird <woods@weird.com>

--pgp-sign-Multipart_Tue_Mar_27_15:27:52_2007-1
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 5.0i for non-commercial use
MessageID: qCwOlDSC8SRe3Nsy6YN11w2jpJg5v22Z

iQA/AwUBRgl+SmZ9cbd4v/R/EQJ4wQCeKsqMweiBAi/GrziCIy1BNh0Ow1wAoI92
FxnifNJvQkNtlPiBRPll1t+2
=cHw3
-----END PGP SIGNATURE-----

--pgp-sign-Multipart_Tue_Mar_27_15:27:52_2007-1--