Subject: Re: soft updates Re: Summer of code ideas
To: Dieter <netbsd@sopwith.solgatos.com>
From: Greg A. Woods <woods@planix.com>
List: netbsd-users
Date: 04/07/2007 21:38:16
--pgp-sign-Multipart_Sat_Apr__7_21:38:13_2007-1
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

At Sat, 07 Apr 2007 11:47:45 +0100, Dieter wrote:
Subject: Re: soft updates Re: Summer of code ideas=20
>=20
> To avoid this you have to use mount -o sync.

and thus lose all benefit of softdep -- it's actually better to not use
softdep in the first place and thus still maintain the benefit of write
cache for file data.

> > The very concept of delaying metadata writes guarantees less safety,
> > regardless of whether they are eventualy done in the correct order or
> > not.
>=20
> For me, the point of softdeps is that it forces the writes to be done
> in the correct order, to protect the integrity of the filesystem.

You still miss the point.

Without softdeps the metadata writes are still (supposed to be) done in
the correct order.  This has been a fundamental goal in the design of
traditional Unix filesystems since practically day one.

It doesn't matter how well ordered the writes are if there are megabytes
of them outstanding when you crash the system.  Softdep _should_ still
allow for clean recovery of the filesystem in theory, but in practice
this isn't necessarily always true, and regardless whenever large
amounts of metadata are lost then all kinds of file data can become
irretrievable, sometimes even that which has not been modified.

The difference is that with softdep there can be huge numbers of
metadata changes still pending in core while without softdep this simply
cannot happen (unless you have hardware caches outside the control of
the OS).  Even worse it seems that the softdep metadata cache is often
left dirty for hours or even days.

The mere fact that "rm -rf" on a large tree of tens of thousands of
files can complete in very few seconds with almost no disk activity when
using softdep should be proof enough of these issues.  Without softdep
you get the same safety but none of the performance, as will be seen by
trying to do the same "rm -rf" on the same large tree.

> So you are safe from seeing "run fsck manually".  I used to get this,
> but I haven't seen it since I switched to softdeps.

Then most certainly you have encountered other bugs or adverse hardware
cache loss and/or corruption.  Softdep can only make things worse.  It
cannot, in theory, ever improve such a situation.

--=20
						Greg A. Woods

H:+1 416 218-0098 W:+1 416 489-5852 x122 VE3TCP RoboHack <woods@robohack.ca>
Planix, Inc. <woods@planix.com>       Secrets of the Weird <woods@weird.com>

--pgp-sign-Multipart_Sat_Apr__7_21:38:13_2007-1
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 5.0i for non-commercial use
MessageID: 9XJ/WsUp+NzixZNtu7hPwbXIYbHCvcXs

iQA/AwUBRhhHiGZ9cbd4v/R/EQI11gCgsJc52q1i9SzhwuAl9kv5Lea1SsQAmwQL
cx2Q8E4R63MSvoA3vK9socFw
=USni
-----END PGP SIGNATURE-----

--pgp-sign-Multipart_Sat_Apr__7_21:38:13_2007-1--