Subject: Re: File system performance on i386
To: Bharani Chadalavada <bharani.chadalavada@nexsi.com>
From: David Brownlee <abs@netbsd.org>
List: tech-perform
Date: 02/23/2001 08:32:10
On Thu, 22 Feb 2001, Bharani Chadalavada wrote:

> I already have softdep in there. May be you are write. Our source repos has a
> lot of directories and it involves a big directory walk. A lot of files are
> created in the process, including ofcourse the obj files.
>
	It could be down to the size of the buffer cache. In NetBSD
	1.5 and earlier (ie: all official releases to date) the
	buffer cache is sized at boot time. You can use 'options
	BUFCACHE=25' in a kernel config to reserve 25% of your
	memory for buffer cache.  You might want to also bump up
	MAXUSERS to increase the size of various other tables.

> I am using ext2fs on linux. Does that mean ext2fs is much better than ffs??
>
	NetBSD supports both ext2fs and ffs, and I would expect
	you find ffs performs slightly better than ext2fs under
	NetBSD.  NetBSD defaults filesystems to writing metadata
	synchronously to reduce the risk of filesystem corruption
	on powerfailure. This is better than fully synchronously
	(-o sync), but suffers when a significant portion of the
	writes involve updataing metadata (file creation, and
	deletion being the most obvious).

	Linux runs everything async which is faster, but at a higher
	risk (and don't even ask about what it does for programs
	that rely on fsync() to ensure commits have hit the disk).
	You can mount filesystems -o async under NetBSD if you like
	that sort of thing, but make sure you have better backups...

	Soft dependencies (softdep) is a better solution to the problem -
	it maintains an ordered list of metadata updates and always writes
	them to the disk in such an order that the ondisk meta data remains
	consistent, so power should be able to fail at any point without
	serious corruption of the disk.

	You might want to try the test without any mount flags, then with
	-o softdep, and finally -o async just to get a feel for the numbers.
	At which point play with BUFCACHE and MAXUSERS.

	In -current NetBSD supports a unified buffer cache, which is a
	more sophiticated form of Linux's resizable buffer cache. Its
	still being developed but its looking to be a very nice feature.

		David/absolute		-- www.netbsd.org: No hype required --