Patches for journalling support

[Simon Burge <simonb%wasabisystems.com@localhost>]

Hi folks,

Wasabi Systems Inc is pleased to make our journalling code available
to the NetBSD community.  This code is known as WAPBL - Write Ahead
Physical Block Logging, and has been used to provide meta-data
journalling in production environments for over 4 years.  WAPBL
journals meta-data only - not file data, and has been used on
filesystems ranging from from 16MB to multiple terabytes in size.

WAPBL was originally written by Darrin B. Jewell while at Wasabi.
Many thanks to Antti Kantee, Andy Doran and Greg Oster for their work
in adapting our journalling code to NetBSD -current (our contribution
was based on the netbsd-4 branch) and testing.

WAPBL is enabled in the kernel with:

        options WAPBL

Currently WAPBL uses space after the filesystem (but before the
partition's end) for its journal.  The suggested method is to use:

        newfs -s -64m <other options> /dev/<foo>

to leave 64MB of space for the journal.  Obviously this will destroy
any existing data in the filesystem on /dev/<foo> and you should, of
course, back up any data on this filesystem first.  The journal size
itself is still the subject of some research.  Solaris uses "10% of
filesystem size to a maximum of 64MB", and at Wasabi we've always used
a 64MB journal on larger filesystems.

As well as the kernel, the following binaries need to be updated:

        /sbin/fsck
        /sbin/fsck_ffs
        /sbin/mount
        /sbin/mount_ffs

The easiest way to do this is to just rebuild using build.sh and
reinstall either base.tgz or the above binaries.

There are three known issues:

 - "mount -u -o log" doesn't work yet, so you can't have logging enabled
   on the root filesystem. "mount -u -o log" is disabled in this patch
   (see around line 222 of sys/kern/vfs_syscalls.c).

 - Under some very heavy workloads there seems to be some sort of
   bottleneck.  20 parallel extracts of a src.tar.gz (a tar of the
   NetBSD source tree) on a quad-core machine has triggered this
   problem, but we can't reproduce it on a dual-core machine.  Using
   "options WAPBL_DEBUG_SERIALIZE" seems to be a workaround.

 - The log itself must currently be at the end of the partition (see the
   newfs command above).  We have plans to implement a log that lives in
   the filesystem itself and is allocated at mount time.

The patches are available at:

    http://www.NetBSD.org/~simonb/wapbl-current-20080302.diff

Note that even though the basis of these patches have been in
production use for a few years, these patches should still be
considered experimental as they are against NetBSD -current and not
the same source code base we use at Wasabi.

Pending serious problems, I'd like to integrate these changes in two
or three weeks time, depending on feedback.

On behalf of Wasabi Systems,
Simon.
--
Simon Burge                                   
<simonb%wasabisystems.com@localhost>
NetBSD Development, Support and Service:   http://www.wasabisystems.com/

This email and any attached documents may be confidential and property
of Wasabi Systems.  If you are not the intended recipient, you may not
disclose, copy, distribute, or act in reliance on the information in
this email.