Subject: Re: disk caching (was: Re: Really bad performance when doing newfs)
To: None <netbsd@sopwith.solgatos.com>
From: Chuck Swiger <cswiger@mac.com>
List: netbsd-users
Date: 02/04/2006 14:26:16
Dieter wrote:
> Thank you for pointing out
> http://www.freebsd.org/doc/en_US.ISO8859-1/books/arch-handbook/driverbasics-block.html
> 
> And yet, FreeBSD defaults to turning on the write cache in disks.

Correct.  People who cared about performance more than reliability screamed
loudly when hw.ata.wc defaulted to 0.  People who care more about reliability
either set the sysctl now, or use RAID hardware like this:

aac0: <Dell PERC 3/Di> mem 0xf0000000-0xf7ffffff irq 31 at device 2.1 on pci2
aac0: i960RX 100MHz, 118MB cache memory, optional battery present
                                         ^^^^^^^^^^^^^^^^^^^^^^^^
aac0: Kernel 2.5-0, Build 2991, S/N xxxxx
aac0: Supported Options=0
aacd0: <RAID 1 (Mirror)> on aac0
aacd0: 17355MB (35544576 sectors)

...which provides backup power long enough to ensure that the RAID cache and
drives can flush pending data even if the AC line drops.

>> This caching makes block-devices almost unusable
> 
> I find them quite usable.

That's a fine opinion.

It's not clear that "block devices" mean anything special to an OS which uses
the notion of a unified buffer cache, but for an OS which has an explicit
block-IO cache, perhaps the distinction still has meaning.

>> This makes predictable and reliable crash recovery of on-disk data structures
>> (filesystems, databases etc.) impossible.
> 
> I was under the impression that filesystems had the ability to force
> the order of writes when they needed to?  Am I misremembering?

The filesystem can attempt to enforce write order.

Unfortunately, most IDE drives will reorder writes internally and will lie about
whether they have actually completed transactions.  This is true for many drives
even if you tell the drive not to enable the write cache.

Maybe NCQ and SATA-II will improve matters, but if you want that level of
consistency, you should be looking towards SCSI + TCQ and RAID solutions.

> Is this a problem in NetBSD?

It's an issue which affects all operating systems.

-- 
-Chuck