> > about this: 1) it's only a problem when the kernel allocates buffers
> > for its own internal use, during read()s and write()s, but not mmap().
>
> mmap can probably trigger the problem as well. Anything that will create
> a large buffered write will.

and mmap with be used more and more often, especially with Thorpej's patch
for direct memory to ethernet transfers.

> > seconds for dd if=/dev/zero to flush all of its buffers.
>
> An I/O scheduler does probably make sense. Other mechanism can help too.
> First we probably need a per-partition I/O queue, instead of per device.

i don't think partitions have anything with that. doing that will make
linux-like case (at least linux 2.2. kernels) where lots of partitions
means even slower work and lots of disk seeks as it sorts request per
partition not real device