Subject: Re: disk caching
To: Eduardo Horvath <email@example.com>
From: Manuel Bouyer <firstname.lastname@example.org>
Date: 04/13/2000 22:38:50
On Wed, Apr 12, 2000 at 09:54:21AM -0700, Eduardo Horvath wrote:
> I really hope I don't end up triggering a flame war with this.
> I have recently had the opportunity to run both NetBSD and Linux under
> heavy load on the same machine (no, not at the same time.) The
> interactive performance of NetBSD under those conditions is undenyably
> superior. This experience is making me very uncomfortable with the future
> move to UBC.
> The box is a 500MHz PIII with 128MB RAM and an IDE disk. What I did was
Did both use IDE DMA ? Linux doesn't seem to use DMA by default, at last with
some chips ....
> fire up a parallel compile to the point where the CPU idle time goes to
> 0%, while using X applications on the console. Under NetBSD you can
> bearly notice the machine is under load. Under Linux the machine becomes
> unbearably sluggish.
> I believe what happens under Linux is that since the buffer cache is
> competing with the VM system, application code and data pages are being
> reclaimed by the buffer cache and swapped out. Then when you try to move
> the mouse or hit a key in your terminal window, the appropriate pages need
> to be paged back in. Since we're talking about interactive applications
> here, the critical code paths are only being traversed when there's I/O
> going on, which tends to be sporadic, thus there is a tendency for these
> critical pages to be reclaimed.
Actually Linux doesn't have a merged VM/buffer cache (which UBC is, hence
the name :) but just a variable size traditional buffer cache. This makes
it harder for file I/O and VM to cooperate.
> As I've stated before, I think we need some way to limit the growth of the
> buffer cache before it starts replacing application data. There are a
> number of pathalogical case with UBC systems where linear disk traversal
> (say using `find / -exec grep .....') or generating and writing large
> amounts of data tend to replace all application pages with cached disk
> pages that are unlikely to ever be re-referenced.
> There's another issue with the way Linux seems to handle the buffer
> cache. It seems to like to delay writes until the last possible moment,
> which makes the machine seem to run really fast until it needs to flush
> everything to disk at which point the machine siezes up.
NetBSD used to behave the same way with the traditionnal update(8) and a large
buffer cache (although this was sync(2) which freese the I/O, I don't know
for linux). Now that with soft updates we got the ioflush thread, this
doesn't happen (unless someone type 'sync' maybe, I didn't try :)
> I suppose I'll have to grab hold of UBC and do some performance testing
> one of these days. But the UBC branch looks a bit ancient and doesn't
> seem to handle softdep yet.
> > The static size of the buffer cache can currently be controlled
> > by the "BUFCACHE" option, which is a percentage of available memory.
> I believe there are also some issues with the amount of kernel address
> space available for use in the buffer cache.
You may have to tweak NKMEMPAGES, I think ...
Manuel Bouyer <email@example.com>