tech-userlevel archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: Kernel VS application file caching

On Thu, Jan 21, 2010 at 09:16:34PM +0000, Sad Clouds wrote:
> On Thursday 21 January 2010 14:52:04 Steven Bellovin wrote:
> > On Jan 21, 2010, at 9:25 AM, Sad Clouds wrote:
> > > As far as I know Unix kernels will transparently cache files into any
> > > available memory to speed up future I/O on those files.
> > >
> > > For applications like Internet servers, which serve many static files
> > > from disk, is there any point in implementing file caching at application
> > > level? It seems like you would end up with 2 copies of the same data -
> > > one copy cached by kernel, another copy cached by application.
> > 
> > To avoid kernel-to-userland copies?
> > 
> > What is your real performance limit?  CPU?  RAM?  I/O bandwidth?  Network
> >  bandwidth?
> Well the idea is to keep frequently accessed data in RAM. If you have a 
> machine with 1 Gig of RAM, you can load up a lot of small files. But if 
> someone occasionally decides to download say a 2 Gig .iso file, the data 
> cached just from that one file can evict all other files from RAM.

I'd gently suggest at some point to visit, for instance,  Given how little you
tell about the grand scheme behind all your questions, it might be
better for you to work that way.

> Ideally there would be a way to tell kernel the maximum amount of data to 
> load 
> into system cache when opening a file. I've looked at madvise(), but it seems 
> to only provide hints to the system after you mmap() a file. It would be 
> useful to set hard limits before loading a file, to avoid cache "pollution".

I'm not quite sure from where you get the idea that calling mmap on a
file makes the kernel load data into memory.

Quentin Garnier - -
"See the look on my face from staying too long in one place
[...] every time the morning breaks I know I'm closer to falling"
KT Tunstall, Saving My Face, Drastic Fantastic, 2007.

Attachment: pgpLlPitXRWse.pgp
Description: PGP signature

Home | Main Index | Thread Index | Old Index