[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
Re: Kernel VS application file caching
On Thursday 21 January 2010 14:52:04 Steven Bellovin wrote:
> On Jan 21, 2010, at 9:25 AM, Sad Clouds wrote:
> > As far as I know Unix kernels will transparently cache files into any
> > available memory to speed up future I/O on those files.
> > For applications like Internet servers, which serve many static files
> > from disk, is there any point in implementing file caching at application
> > level? It seems like you would end up with 2 copies of the same data -
> > one copy cached by kernel, another copy cached by application.
> To avoid kernel-to-userland copies?
> What is your real performance limit? CPU? RAM? I/O bandwidth? Network
Well the idea is to keep frequently accessed data in RAM. If you have a
machine with 1 Gig of RAM, you can load up a lot of small files. But if
someone occasionally decides to download say a 2 Gig .iso file, the data
cached just from that one file can evict all other files from RAM.
Ideally there would be a way to tell kernel the maximum amount of data to load
into system cache when opening a file. I've looked at madvise(), but it seems
to only provide hints to the system after you mmap() a file. It would be
useful to set hard limits before loading a file, to avoid cache "pollution".
Main Index |
Thread Index |