Re: Debugging kernel memory allocation

[Tom Ivar Helbekkmo <tih%hamartun.priv.no@localhost>]

Tom Ivar Helbekkmo <tih%hamartun.priv.no@localhost> writes:

> Here's a possibly interesting data point: yesterday, I shut down
> MooseFS, which freed up about half a gig of memory.  Since all the
> interesting bits from the disk were already in the (6GB) allocated file
> cache, it stayed like that over the afternoon and evening, with no
> events.  During the night, however, my backups ran, and this morning the
> system was back to very little free.
>
> Now for the interesting bit: a 'systat bufcache' that had been running
> all night, is still running, but shows an error message: "can't get
> buffers: Resource temporarily unavailable".

With 6GB out of a total of 8GB used for file cache, so that my system is
definitely not short of RAM, I'd think it shouldn't be having trouble
allocating memory at need.  Nevertheless, if I keep 'systat bufcache'
running, this keeps happening.

I notice one interesting side effect, though:

With four of these 'systat bufcache' processes running, all requesting
and freeing memory, the amount indicated as free has stabilized at a
higher value (in the area of 100MB instead of a typical 4MB), and I
don't seem to be getting the hangs.

Now, I may be jumping to conclusions here, but I'm starting to suspect
that a modern system with a large memory and fast CPUs needs to keep
more RAM free and available than those of yesteryear.  Stuff like this,
in uvm_pdaemon.c, starts looking suspect to me:

        /*
         * try to keep 0.5% of available RAM free, but limit to between
         * 128k and 1024k per-CPU.  XXX: what are these values good for?
         */
        val = uvmexp.npages / 200;
        val = MAX(val, (128*1024) >> PAGE_SHIFT);
        val = MIN(val, (1024*1024) >> PAGE_SHIFT);
        val *= ncpu;

Two problems with that: it doesn't do what the comment says it does, and
such magic constants start to smell after a few years.  I think I'm
going to comment out the "val = MIN(..." line, and see what happens.

-tih
-- 
It doesn't matter how beautiful your theory is, it doesn't matter how smart
you are. If it doesn't agree with experiment, it's wrong.  -Richard Feynman