Re: kmem-pool-uvm

[Lars Heidieker <lars%heidieker.de@localhost>]

Hi,

On 05/26/11 03:51, YAMAMOTO Takashi wrote:
> hi,
> 
>>> Findings after having run the system for a while and having about 1.1gig
>>> in the pool(9)s:
>>> Option a: about 30000 allocated kernel map_entries (not in the map but
>>> allocated)
>>> Option b: about 100000 allocated boundary tags.
>>> Option c: about 400000 allocated boundary tags.
>>>
>>> With boundary tags beeing about half the size of vm_map_entries the vmem
>>> version uses slightly more memory but not so much.
> 
> why did you use different numbers for heap_va_arena's qcache_max
> (8 * PAGE_SIZE) and VMK_VACACHE_MAP_QUANTUM (32 * PAGE_SIZE)?
> 
> if i read your patches correctly, the number of map entries/boundary tags
> will be smaller if these constants are bigger, right?
> 

I choose the 8 * PAGE_SIZE for qcache_max as the quantum caches are
pool_caches, so if we have only two or three allocation of a particular
size made by different cpus we have 2 or 3 times the va in the pool
caches, with a lot va wasted.
This might or might not be a point but was the motivation to start with
a lower value.
If the size is increased the amount of boundary tags goes down a bit
further, these caches very much have an influence on the control
structure allocation count.

One could argument that the vmk_vacaches should be pool_caches as well
(I tried that, no problem to switch them) for scalability reasons, then
they will have to deal with the same wastage argument.
Currently having these as pool_caches doesn't buy us much, as they have
to get backed with physical memory, which is a process most likely
serializing the allocation anyway... But this is no different between
the two options ;-)

>>> Both versions use a modified kmem(9) that interfaces either with vmem or
>>> the extended kva caches, which has page_aligned memory for allocations
>>> of page_size and larger and cache_line aligned allocations for
>>> allocations between cache_line size and page_size.
>>> This should resolve some problems xen-kernels do have.
> 
> does the original (solaris) version of kmem_alloc provide aligned
> allocations?
> 

Yes it does, it switches to cache_line size for alignment for
allocations >= cache_line size and to page_size alignment for
allocations >= page_size.

> YAMAMOTO Takashi
> 

Lars