Re: kmem-pool-uvm

[Lars Heidieker <lars%heidieker.de@localhost>]

hi,

On 06/15/11 06:04, YAMAMOTO Takashi wrote:
> hi,
> 
> On 06/09/11 01:44, Mindaugas Rasiukevicius wrote:
>>>> yamt%mwd.biglobe.ne.jp@localhost (YAMAMOTO Takashi) wrote:
>>>>>> Yes it does, it switches to cache_line size for alignment for
>>>>>> allocations >= cache_line size and to page_size alignment for
>>>>>> allocations >= page_size.
>>>>>
>>>>> kmem_alloc(9F) says:
>>>>>
>>>>>   The allocated memory is at least double-word aligned, so it  can
>>>>>   hold  any  C  data  structure.  No  greater alignment can be
>>>>>   assumed.
>>>>>
>>>>> % uname -sr
>>>>> SunOS 5.10
>>>>>
>>>>> so i don't think it's api-wise guaranteed.
>>>>> IMO it's better to use a low-level allocator (eg. uvm_km_alloc) for
>>>>> alignment-sensitive users.
>>>>
>>>> While for page-size alignment, it makes sense to use uvm_km(9) allocator,
>>>> there are quite a few allocations when it is useful to give a separate
>>>> cache-line for a structure.  I am not sure if it is desirable to sprinkle
>>>> pad & align magic each time in the caller's side, instead of adding such
>>>> support to kmem(9).  Perhaps kmem_cacheline_{alloc,free}?
>>>>
> 
> It's not api-wise guaranteed, but what do we loose if we get those
> alignments? Nothing I think, they are in place with the OpenSolaris
> implementation as well ;-)
> 
>> we loose the flexibility of the implementation.
> 

With the flexibility you mean(?) that we can allocate lets say something
like 5200 bytes (as 8 or 64 byte aligned memory) in both my patches it
would e propagated to 8192 bytes eg?
I see the point there. If that's the case I still think it would be
better with the cache-line-aligned allocations and once we have sizes
greater than the kmem-pools we switch to a, not yet there arena, with a
quantum-size of cache-line-size for large allocations.
This way we avoid having a lot of boundary tags for small allocations
(because of small pool-page-sizes in the current implementation) and
retain this flexibility.

> I don't think it's a good idea to have a different api for
> cache-line-aligned memory, this would require different pools for
> cache-size-aligned memory and not cache-line-aligned memory just
> spreading out the allocations and increasing fragmentation.
> 
> If the kva is controlled by a vmem arena then those page-size aligned
> allocation should go to that arena, which quantum is page-size anyway.
> 
>> i guess page-alignment and cache-line-alignemnt should be considered
>> separately.
>> i currently have no idea how many of kmem_alloc users are sensitive to
>> cache-line-alignment.
> 

Those two are distinct except for that page-alignment includes
cache-line-alignment (at least I don't know of any architecture other
than this) ;-)
Putting allocations in different cache-lines will have a general
benefiting effect.

>> YAMAMOTO Takashi
> 
> 
> Lars

Lars