Re: kmem-pool-uvm

[Lars Heidieker <lars%heidieker.de@localhost>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

On 04/14/11 09:05, YAMAMOTO Takashi wrote:
> why do you want to make subr_kmem use uvm_km directly?
> to simplify the code?
> i don't want to see that change, unless there's a clear benefit.
>
The reason was to simplify the code, yes, and reduce redundancy
because in the current implementation the vmem allocates PAGE_SIZE
memory from the uvm_km backend for requests <= PAGE_SIZE not utilizing
the vacache and more importantly vmem is essentially just taking the
address allocations made by uvm_map.
With the changes I see about 15% less kernel map entries.
> let me explain some background. currently there are a number of
> kernel_map related problems:
>
> A-1. vm_map_entry is unnecessarily large for KVA allocation purpose.
>
> A-2. kernel-map-entry-merging is there to solve A-1. but it introduced
> the allocate-for-free problem. ie. to free memory, you might need to
> split map-entries thus allocate some memory.
>
> A-3. to solve A-2, there is map-entry-reservation mechanism. it's
complicated
> and broken.
>
> B. kernel fault handling is complicated because it needs memory allocation
> (eg. vm_anon) which needs some trick to avoid deadlock.
>
> C. KVA allocation is complicated because it needs memory allocation
> (eg. vm_map_entry) which needs some trick to avoid deadlock.
>
> the most of the above can be solved by separating KVA allocation and
> kernel fault handling. (except C, which will be merely moved to a
> different place.)
>
A-1 with vmem_btag being slightly less then half the size of
vm_map_entry...
A-2 solves A1 but A-3 solves A2 with the pitfall of reintroducing a
part of A1 as we still have less map entries in the map but we don't
save memory as all the entries not in the map cached aside for
potential merging.
In this sense it seems broken to me and that it is complicated.
Reducing the overall allocated map_entries will help here, as vacaches do.

C seems to be inevitable it's only a question where it happens...

B is a result of having pageable memory, which can fault and
non-pageable memory in the same map, with the need to allocated
non-pageable memory in the event of a page fault.
> i implemented subr_vmem so that eventually it can be used as the primary
> KVA allocator. ie. when allocating from kernel_map, allocate KVA from
> kernel_va_arena first and then, if and only if necessary, register it to
> kernel_map for fault handling. it probably allows us to remove VACACHE
> stuff, too. kmem_alloc will be backed by a vmem arena which is backed by
> kernel_va_arena.
>
Originally I thought about two options with option one being what my
patch does and two:

If vmem is made the primary kva allocator, we should carve out a
kernel heap entirely controlled by vmem, probably one special
vm_map_entry in the kernel_map that spans the heap or a submap that
never has any map_entries.
Essentially separating pageable and non-pageable memory allocations,
this would allow for removing the vacaches in the kernel-maps as well
as the map-entry-reservation mechanism.

Questions that follow:
- - how to size it probably.....
- - this might be the kmem_map? or two heaps an interrupt safe one and
one non interrupt safe?

I think having two "allocators" (vmem and the vm_map_(entries) itself)
controlling the kernel_map isn't a good idea as both have to be in
sync, at least every allocation that is made by vm_map_entries need to
be made in vmem as well. There is no clear responsibility for either.

Option two is more challenging and will solve problems B and As while
option one solves most of the As leaving B untouched.

Lars

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2oFQUACgkQcxuYqjT7GRb4eACgt0ra+vpcQx8UTOivOgZcpsQe
Nl0AoK8YnsJoYS5wdaSidLLB0OifWqeI
=xLwx
-----END PGP SIGNATURE-----