Re: Alloc userspace process to access mapped grant memory regions

[Roger Pau Monné <roger.pau%citrix.com@localhost>]

On 23/11/12 22:26, Manuel Bouyer wrote:
> On Fri, Nov 23, 2012 at 08:35:27PM +0100, Roger Pau Monné wrote:
>> Hello,
>>
>> I'm trying to port something similar to Linux gntdev to NetBSD, but I'm
>> finding some problems. The basic functionality of gntdev is that using a
>> set of ioctls you can map a grant ref, and then you can access it doing
>> a mmap to the gntdev pseudo-device. I've tried to do the same in NetBSD,
>> but I'm finding some problems in the mmap device handler. What I've done
>> is basically the following (very simplified of course):
>>
>> paddr_t
>> gntdev_mmap(...) {
>>      xen_shm_map(nentries, domid, grefp, &va, handlep, flags);
>>      pmap_extract(pmap_kernel(), va, &pa);
>>      return x86_btop(pa);
>> }
>>
>> This however doesn't work. The user-space caller ends up with a vaddr
>> that points to the same physical address as the kernel vaddr (so pa is
>> equal in kernel and userspace), but the userspace vaddr doesn't point to
>> the same mfn as the kernel vaddr (va in the example).
> 
> I don't follow you here. pa is the physical address, I'm not sure how you
> can compare that is virtual user or kernel spaces.
> 
>> Does someone know
>> of a way of returning a mfn instead of a pfn from the device mmap
>> handler? Or is there anyway to get the vaddr that will be returned to
>> the userspace program, so I can map the mfn directly to that vaddr?
> 
> returning the physical address should work. Did you check that the pa you
> get from pmap_extract() points to the right mfn ?

Thanks for the help, I've changed the code a little bit and tried doing
something similar to what is done in the mmap function of xenevt:

(inside device mmap handler):
xen_shm_map(nentries, domid, grefp, &va, handlep, flags);
pmap_extract_ma(pmap_kernel(), map->va, &ma)
printf("Kernel MA: %p\n", (void *)ma);
return x86_btop(xpmap_mtop(ma));

This device also has an ioctl where the user-space process can pass a VA
(in the user space) and get the grant associated with this VA (basically
the passed VA is the return value of mmap(gntdev...)). I do the
following in the ioctl handler:

(passed_va is the user provided VA through the ioctl)

paddr_t u_pa, k_pa;
paddr_t u_ma, k_ma;
pmap_t user_pmap = vm_map_pmap(&user_lwp->l_proc->p_vmspace->vm_map);

pmap_extract(pmap, passed_va, &u_pa);
foreach(kernel_va, list_of_mapped_grants) {
        /* kernel_va is map->va in the upper example */
        pmap_extract(pmap_kernel(), kernel_va, &k_pa);
        if (k_pa == u_pa) {
                printf("Find matching PA\n");
                pmap_extract_ma(pmap, passed_va, &process_ma);
                pmap_extract_ma(pmap_kernel(), kernel_va, &kernel_ma);
                printf("Kernel MA: %p - Userspace MA: %p\n",
                        (void *)k_ma, (void *)u_ma);
        }
}

The output when performing the mmap is the following:

Kernel MA: 0xb24c9000

And the output when the user performs the ioctl to find the gref of the
user-space VA is:

Find matching PA
Kernel MA: 0xb24c9000 - Userspace MA: 0xbda71000

So it seems like I return the correct MA in the mmap handler, but the
userspace process gets a wrong MA associated with it's VA?

Thanks, Roger.

(I don't post the full device code to simplify the debugging a little
bit, but if it's going to help I don't have any problem in posting it)