Re: tracking P->V for unmanaged device pages

[Taylor R Campbell <campbell+netbsd-tech-kern%mumble.net@localhost>]

   Date: Thu, 9 Apr 2015 09:45:15 +0900
   From: Masao Uebayashi <uebayasi%gmail.com@localhost>

   On Wed, Apr 8, 2015 at 11:34 PM, Taylor R Campbell
   <campbell+netbsd-tech-kern%mumble.net@localhost> wrote:
   >    Date: Wed, 8 Apr 2015 12:29:13 +0900
   >    From: Masao Uebayashi <uebayasi%gmail.com@localhost>
   >
   >    Considering these, PV should be kept in struct vm_physseg, instead of
   >    struct vm_page, ideally.
   >
   > The only difficulty with that is that you still need a fast way to go
   > from a struct vm_page to the PV tracking record.  Since vm_page has no
   > pointer to vm_physseg, you'd need to add one in struct vm_page_md.
   > But then you might as well just include the PV tracking record in the
   > struct vm_page_md anyway.

   If you know a) the base address of struct vm_physseg, and b) the
   offset within it, you can look up the matching PV entry (in vector or
   table or whatever) fast.

Yes -- that's what I meant by `Since vm_page has no pointer to
vm_physseg, you'd need to add one in struct vm_page_md.'  Of course,
it could instead be put in the MI part of struct vm_page, &c.

   > In my pmap_pv API, there's no need to change pmap_enter.  All you need
   > to do is to call pmap_pv_track on driver attach to mark the pages as
   > device pages.  Then pmap can do the bookkeeping inside.

   You don't need to change pmap_enter(), but you need to change fault
   handlers.  That didn't really work for XIP, which has to share the
   generic fault handler.

I don't understand.  Why do you need to change fault handlers?  I
think the only real reason the drm drivers need custom fault handlers
is that they need to hold a lock over pinning graphics buffers in GPU
VA and entering the corresponding aperture PAs into the pmap, and
neither the cdev_mmap nor pgo_getpages/putpages abstraction supports
this.

Is the uebayasi-xip branch relevant?  Are there changes there which
might be instructive?

   > It sounds like what you're proposing is to replace pmap_pv_track by a
   > version of uvm_page_physload that will not allocate struct vm_page for
   > each page in the region, but only struct vm_pv.  I have no objection
   > to this either, but someone has to do the work.

   It doesn't need to be done quickly.  I just want to know the
   direction.  The abstraction of physical address and memory in UVM is
   very poor ATM.  I want to change that, and want awareness of the
   problem.  (Future memory technologies like persistent memory is coming
   soon.)

I don't have enough background to make any statements about the
direction.