Hi all,

during the last days, I've implemented very clumsy read/write vnode
operations for tmpfs, just to familiarize myself with uiomove and the
like (they are not in the CVS).  I've also been reading the first chapters
of Cranor's UVM dissertation to see how to manage anonymous
memory, although I still have many doubts.

The thing is that I don't know how to store file contents in memory
to comply to the following requirements:
- Use pageable (unwired) memory (i.e., anonymous memory, right?).
- Avoid multiple copies of the same data on memory.
- Be careful to not introduce a lot of overhead on the memory manager.

I see two possibilities as regards how to manage each file:

- One of them is to allocate space page by page (just as a file-system
  allocates blocks) and attach these pages to the files needing them.
  The fact that they are pages does not matter in the idea: we'd simply
  be dealing with a structure whose size matches a page, but that's all.
  That is, keep a sorted list (or another structure with better random
  access times) of pages (blocks) that contain the file.  This'd be like
  having a pool of fixed size structures, but over pageable memory.
  In order to do this, I'd be nice if I could allocate a big address space
  and allocate/deallocate individual pages easily.

- Have an independent virtual address space for each file, backed by
  anonymous memory, so that reads and writes are trivial: just read
  and write from memory at a specific offset within the address
  space.  Page mapping could be automatic upon faults (extra steps
  could be needed to unmap unused pages to avoid swap leaks).
  I don't know if this implementation is possible at all, or if it could
  cause too much overhead on UVM... but if possible, it'd simplify
  things a lot.

- Of course, there may be other better possibilities, but I can't see
  them with my current knowledge.

The problem is that... aside not knowing which approach should I
follow, I don't know how to code them as regards memory
management.  I've read the uvm(9) manpage and searched for
usage examples of its functions within the kernel.  Unfortunately,
I can't find much: just some calls in process management and
SHM... but these don't seem to be what I need (or at least I can't
find the appropriate examples).

Some of my doubts are...
- Do I have to keep an aobj for each file?  If so, aobj's are created
  with uao_create, right?  Which size should they have (as it's not
  known beforehand)?
- Once I have an aobj, how do I map space within it?  Maybe I have
  to use uvm_map, but I'm afraid that calling that function to request
  single pages could introduce a lot of overhead...  Or can it be
  done in a more automated way, as I described in the second
  approach above?
- When some people mentioned that I should avoid having multiple
  copies of the same data in memory, they were referring to one
  copy of the data managed by the filesystem and another one
  stored inside the vnode's uobj, right?  If so, can't this be avoided
  by having a getpages operation that simply loans pages from the
  filesystem to the vnode (thus just keeping one real copy)?

I'm sorry for so many questions, but I'm really lost in this area.
I will appreciate any suggestion, pointers to documentation or code,
or even detailed explanations of the process I should follow (e.g.,
which is the basic idea to store the files, which functions should I
look at, etc.)...

Thank you very much,

PS: FWIW, I've been playing with uao_create and uvm_map, and been
able to allocate pageable memory (can see how swap is used when
storing lots of data).  What I've done is towards the first approach
described in the mail, though...

--=20
Julio M. Merino Vidal <jmmv84@gmail.com>
http://www.livejournal.com/users/jmmv/
The NetBSD Project - http://www.NetBSD.org/