Subject: tmpfs: Storing file contents
To: None <firstname.lastname@example.org>
From: Julio M. Merino Vidal <email@example.com>
Date: 08/12/2005 12:27:56
during the last days, I've implemented very clumsy read/write vnode
operations for tmpfs, just to familiarize myself with uiomove and the
like (they are not in the CVS). I've also been reading the first chapters
of Cranor's UVM dissertation to see how to manage anonymous
memory, although I still have many doubts.
The thing is that I don't know how to store file contents in memory
to comply to the following requirements:
- Use pageable (unwired) memory (i.e., anonymous memory, right?).
- Avoid multiple copies of the same data on memory.
- Be careful to not introduce a lot of overhead on the memory manager.
I see two possibilities as regards how to manage each file:
- One of them is to allocate space page by page (just as a file-system
allocates blocks) and attach these pages to the files needing them.
The fact that they are pages does not matter in the idea: we'd simply
be dealing with a structure whose size matches a page, but that's all.
That is, keep a sorted list (or another structure with better random
access times) of pages (blocks) that contain the file. This'd be like
having a pool of fixed size structures, but over pageable memory.
In order to do this, I'd be nice if I could allocate a big address space
and allocate/deallocate individual pages easily.
- Have an independent virtual address space for each file, backed by
anonymous memory, so that reads and writes are trivial: just read
and write from memory at a specific offset within the address
space. Page mapping could be automatic upon faults (extra steps
could be needed to unmap unused pages to avoid swap leaks).
I don't know if this implementation is possible at all, or if it could
cause too much overhead on UVM... but if possible, it'd simplify
things a lot.
- Of course, there may be other better possibilities, but I can't see
them with my current knowledge.
The problem is that... aside not knowing which approach should I
follow, I don't know how to code them as regards memory
management. I've read the uvm(9) manpage and searched for
usage examples of its functions within the kernel. Unfortunately,
I can't find much: just some calls in process management and
SHM... but these don't seem to be what I need (or at least I can't
find the appropriate examples).
Some of my doubts are...
- Do I have to keep an aobj for each file? If so, aobj's are created
with uao_create, right? Which size should they have (as it's not
- Once I have an aobj, how do I map space within it? Maybe I have
to use uvm_map, but I'm afraid that calling that function to request
single pages could introduce a lot of overhead... Or can it be
done in a more automated way, as I described in the second
- When some people mentioned that I should avoid having multiple
copies of the same data in memory, they were referring to one
copy of the data managed by the filesystem and another one
stored inside the vnode's uobj, right? If so, can't this be avoided
by having a getpages operation that simply loans pages from the
filesystem to the vnode (thus just keeping one real copy)?
I'm sorry for so many questions, but I'm really lost in this area.
I will appreciate any suggestion, pointers to documentation or code,
or even detailed explanations of the process I should follow (e.g.,
which is the basic idea to store the files, which functions should I
look at, etc.)...
Thank you very much,
PS: FWIW, I've been playing with uao_create and uvm_map, and been
able to allocate pageable memory (can see how swap is used when
storing lots of data). What I've done is towards the first approach
described in the mail, though...
Julio M. Merino Vidal <firstname.lastname@example.org>
The NetBSD Project - http://www.NetBSD.org/