At Wed, 3 Jun 2015 06:01:27 +0000, David Holland <dholland-tech%netbsd.org@localhost> wrote:
Subject: memory-mapped I/O (was: Re: Removing ARCNET stuffs)
>
> On Mon, Jun 01, 2015 at 02:23:15PM -0700, Greg A. Woods wrote:
> > That reminds me though.... I've had some ideas of how one might
> > implement stdio solely using memory mapped I/O (which I was thinking of
> > more for the purpose of how one might support POSIX on a Multics-like
> > system, but perhaps it has advantages for a Unix-y system with memmap).
>
> People who are working on nvram storage (phase-change memory,
> memristors, stuff like that) will like this, as they have pretty much
> taken it as an article of faith that all I/O needs to be memory-mapped
> to avoid all the horrible overhead of going through the kernel for
> data transfer.
Hmmm... that's interesting....
> > Of course if one were actually implementing for a virtual memory based
> > system like Multics then one would not use silly old stdio and other
> > file stream abstractions -- there's just no point to contort all one's
> > algorithms into that corset of an abstraction.
>
> We've had mmap in Unix for going on 30 years, and it remains a poor
> second cousin. At least for writing. Coincidence? You decide.
Yeah, well a proper from-the-ground-up universally virtual-memory
operating system has some major advantages over Unix with mmap, not to
mention that mmap itself wasn't available in widely used Unix systems
until somewhat more recently, and still seems to suffer
incompatibilities. I.e. mmap specifically on Unix doesn't loosen the
corset very much at all -- it's just not worth the effort to use it
vs. using stream I/O.
I think it takes a bigger paradigm shift in thinking than most people
expect it will take to fully appreciate what it means when you never
actually do a "write" for anything at all, ever (well, except for
serialised devices, such as terminals and networks). You just put bits
in the places you want them and, well, there is no "and" -- you are
_done_.
When I was a Multics user I struggled constantly with understanding what
it meant to access all storage as virtual memory -- I had already been
deeply indoctrinated with the paradigm that "everything is a file and
all files are streams of data that cannot be directly addressed" (as
well all too much of the user-level view of Multics catered to the
stream I/O paradigm because that's what the rest of the world did). It
wasn't until much later -- long after the last Multics was shut down,
that I finally "got it".
Files are not (ever!) truly accessed with I/O -- not even devices use
I/O in the manner we think of in Unix.
On Multics, for example, you don't ever open a file and get a file
descriptor and then read and write (and seek), and finally close using
that descriptor as a handle which has an implied cursor position (unless
you're using a user-level abstraction that conjures this abstraction).
Or alternately you don't have to ask for some (or all, if it will fit)
of that file to be mapped into your process memory space after you've
opened it.
You just ask the filesystem what region of memory is represented by some
name you know it as and if you can have access to it, and the filesystem
and memory manager arrange it so that your process won't get a SIGSEGV
when you access that memory. That file is (and always was and always
will be) just an array in memory. (of course there are some slightly
messier details if you're using segmented addressing architecture, which
is what the real Multics was designed for at the time, but those are
implementation details, not necessarily pre-requisites to the design)
We can do this in some sense with mmap, but with true universal virtual
memory there are no issues with trying to maintain consistency between
the stream and mmap interface. There are no issues with the size of the
file vs. limits in the virtual size of your process. There is no issue
with tracking if either swap or some specific filesystem is backing the
region (it's always _the_ filesystem). You don't even have to close the
file, or give up access to one large file in order to access another
large file. In theory you can access _all_ the files in the whole
system simultaneously. _Everything_ is just, and always, a memory
address (except again for serialised devices -- they are actually other
programs that you talk to and which use device-independent protocols to
provide and manage access to the underlying hardware devices).
The equivalent of combining open()+mmap() is effectively the _only_
primary data access interface in Multics -- anything else like open(),
read(), write(), etc. is just an unnecessary library abstraction on
arrays of memory.
Multics had its own issues, some to do with constraints of the hardware
of the day, and as a result it did require some special features in the
hardware to optimize certain operations, features we may still lack in
modern hardware. There are some good papers about the real-world
problems Multics faced and some include ideas of how Multics and its
hardware could have been improved.
Unix went and turned everything from Multics upside down, so to speak,
and made everything _including_ serialised and non-serialised devices
all look like files that do stream I/O (*). And then someone had the
silly idea to add mmap back on top of (some) files. That's where
your(our) problem is!
A universally virtual memory based system doesn't use mmap as an extra
abstraction layer on top of a file. There is only mmap, and there are
no files per se.
One of the oft-heard arguments against using mmap more in Unix is that
copying data in memory isn't as expensive as one might think vs. the
overheads of virtual memory management, and it's true in Unix but that's
not really the point if your whole system deals only via virtual memory
for all data on secondary storage devices.
Obviously the OS has to copy data from secondary storage to main memory
and back. The question is how does it manage access to main memory and
how many more times does the OS or a user program have to copy bits of a
file within main memory to access, process, and write; and is that
overhead of copying and additional memory requirements higher than the
overhead of managing page table entries and other such VM and cache
management overhead. It's not really fair to compare mmap in Unix with
a true from-the-ground-up universally virtual memory OS. Just getting
rid of the overhead of a separate user-level ld.so process running for
every exec() can pay huge dividends, for example.
(*) well Multics already had device-independent I/O, but in a different
way via what might best be called device driver programs, and
Multics of course supported traditional stream I/O abstractions for
languages such as FORTRAN which had such concepts embedded in them,
which is were my thinking about POSIX support for Multics-like
systems started
--
Greg A. Woods
Planix, Inc.
<woods%planix.com@localhost> +1 250 762-7675 http://www.planix.com/
Attachment:
pgpEkImf4Fo15.pgp
Description: PGP signature