[src/trunk]: src/lib/librumpuser document the hypercall interface

[pooka <pooka%NetBSD.org@localhost>]

details:   https://anonhg.NetBSD.org/src/rev/3bc935e3408c
branches:  trunk
changeset: 786531:3bc935e3408c
user:      pooka <pooka%NetBSD.org@localhost>
date:      Tue Apr 30 21:18:40 2013 +0000

description:
document the hypercall interface

diffstat:

 lib/librumpuser/rumpuser.3 |  601 ++++++++++++++++++++++++++++++++++++++++++--
 1 files changed, 573 insertions(+), 28 deletions(-)

diffs (truncated from 626 to 300 lines):

diff -r 0b18b4317601 -r 3bc935e3408c lib/librumpuser/rumpuser.3
--- a/lib/librumpuser/rumpuser.3        Tue Apr 30 21:03:43 2013 +0000
+++ b/lib/librumpuser/rumpuser.3        Tue Apr 30 21:18:40 2013 +0000
@@ -1,6 +1,6 @@
-.\"     $NetBSD: rumpuser.3,v 1.2 2010/03/01 17:20:44 pooka Exp $
+.\"     $NetBSD: rumpuser.3,v 1.3 2013/04/30 21:18:40 pooka Exp $
 .\"
-.\" Copyright (c) 2010 Antti Kantee.  All rights reserved.
+.\" Copyright (c) 2013 Antti Kantee.  All rights reserved.
 .\"
 .\" Redistribution and use in source and binary forms, with or without
 .\" modification, are permitted provided that the following conditions
@@ -23,42 +23,587 @@
 .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 .\" SUCH DAMAGE.
 .\"
-.Dd March 1, 2010
+.Dd April 30, 2013
 .Dt RUMPUSER 3
 .Os
 .Sh NAME
 .Nm rumpuser
-.Nd rump hypervisor interface
+.Nd rump kernel hypercall interface
 .Sh LIBRARY
 rump User Library (librumpuser, \-lrumpuser)
 .Sh SYNOPSIS
 .In rump/rumpuser.h
 .Sh DESCRIPTION
-.Nm
-is the hypervisor interface for
-.Xr rump 3
-style kernel virtualization.
-A virtual rump kernel can make calls to the host operating system
-libraries and kernel (system calls) using
-.Nm
-interfaces.
-Any "slow" hypervisor calls such as file I/O, sychronization wait,
-or sleep will cause rump to unschedule the calling kernel thread
-from the virtual CPU and free it for other consumers.
-When the hypervisor call returns to the kernel, a new scheduling
-operation takes place.
-.Pp
-For example, rump implements kernel threads directly as hypervisor
-calls to host
-.Xr pthread 3 .
-This avoids the common virtualization drawback of multiple overlapping
-and possibly conflicting implementations of same functionality in
-the software stack.
-.Pp
 The
 .Nm
-interface is still under development and interface documentation
-is available only in source form from
-.Pa src/lib/librumpuser .
+hypercall interfaces allow a rump kernel to access host resources.
+A hypervisor implementation must implement the routines described in
+this document to allow a rump kernel to run on the host.
+The implementation included in
+.Nx
+is for POSIX hosts.
+This document is divided into sections based on the functionality
+group of each hypercall.
+.Sh UPCALLS AND RUMP KERNEL CONTEXT
+A hypercall is always entered with the calling thread scheduled in
+the rump kernel.
+In case the hypercall intends to block while waiting for an event,
+the hypervisor must first release the rump kernel scheduling context.
+In other words, the rump kernel context is a resource and holding
+on to it while waiting for a rump kernel event/resource may lead
+to a deadlock.
+Even when there is no possibility of deadlock in the strict sense
+of the term, holding on to the rump kernel context while performing
+a slow hypercall such as reading a device will prevent other threads
+(including the clock interrupt) from using that rump kernel context.
+.Pp
+Releasing the context is done by calling the
+.Fn hyp_backend_unschedule
+upcall which the hypervisor received from rump kernel as a parameter
+for
+.Fn rumpuser_init .
+Before a hypercall returns back to the rump kernel, the returning thread
+must carry a rump kernel context.
+In case the hypercall unscheduled itself, it must reschedule itself
+by calling
+.Fn hyp_backend_schedule .
+.Sh HYPERCALL INTERFACES
+.Ss Initialization
+.Ft int
+.Fn rumpuser_init "int version" "struct rump_hyperup *hyp"
+.Pp
+Initialize the hypervisor.
+.Bl -tag -width "xalignmentx"
+.It Fa version
+hypercall interface version number that the kernel expects to be used.
+In case the hypervisor cannot provide an exact match, this routine must
+return a non-zero value.
+.It Fa hyp
+pointer to a set of upcalls the hypervisor can make into the rump kernel
+.El
+.Ss Memory allocation
+.Ft int
+.Fn rumpuser_malloc "size_t len" "int alignment" "void **memp"
+.Bl -tag -width "xalignmentx"
+.It Fa len
+amount of memory to allocate
+.It Fa alignment
+size the returned memory must be aligned to.
+For example, if the value passed is 4096, the returned memory
+must be aligned to a 4k boundary.
+.It Fa memp
+return pointer for allocated memory
+.El
+.Pp
+.Ft void
+.Fn rumpuser_free "void *mem" "size_t len"
+.Bl -tag -width "xalignmentx"
+.It Fa mem
+memory to free
+.It Fa len
+length of allocation.
+This is always equal to the amount the caller requested from the
+.Fn rumpuser_malloc
+which returned
+.Fa mem .
+.El
+.Ss Files and I/O
+.Ft int
+.Fn rumpuser_open "const char *name" "int mode" "int *fdp"
+.Pp
+Open a file for I/O.
+Notably, there needs to be no mapping between
+.Fa name
+and the host, but for example on a POSIX system it may be convenient
+to let
+.Fa name
+denote the host file system namespace.
+.Bl -tag -width "xalignmentx"
+.It Fa name
+the identifier of the file to open for I/O
+.It Fa mode
+combination of the following:
+.Bl -tag -width "XRUMPUSER_OPEN_CREATE"
+.It Dv RUMPUSER_OPEN_RDONLY
+open only for reading
+.It Dv RUMPUSER_OPEN_WRONLY
+open only for writing
+.It Dv RUMPUSER_OPEN_RDWR
+open for reading and writing
+.It Dv RUMPUSER_OPEN_CREATE
+do not treat missing
+.Fa name
+as an error
+.It Dv RUMPUSER_OPEN_EXCL
+combined with
+.Dv RUMPUSER_OPEN_CREATE ,
+flag an error if
+.Fa name
+already exists
+.It Dv RUMPUSER_OPEN_BIO
+the caller will use this file for block I/O, usually used in
+conjunction with accessing file system media.
+The hypervisor should treat this flag as advisory and possibly
+enable some optimizations for
+.Fa *fdp
+based on it.
+.El
+Notably, the permissions of the created file are left up to the
+hypervisor implementation.
+.It Fa fdp
+An integer value denoting the open file is returned here.
+.El
+.Pp
+.Ft int
+.Fn rumpuser_close "int fd"
+.Pp
+Close a previously opened file descriptor.
+.Pp
+.Ft int
+.Fn rumpuser_getfileinfo "const char *name" "uint64_t *size" "int *type"
+.Bl -tag -width "xalignmentx"
+.It Fa name
+file for which information is returned.
+The namespace is equal to that of
+.Fn rumpuser_open .
+.It Fa size
+If non-NULL, size of the file is returned here.
+.It Fa type
+If non-NULL, type of the file is returned here.
+The options are
+.Dv RUMPUSER_FT_DIR ,
+.Dv RUMPUSER_FT_REG ,
+.Dv RUMPUSER_FT_BLK ,
+.Dv RUMPUSER_FT_CHR ,
+or
+.Dv RUMPUSER_FT_OTHER
+for directory, regular file, block device, character device or unknown,
+respectively.
+.El
+.Pp
+.Ft void
+.Fo rumpuser_bio
+.Fa "int fd" "int op" "void *data" "size_t dlen" "off_t off"
+.Fa "rump_biodone_fn biodone" "void *donearg"
+.Fc
+.Pp
+Initiate block I/O and return immediately.
+.Bl -tag -width "xalignmentx"
+.It Fa fd
+perform I/O on this file descriptor.
+The file descriptor must have been opened with
+.Dv RUMPUSER_OPEN_BIO .
+.It Fa op
+Transfer data from the file descriptor with
+.Dv RUMPUSER_BIO_READ
+and transfer data to the file descriptor with
+.Dv RUMPUSER_BIO_WRITE .
+Unless
+.Dv RUMPUSER_BIO_SYNC
+is specified, the hypervisor may cache a write instead of
+committing it to permanent storage.
+.It Fa data
+memory address to transfer data to/from
+.It Fa dlen
+length of I/O.
+The length is guaranteed to be a multiple of 512.
+.It Fa off
+offset into
+.Fa fd
+where I/O is performed
+.It Fa biodone
+To be called when the I/O is complete.
+Accessing
+.Fa data
+is not legal after the call is made.
+.It Fa donearg
+opaque arg that must be passed to
+.Fa biodone .
+.El
+.Pp
+.Ft int
+.Fo rumpuser_iovread
+.Fa "int fd" "struct rumpuser_iovec *ruiov" "size_t iovlen"
+.Fa "off_t off" "size_t *retv"
+.Fc
+.Pp
+.Ft int
+.Fo rumpuser_iovwrite
+.Fa "int fd" "struct rumpuser_iovec *ruiov" "size_t iovlen"
+.Fa "off_t off" "size_t *retv"
+.Fc
+.Pp
+These routines perform scatter-gather I/O which is not
+block I/O by nature and therefore cannot be handled by
+.Fn rumpuser_bio .
+.Pp
+.Bl -tag -width "xalignmentx"
+.It Fa fd
+file descriptor to perform I/O on
+.It Fa ruiov
+an array of I/O descriptors.
+It is defined as follows:
+.Bd -literal -offset indent -compact
+struct rumpuser_iovec {
+       void *iov_base;
+       size_t iov_len;
+};
+.Ed
+.It Fa iovlen
+number of elements in
+.Fa ruiov
+.It Fa off
+offset of
+.Fa fd
+to perform I/O on.
+This can either be a non-negative value or
+.Dv RUMPUSER_IOV_NOSEEK .
+The latter denotes that no attempt to change the underlying objects
+offset should be made.
+Using both types of offsets on a single instance of
+.Fa fd
+results in undefined behavior.
+.It Fa retv
+number of bytes successfully transferred is returned here
+.El
+.Ss Clocks
+The hypervisor should support two clocks, one for wall time and one
+for monotonically increasing time, the latter of which may be based
+on some arbitrary time (e.g. system boot time).
+If this is not possible, the hypervisor must make a reasonable effort to
+retain semantics.
+.Pp
+.Ft int
+.Fn rumpuser_clock_gettime "enum rumpclock clk" "uint64_t *sec" "uint64_t *nsec"
+.Pp
+.Bl -tag -width "xalignmentx"
+.It Fa clk
+specifies the clock type.
+In case of
+.Dv RUMPUSER_CLOCK_RELWALL