[src/trunk]: src/share/man/man4 expand driver description

[jdolecek <jdolecek%NetBSD.org@localhost>]

details:   https://anonhg.NetBSD.org/src/rev/6935c5bc9141
branches:  trunk
changeset: 818043:6935c5bc9141
user:      jdolecek <jdolecek%NetBSD.org@localhost>
date:      Wed Sep 21 20:01:03 2016 +0000

description:
expand driver description

diffstat:

 share/man/man4/nvme.4 |  111 +++++++++++++++++++++++++++++++++++++++++--------
 1 files changed, 93 insertions(+), 18 deletions(-)

diffs (145 lines):

diff -r 84621b35357c -r 6935c5bc9141 share/man/man4/nvme.4
--- a/share/man/man4/nvme.4     Wed Sep 21 19:18:01 2016 +0000
+++ b/share/man/man4/nvme.4     Wed Sep 21 20:01:03 2016 +0000
@@ -1,4 +1,4 @@
-.\"    $NetBSD: nvme.4,v 1.4 2016/09/16 12:43:37 wiz Exp $
+.\"    $NetBSD: nvme.4,v 1.5 2016/09/21 20:01:03 jdolecek Exp $
 .\"    $OpenBSD: nvme.4,v 1.2 2016/04/14 11:53:37 jmc Exp $
 .\"
 .\" Copyright (c) 2016 David Gwynne <dlg%openbsd.org@localhost>
@@ -15,7 +15,7 @@
 .\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
 .\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
 .\"
-.Dd September 16, 2016
+.Dd September 21, 2016
 .Dt NVME 4
 .Os
 .Sh NAME
@@ -29,10 +29,77 @@
 driver provides support for NVMe, or NVM Express,
 storage controllers conforming to the
 Non-Volatile Memory Host Controller Interface specification.
+Controllers complying to specification version 1.1 and 1.2 are known to work.
+Other versions should too for normal operation with exception of some
+passthrough commands.
+.Pp
+Driver supports following features:
+.Bl -bullet -compact -offset indent
+.It
+controller and namespace configuration and management using
+.Xr nvmectl 1
+.It
+highly parallel I/O using per-CPU I/O queues
+.It
+PCI MSI/MSI-X attachment, and INTx for legacy systems
+.El
+.Pp
+On systems supporting MSI/MSI-X,
+.Nm
+driver uses per-CPU IO queue pairs for lockless and highly parallelized I/O.
+Interrupt handlers are scheduled on distinct CPUs.
+Driver allocates as many interrupt vectors as available, up to number
+of CPUs + 1.
+MSI supports up to 32 interrupt vectors within the system,
+MSI-X can have up to 2k.
+Each I/O queue pair has separate command circular buffer.
+.Nm
+specification allows up to 64k commands per queue, driver currently allocates
+1024 items per queue by default.
+Command submissions are done always on current CPU, command completion
+interrupt is handled on CPU according to I/O queue ID - first I/O queue on CPU0,
+second I/O queue on CPU1 etc.
+Admin queue command completion is not tied to any CPU, it's handled by
+any CPU.
+To keep lock contention to minimum, it's recommended to keep this assignment,
+even thought it is possible to reassign the interrupt handlers differently,
+using
+.Xr intrctl 1 .
+Driver also uses soft interrupts to process command completions, in order to
+increase total system I/O capacity and throughput.
+.Pp
+On systems without MSI, driver uses single HW interrupt handler, for
+both admin and standard I/O commands.
+Command submissions are done on current CPU, command completion interrupt
+is handled on any available CPU. This leads to some lock contention,
+especially on command ccbs.
+Also, command completion handling must be done within the HW interrupt
+handler.
+.Sh FILES
+.Bl -tag -width /dev/nvmeX -compact
+.It Pa /dev/nvme*
+nvme device special files used by
+.Xr nvmectl 1 .
+.El
 .Sh SEE ALSO
 .Xr intro 4 ,
 .Xr ld 4 ,
-.Xr pci 4
+.Xr pci 4 ,
+.Xr nvmectl 1 ,
+.Xr MAKEDEV 8 ,
+.Xr intrctl 1
+.Rs
+.%A NVM Express, Inc.
+.%T "NVM Express \- scalable, efficient, and industry standard"
+.%D 2016-06-12
+.%U http://nvmexpress.org/
+.Re
+.Rs
+.%A NVM Express, Inc.
+.%T "NVM Express Revision 1.2.1"
+.%D 2016-06-05
+.%U http://www.nvmexpress.org/wp-content/uploads/NVM_Express_1_2_1_Gold_20160603.pdf
+.Re
 .Sh HISTORY
 The
 .Nm
@@ -54,24 +121,32 @@
 by
 .An NONAKA Kimihiro
 .Aq Mt nonaka%NetBSD.org@localhost .
-.Sh BUGS
-Emulated
-.Nm
-device under QEMU doesn't currently work - controller identify command times
-out.
-.Pp
-Emulated
+.An Jaromir Dolecek
+.Aq Mt jdolecek%NetBSD.org@localhost
+contributed to making this driver MPSAFE.
+.Sh NOTES
+At least some
+.Tn Intel
 .Nm
-device under
+adapter cards are known to require
+.Tn PCIe
+Generation 3 slot. Such cards do not even probe when plugged
+into older generation slot.
+.Pp
+Driver attaches and works fine also for emulated
+.Nm
+device under QEMU and
 .Tn Oracle
+.Tn VirtualBox .
+However,
+there seems to be some broken interaction between
 .Tn VirtualBox
-attaches and works fine, but sometimes after reboot device no longer works.
-When this happens, controller fails to properly disable/enable during probe,
-so the
-.Xr ld 4
-is not attached.
-Virtual machine must be completely powered off and started
-again to get the device working again.
+5.1.6
+and the driver, emulated
+.Nm
+controller responds to commands only first time it's attached, after reboot or
+module reload stops responding. Virtual machine must be completely powered off
+(or even killed) to fix.
 .Pp
 .Nm
 kernel module is currently only loadable for kernels configured with