Chuck Silvers wrote:

> btw, a brief look at mdconfig shows that it uses exactly the same cute hack
> to provide backing memory for its devices, so it should behave identically
> to mfs with respect to memory usage.  it would be fairly easy to change md
> to have the kernel allocate swap-backed memory directly, which would save
> the context switch currently required to access data in the user-process
> backing-store, but that wouldn't help with the double-memory-usage issue.

mdconfig unconditionally passes MD_UMEM_SERVER as the type to the
driver.  md(4) itself also accepts MD_KMEM_ALLOCATED, where it then uses
uvm_km_zalloc() to allocated wired kernel memory instead of using the
mdconfig process's address space.  Interestingly, the kmem md appears to
be slower than the umem md, at least on a 600MHz mips board when using a
64MB md with "bonnie -s 50":

           -------Sequential Output-------- ---Sequential Input--- ---Random--
           -Per Char- --Block--- -Rewrite-- -Per Char- ---Block--- ---Seeks---
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
umem    50 20610 83.8 68319 60.8 47663 69.9 16835 100. 128537 100. 6430.3 93.6  
umem    50 21651 87.5 68335 60.7 48917 71.4 16843 100. 127982 100. 6689.0 97.2  
umem    50 21640 87.4 68619 60.7 48995 71.4 16822 100. 128799 100. 6673.2 96.8  

kmem    50 21254 97.3 67388 94.7 31560 92.8 14532 97.6  58939 90.2 4081.4 92.4
kmem    50 21540 98.2 67810 94.6 35945 93.1 14557 97.6  58888 90.3 4060.2 92.4
kmem    50 21545 98.2 66975 94.6 35662 93.1 14510 97.3  58988 90.2 4013.8 92.4

Note that block input is half the speed and rewrite is noticable slower.

For the umem test, there were  7699,  7699 and  7647 context switches.
For the kmem test, there were 11496, 11293 and 11380 context switches.

This numbers seem at odds with the "there shouldn't be context switches
with a kmem md".  Disabling the "random seeks" part of bonnie didn't
affect the context switch numbers too noticable (both were less, but
umem was still well less than kmem).  As a separate test, a dd with 1024
64k blocks got the following for umem:

67108864 bytes in 0.460 secs (145888834 bytes/sec), 2162 context switches
67108864 bytes in 0.539 secs (124506241 bytes/sec), 2086 context switches
67108864 bytes in 0.537 secs (124969951 bytes/sec), 2087 context switches

and kmem:

67108864 bytes in 0.517 secs (129804379 bytes/sec), 56 context switches
67108864 bytes in 0.518 secs (129553791 bytes/sec), 45 context switches
67108864 bytes in 0.517 secs (129804379 bytes/sec), 43 context switches

These figures seem to line up better with "2 context switches per IO"
with the umem md.  Also interesting is that the first dd after an
mdconfig always gets ~145MB/sec, but following ones gets ~125MB/sec.
This is repeatable.

I'm not sure what any of the above proves, but the results seem
interesting...

Simon.
--
Simon Burge                            <simonb@wasabisystems.com>
NetBSD Support and Service:         http://www.wasabisystems.com/