Port-xen archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
Re: Poor disk performance in NetBSD DomU, again
On Tue, May 13, 2014 at 9:29 PM, Michael L. Hitch
<mhitch%lightning.msu.montana.edu@localhost> wrote:
> On Sun, 11 May 2014, Yi Qiao wrote:
>
>> So at this point I'm lost in terms of how to remedy the DomU write
>> speed loss, since there is little to tinker with in the config file.
>> I'm not ready to give NetBSD up just yet. I stopped running Linux
>> since kernel 2.6.32, and every time I'm forced to run it, I feel dirty
>> and obese. Any help would be greatly appreciated ;)
>
>
> You might try running 'iostat -d -x wd0 5' on the DOM0 (and the equivalent
> on the DOMU) and compare the reads vs writes. I experienced a similar
> problem when I tried using a disk image file on the DOM0, and I saw lots of
> reads on the disk as well as writes, while iostat on the DOMU was showing
> just writes. I presumed that the DOM0 was doing read-modify-write
> operations which are quite slow. If I remember correctly, using a disk
> image was done with vnd(4), so I could also run iostat on the vnd device. I
> think you can also pass an LVM device to iostat, so you can see what it
> shows for reads and writes.
>
> I ended up using partitions on my DOM0 disk and was able to get reasonable
> write rates. I would think that lvm should be similar, but it's been a
> while since I've done anything with lvm. I think I was contemplating using
> lvm on the system where I ended up using disk partitions, but probably ran
> out of time to experiment.
Hi Mike
Thanks for the advise. I tried various tests (Dom0, lvm backed DomU,
physical partition backed DomU), and the results are attached at the
end of the message. In conclusion:
* There was no read activity during a write operation in DomU.
* physical partition (wd0e) backed DomU suffers from slow write too.
* dd to a block lvm device is slower than dd to a raw lvm device
* In DomU, whenever the write MB/s is low, the w/s is also low
Dom0 Tests
==========
Dom0, write to wd0a
-------------------
CMD: dd if=/dev/zero of=testfile bs=32k count=131072
device read KB/t r/s time MB/s write KB/t w/s time MB/s
wd0 0.00 0 1.00 0.00 63.88 1884 1.00 117.53
wd0 0.00 0 1.00 0.00 63.99 1969 1.00 123.05
Dom0, read from wd0a
--------------------
CMD: dd if=testfile of=/dev/null bs=32k
device read KB/t r/s time MB/s write KB/t w/s time MB/s
wd0 63.98 1888 0.99 117.94 0.00 0 0.99 0.00
wd0 63.98 1911 0.98 119.39 42.38 1 0.98 0.03
Dom0, write to lvm volume (newfs-ed, mounted with log)
------------------------------------------------------
CMD: dd if=/dev/zero of=/mnt/testfile bs=32k count=131072
device read KB/t r/s time MB/s write KB/t w/s time MB/s
vg0-test 16.00 2 0.03 0.02 63.82 1783 0.03 111.16
vg0-test 16.00 2 0.04 0.03 64.00 1908 0.04 119.26
Dom0, read from lvm volume
--------------------------
CMD: dd if=/mnt/testfile of=/dev/null bs=32k
device read KB/t r/s time MB/s write KB/t w/s time MB/s
vg0-test 64.00 1965 0.01 122.80 0.00 0 0.01 0.00
vg0-test 64.00 1988 0.00 124.28 0.00 0 0.00 0.00
Dom0, write to lvm volume directly (raw device)
-----------------------------------------------
CMD: dd if=/dev/zero of=/dev/vg0/rtestlv bs=32k count=131072
device read KB/t r/s time MB/s write KB/t w/s time MB/s
vg0-test 0.00 0 0.02 0.00 32.00 3608 0.02 112.75
vg0-test 0.00 0 0.01 0.00 32.00 3671 0.01 114.73
Dom0, write to lvm volume directly (block device)
-------------------------------------------------
CMD: dd if=/dev/zero of=/dev/vg0/testlv bs=32k count=131072
device read KB/t r/s time MB/s write KB/t w/s time MB/s
vg0-test 0.00 0 0.95 0.00 2.00 20142 0.95 39.34
vg0-test 0.00 0 0.97 0.00 2.00 16881 0.97 32.97
DomU Tests
==========
DomU, write to root (backed by lvm)
-----------------------------------
CMD: dd if=/dev/zero of=testfile bs=32k count=131072
### iostat in Dom0, checking the lvm volume
device read KB/t r/s time MB/s write KB/t w/s time MB/s
vg0-netb 0.00 0 0.00 0.00 32.01 455 0.00 14.22
vg0-netb 0.00 0 0.00 0.00 31.92 292 0.00 9.10
vg0-netb 14.00 1 0.01 0.01 31.99 459 0.01 14.36
vg0-netb 0.00 0 0.00 0.00 32.13 351 0.00 11.03
### iostat in DomU, checking xbd0
device read KB/t r/s time MB/s write KB/t w/s time MB/s
xbd0 16.00 0 1.00 0.00 31.96 356 1.00 11.11
xbd0 16.00 0 1.00 0.00 31.95 187 1.00 5.85
xbd0 16.00 0 1.00 0.00 31.92 302 1.00 9.42
xbd0 0.00 0 1.00 0.00 31.97 326 1.00 10.19
DomU, read from root (backed by lvm)
------------------------------------
CMD: dd if=testfile of=/dev/null bs=32k
### iostat in Dom0, checking the lvm volume
device read KB/t r/s time MB/s write KB/t w/s time MB/s
vg0-netb 62.45 1947 0.01 118.75 7.43 4 0.01 0.03
vg0-netb 62.48 1930 0.01 117.78 12.67 5 0.01 0.06
### iostat in DomU, checking xbd0
device read KB/t r/s time MB/s write KB/t w/s time MB/s
xbd0 31.99 3800 1.00 118.72 7.76 3 1.00 0.03
xbd0 32.00 3772 1.00 117.87 11.85 5 1.00 0.06
DomU, write to root (backed by physical partition wd0e)
-------------------------------------------------------
CMD: dd if=/dev/zero of=testfile bs=32k count=131072
### iostat in Dom0, checking wd0
device read KB/t r/s time MB/s write KB/t w/s time MB/s
wd0 16.00 0 0.94 0.01 32.01 327 0.94 10.23
wd0 16.00 0 1.00 0.01 31.98 532 1.00 16.62
wd0 16.00 0 0.93 0.01 32.19 407 0.93 12.78
### iostat in DomU, checking xbd0
device read KB/t r/s time MB/s write KB/t w/s time MB/s
xbd0 16.00 0 0.94 0.01 31.81 433 0.94 13.45
xbd0 16.00 1 1.00 0.01 31.99 448 1.00 13.99
xbd0 16.00 0 0.93 0.01 31.80 405 0.93 12.58
DomU, read from root (backed by physical partition wd0e)
-------------------------------------------------------
CMD: dd if=testfile of=/dev/null bs=32k
### iostat in Dom0, checking wd0
device read KB/t r/s time MB/s write KB/t w/s time MB/s
wd0 62.28 1916 0.98 116.52 13.75 2 0.98 0.03
wd0 62.12 1985 1.00 120.45 0.00 0 1.00 0.00
### iostat in DomU, checking xbd0
device read KB/t r/s time MB/s write KB/t w/s time MB/s
xbd0 31.98 3864 1.00 120.66 0.00 0 1.00 0.00
xbd0 31.98 3848 0.99 120.15 20.50 0 0.99 0.00
>
> Mike
>
> ---
> Michael L. Hitch mhitch%montana.edu@localhost
> Operations Consulting, Information Technology Center
> Montana State University, Bozeman, MT USA
Home |
Main Index |
Thread Index |
Old Index