Port-xen archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: Poor disk performance in NetBSD DomU, again

On Wed, 14 May 2014, Yi Qiao wrote:

 Thanks for the advise. I tried various tests (Dom0, lvm backed DomU,
physical partition backed DomU), and the results are attached at the
end of the message. In conclusion:

 * There was no read activity during a write operation in DomU.
 * physical partition (wd0e) backed DomU suffers from slow write too.

  This one is rather interesting.

 * dd to a block lvm device is slower than dd to a raw lvm device

This is to be expected. Writing to a block device copies the data into a kernel buffer, and then writes the buffer to the disk device. If the block device is not an FFS partition, the buffer size will be 2K. So each write of 64K bytes is broken up into 32 separate operations, which can be much slower.

 * In DomU, whenever the write MB/s is low, the w/s is also low

Dom0, write to lvm volume directly (block device)
CMD: dd if=/dev/zero of=/dev/vg0/testlv bs=32k count=131072
device  read KB/t    r/s   time     MB/s write KB/t    w/s   time     MB/s
vg0-test     0.00      0   0.95     0.00       2.00  20142   0.95    39.34
vg0-test     0.00      0   0.97     0.00       2.00  16881   0.97    32.97

Note that the KB/transfer is 2KB here, representing the buffer size used by block devices. Also, the time is quite close to 1, which I think indicates disk is nearly 100% busy. That could show that the writes are backing up because the disk can't process them fast enough. Hmm, if the disk has internal 4K sectors, writing 2K would require the disk to read a 4K block, update the 2K data, and write the 4K back.

DomU, write to root (backed by physical partition wd0e)
CMD: dd if=/dev/zero of=testfile bs=32k count=131072
### iostat in Dom0, checking wd0
device  read KB/t    r/s   time     MB/s write KB/t    w/s   time     MB/s
wd0         16.00      0   0.94     0.01      32.01    327   0.94    10.23
wd0         16.00      0   1.00     0.01      31.98    532   1.00    16.62
wd0         16.00      0   0.93     0.01      32.19    407   0.93    12.78

This is also showing the disk as nearly 100% busy. What is the starting sector of the wd0e partition? Does it start on a 4K boundary?

I'm still not sure how to explain why using the lvm volume is slow though.


Michael L. Hitch                        mhitch%montana.edu@localhost
Operations Consulting,  Information Technology Center
Montana State University, Bozeman, MT     USA

Home | Main Index | Thread Index | Old Index