PV domU boot hung when using an lvm logical volume as a disk

[Toby Karyadi <toby.karyadi%gmail.com@localhost>]

Hi,

I've got a netbsd 6.99.4 dom0 on top of Xen 4.1.2. I also had a PV domU 
that was also running netbsd of the same build as the dom0 and had a 
disk referring to a raw LVM logical volume (LV). It would hang during 
boot (see DETAILS below). The boot up process hung just after setup of 
the balloon driver (see DETAILS below). When I removed the raw LV disk 
definition from the domU configuration file, the domU booted up fine, 
that is using a wedge (/dev/rdk*) as a disk for the domU worked properly.

When I excluded the raw LV from the domU config so that it could boot up 
and did 'xm block-attach <domname> phy:/dev/vg00/rlv1 hdc w', the disk 
will appear in the domU as xbd2. I noticed however that the dom0 console 
reported something like below:

  xbdback backend/vbd/2/5632: unknown device 0x7665642f

With vnd or dk based disk, it would actually refer to the name of the 
block device, e.g. vnd0d, or dk1, instead of 'unknown device'.

DomU console, on the other hand, reported:
  xbd2 at xenbus0 id 5632: Xen Virtual Block Device Interface

However, if  I did a 'disklabel xbd2' (in the domU, obviously), I'll get 
an error from disklabel:

  disklabel: ioctl DIOCGDINFO: Device not configured

I'm guessing, the hang occurred because the boot process couldn't get 
disk geometry info when encountering the xbd that was backed by a raw 
LV. There is this ML thread that might be relevant: 
http://mail-index.netbsd.org/tech-userlevel/2009/05/29/msg002132.html, 
but not sure if there is or should have any relation to xbdback.

Does anyone else encountered this problem? I thought this is a common 
use case with raw LVM LVs + PV domUs. Should I create a PR?

Cheers,
Toby


DETAILS
-------

My disk stack is as follows:
wd0f + wd1f + wd2f -> raid0 -> GPT -> dk0 -> vg00 -> lv01
                                   -> dk1

The domU config:

name = "dadomu01"
kernel = "/mnt/v01/vhosts/dadomu01/netbsd-XEN3_DOMU.gz"
memory = 64
vcpus = 1
disk = [ 'file:/mnt/v01/vhosts/dadomu01/dadomu01.disk0.img,hda,w',
         'phy:/dev/rdk1,hdb,w',
         'phy:/dev/vg00/rlv1,hdc,w',]
vif = [ 'mac=00:16:3e:55:cd:aa,bridge=bridge0' ]
on_reboot = 'restart'
on_crash = 'destroy'


The domU console with the above config:

dahost# xm create -c dadomu01
Using config file "/usr/pkg/etc/xen/dadomu01".
Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
    2006, 2007, 2008, 2009, 2010, 2011, 2012
    The NetBSD Foundation, Inc.  All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
    The Regents of the University of California.  All rights reserved.

NetBSD 6.99.4 (XEN3_DOMU) #1: Thu Mar 22 11:01:17 EDT 2012
        
root@dahost:/mnt/v01/build/src/2012.03.16.23.20.00/obj/sys/arch/amd64/compile/XEN3_DOMU
total memory = 65536 KB
avail memory = 54380 KB
mainbus0 (root)
hypervisor0 at mainbus0: Xen version 4.1
vcpu0 at hypervisor0: AMD Athlon(tm) 64 X2 Dual Core Processor 3600+, id 
0x60fb1
xenbus0 at hypervisor0: Xen Virtual Bus Interface
xencons0 at hypervisor0: Xen Virtual Console Driver
xbd0 at xenbus0 id 768: Xen Virtual Block Device Interface
xbd1 at xenbus0 id 832: Xen Virtual Block Device Interface
xbd2 at xenbus0 id 5632: Xen Virtual Block Device Interface
xennet0 at xenbus0 id 0: Xen Virtual Network Interface
xennet0: MAC address 00:16:3e:55:cd:aa
balloon0 at xenbus0 id 0: Xen Balloon driver
balloon0: current reservation: 65536 KiB
xennet0: using RX copy mode
balloon0: current reservation: 16384 pages => target: 16384 pages

... then it just sat there.