Xen boot failures with disks >16K (tap device issues)

[David Hopper <dhop%nwlink.com@localhost>]

Hi all, I have a very odd condition with this NetBSD-current dom0 / Xen 
3.3.2nb1 in trying to start up Red Hat 6 domU instances.  I may need help 
understanding if I'm missing some virtual block devices in the domU based on 
the symptoms below, or if this could be a dom0 or script issue-- but the tap 
device will not connect.

I cannot start a VM when attaching raw images larger than 16K, and I cannot 
boot any method other than file: (any size qcow fails to boot as well).  It 
took some trial-and-error to find the 16K threshold-- needless to say the 
working images aren't very useful to me.

The disk definition in vm.install:  'file:/vnod/vm/vm.img,xvda,w'  (also tried 
tap:qcow).

# qemu-img create -f raw ./vm.img 16K  <== 16 kilobyte image works
Formatting './vm.img', fmt=raw size=16384 
# xm create -c vm.install
[ boots normally ]

---
# qemu-img create -f raw ./vm.img 17K  <== 17 kilobyte image fails
Formatting './vm.img', fmt=raw size=17408 
# xm create -c vm.install
Using config file "/usr/pkg/etc/xen/vm.install".
# Error: Device 51712 not connected
[hangs]
Error: Domain 'vm' does not exist.

---
(and using tap:qcow as disk access method):

# qemu-img create -f qcow ./vm.img 16K  <== 16 kilobyte image fails
Formatting './vm.img', fmt=qcow size=16384 encryption=off
# xm create -c vm.install
Using config file "/usr/pkg/etc/xen/vm.install".
# Error: Device 51712 (tap) could not be connected. Hotplug scripts not working.

I get the following in the logs (using sh -ex as interpreter):

exec /usr/pkg/etc/xen/scripts/block /local/domain/0/backend/vbd/41/51712 6
exec /usr/pkg/etc/xen/scripts/vif-bridge /local/domain/0/backend/vif/41/0 6
+ dirname /usr/pkg/etc/xen/scripts/vif-bridge
+ dir=/usr/pkg/etc/xen/scripts
+ . /usr/pkg/etc/xen/scripts/xen-subr
+ PATH=/bin:/usr/bin:/usr/pkg/bin:/sbin:/usr/sbin:/usr/pkg/sbin
+ export PATH
+ xpath=/local/domain/0/backend/vif/41/0
+ xstatus=6
+ xenstore-rm /local/domain/0/backend/vif/41/0
+ exit 0
Failed to read /local/domain/0/backend/tap/42/51712/script (No such file or 
directory)

Using bash as interpreter:

[2011-03-20 01:26:00 269] DEBUG (XendDomainInfo:1450) 
XendDomainInfo.handleShutdownWatch
[2011-03-20 01:26:00 269] DEBUG (DevController:166) Waiting for devices vif.
[2011-03-20 01:26:00 269] DEBUG (DevController:171) Waiting for 0.
[2011-03-20 01:26:00 269] DEBUG (DevController:656) hotplugStatusCallback 
/local/domain/0/backend/vif/44/0/hotplug-status.
[2011-03-20 01:26:00 269] DEBUG (DevController:656) hotplugStatusCallback 
/local/domain/0/backend/vif/44/0/hotplug-status.
[2011-03-20 01:26:00 269] DEBUG (DevController:670) hotplugStatusCallback 1.
[2011-03-20 01:26:01 269] DEBUG (DevController:166) Waiting for devices vkbd.
[2011-03-20 01:26:01 269] DEBUG (DevController:166) Waiting for devices ioports.
[2011-03-20 01:26:01 269] DEBUG (DevController:166) Waiting for devices tap.
[2011-03-20 01:26:01 269] DEBUG (DevController:171) Waiting for 51712.
[2011-03-20 01:26:01 269] DEBUG (DevController:656) hotplugStatusCallback 
/local/domain/0/backend/tap/44/51712/hotplug-status.
[2011-03-20 01:27:41 269] DEBUG (XendDomainInfo:2409) XendDomainInfo.destroy: 
domid=44
[2011-03-20 01:27:41 269] DEBUG (XendDomainInfo:1946) Destroying device model
[2011-03-20 01:27:41 269] DEBUG (XendDomainInfo:1953) Releasing devices
[2011-03-20 01:27:41 269] DEBUG (XendDomainInfo:1966) Removing vif/0
[2011-03-20 01:27:41 269] DEBUG (XendDomainInfo:910) 
XendDomainInfo.destroyDevice: deviceClass = vif, device = vif/0
[2011-03-20 01:27:41 269] DEBUG (XendDomainInfo:1966) Removing console/0
[2011-03-20 01:27:41 269] DEBUG (XendDomainInfo:910) 
XendDomainInfo.destroyDevice: deviceClass = console, device = console/0
[2011-03-20 01:27:41 269] DEBUG (XendDomainInfo:1966) Removing vbd/51712
[2011-03-20 01:27:41 269] DEBUG (XendDomainInfo:910) 
XendDomainInfo.destroyDevice: deviceClass = tap, device = vbd/51712
[2011-03-20 01:27:41 269] ERROR (XendDomainInfo:1973) Device release failed: 
vm; tap; vbd/51712
Traceback (most recent call last):
  File "/usr/pkg/lib/python2.6/site-packages/xen/xend/XendDomainInfo.py", line 
1967, in _releaseDevices
    self.destroyDevice(true_devclass, dev, False);
  File "/usr/pkg/lib/python2.6/site-packages/xen/xend/XendDomainInfo.py", line 
924, in destroyDevice
    rc = self.getDeviceController(deviceClass).destroyDevice(devid, force)
  File "/usr/pkg/lib/python2.6/site-packages/xen/xend/server/blkif.py", line 
187, in destroyDevice
    DevController.destroyDevice(self, devid, force)
  File "/usr/pkg/lib/python2.6/site-packages/xen/xend/server/DevController.py", 
line 255, in destroyDevice
    self.writeBackend(dev, 'online', "0")
  File "/usr/pkg/lib/python2.6/site-packages/xen/xend/server/DevController.py", 
line 501, in writeBackend
    raise VmError("Device %s not connected" % devid)
VmError: Device 51712 not connected

Thank you for any assistance or insights,

David