Current-Users archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: current dom0 panic on domu launch

On 20/10/2009 12:08 PM, Sarton O'Brien wrote:

On 16/10/2009 10:34 AM, Sarton O'Brien wrote:

On 15/10/2009 5:54 AM, Manuel Bouyer wrote:

On Mon, Oct 05, 2009 at 10:43:42PM +1100, Sarton O'Brien wrote:

# uvm_fault(0xffffffff80c2da00, 0x0, 1) -> e
fatal page fault in supervisor mode
trap type 6 code 0 rip ffffffff805651f3 cs e030 rflags 10246 cr2 0 cpl
0 rsp ff
ffa00013ac3ad0
kernel: page fault trap, code=0
Stopped in pid 0.43 (system) at netbsd:sdioctl+0x506: movl 0(%rcx),%r8d
sdioctl() at netbsd:sdioctl+0x506
VOP_IOCTL() at netbsd:VOP_IOCTL+0x31
xbdback_do_io() at netbsd:xbdback_do_io+0xd2
workqueue_worker() at netbsd:workqueue_worker+0x81


Could you see if xbdback_xenbus.c 1.26 fixes your issue ?


Sorry for the delay. This being the latest email I'll give this a shot
asap and report back. Thanks both of you for giving it some attention,
greatly appreciated.


Seems to working and stable again (updated Mon Oct 19 21:00:37 EST
2009), however there seems to be a problem I've had before (regression?).

When shutting down a domU, xm list shows that it has been removed but
the vnd is still tied up. Issuing 'vnconfig -u vnd2' says device busy.
Issuing 'vnconfig -uF vnd2' succeeds but then I'm still unable to
relaunch the domU, an error mentions unable to detach xvif but not much
else.

If there is further information I can provide please let me know. In the
meantime I'm updating xentools, python and expat to see if that helps.


This hasn't helped. The logs report as follows:

## Shutdown running domU ##
[2009-10-20 12:34:42 318] INFO (XendDomainInfo:1641) Domain has shutdown: name=babylon id=4 reason=poweroff. [2009-10-20 12:34:42 318] DEBUG (XendDomainInfo:2409) XendDomainInfo.destroy: domid=4 [2009-10-20 12:34:42 318] DEBUG (XendDomainInfo:1946) Destroying device model 
[2009-10-20 12:34:42 318] DEBUG (XendDomainInfo:1953) Releasing devices
[2009-10-20 12:34:42 318] DEBUG (XendDomainInfo:1966) Removing vif/0
[2009-10-20 12:34:42 318] DEBUG (XendDomainInfo:910) XendDomainInfo.destroyDevice: deviceClass = vif, device = vif/0 
[2009-10-20 12:34:42 318] DEBUG (XendDomainInfo:1966) Removing vbd/0
[2009-10-20 12:34:42 318] DEBUG (XendDomainInfo:910) XendDomainInfo.destroyDevice: deviceClass = vbd, device = vbd/0 
[2009-10-20 12:34:42 318] DEBUG (XendDomainInfo:1966) Removing console/0
[2009-10-20 12:34:42 318] DEBUG (XendDomainInfo:910) XendDomainInfo.destroyDevice: deviceClass = console, device = console/0 Failed to read /local/domain/0/backend/console/4/0/script (No such file or directory) 

## Recreate domU ##
xm create babylon
Using config file "/usr/pkg/etc/xen/babylon".
[2009-10-20 12:35:44 318] DEBUG (XendDomainInfo:89) XendDomainInfo.create(['vm', ['name', 'babylon'], ['memory', 256], ['vcpus', 1], ['on_xend_start', 'ignore'], ['on_xend_stop', 'ignore'], ['image', ['linux', ['kernel', '/domu-netbsd-amd64']]], ['device', ['vbd', ['uname', 'file:/usr/pkg/etc/xen/babylon-vnd.img'], ['dev', '0'], ['mode', 'w']]], ['device', ['vif', ['bridge', 'bridge0'], ['mac', '00:16:3e:61:0b:e3']]]]) [2009-10-20 12:35:44 318] DEBUG (XendDomainInfo:2043) XendDomainInfo.constructDomain [2009-10-20 12:35:44 318] DEBUG (balloon:132) Balloon: 1770684 KiB free; need 2048; done. 
[2009-10-20 12:35:44 318] DEBUG (XendDomain:449) Adding Domain: 5
[2009-10-20 12:35:44 318] DEBUG (XendDomainInfo:2149) XendDomainInfo.initDomain: 5 256 [2009-10-20 12:35:44 318] DEBUG (XendDomainInfo:2230) _initDomain:shadow_memory=0x0, memory_static_max=0x10000000, memory_static_min=0x0. [2009-10-20 12:35:44 318] DEBUG (balloon:132) Balloon: 1770676 KiB free; need 263168; done. 
xc_dom_allocate: cmdline="", features=""
[2009-10-20 12:35:44 318] INFO (image:166) buildDomain os=linux dom=5 vcpus=1 
[2009-10-20 12:35:44 318] DEBUG (image:629) domid          = 5
[2009-10-20 12:35:44 318] DEBUG (image:630) memsize        = 256
[2009-10-20 12:35:44 318] DEBUG (image:631) image = /domu-netbsd-amd64 
[2009-10-20 12:35:44 318] DEBUG (image:632) store_evtchn   = 1
[2009-10-20 12:35:44 318] DEBUG (image:633) console_evtchn = 2
[2009-10-20 12:35:44 318] DEBUG (image:634) cmdline        =
[2009-10-20 12:35:44 318] DEBUG (image:635) ramdisk        =
[2009-10-20 12:35:44 318] DEBUG (image:636) vcpus          = 1
[2009-10-20 12:35:44 318] DEBUG (image:637) features       =
xc_dom_kernel_file: filename="/domu-netbsd-amd64"
xc_dom_malloc_filemap    : 6321 kB
xc_dom_boot_xen_init: ver 3.3, caps xen-3.0-x86_64 xen-3.0-x86_32p
xc_dom_parse_image: called
xc_dom_find_loader: trying ELF-generic loader ... OK
elf_parse_binary: phdr: paddr=0xffffffff80000000 memsz=0x6481c0
elf_parse_binary: memory: 0xffffffff80000000 -> 0xffffffff806481c0
elf_xen_parse: __xen_guest: "GUEST_OS=NetBSD,GUEST_VER=4.99,XEN_VER=xen-3.0,LOADER=generic,VIRT_BASE=0xffffffff80000000,ELF_PADDR_OFFSET=0xffffffff80000000,VIRT_ENTRY=0xffffffff80100000,HYPERCALL_PAGE=0x00000101,BSD_SYMTAB=yes" 
elf_xen_parse_guest_info: GUEST_OS="NetBSD"
elf_xen_parse_guest_info: GUEST_VER="4.99"
elf_xen_parse_guest_info: XEN_VER="xen-3.0"
elf_xen_parse_guest_info: LOADER="generic"
elf_xen_parse_guest_info: VIRT_BASE="0xffffffff80000000"
elf_xen_parse_guest_info: ELF_PADDR_OFFSET="0xffffffff80000000"
elf_xen_parse_guest_info: VIRT_ENTRY="0xffffffff80100000"
elf_xen_parse_guest_info: HYPERCALL_PAGE="0x00000101"
elf_xen_parse_guest_info: BSD_SYMTAB="yes"
elf_xen_addr_calc_check: addresses:
    virt_base        = 0xffffffff80000000
    elf_paddr_offset = 0xffffffff80000000
    virt_offset      = 0x0
    virt_kstart      = 0xffffffff80000000
    virt_kend        = 0xffffffff806c9d68
    virt_entry       = 0xffffffff80100000
xc_dom_load_elf_symtab/parse: bsd_symtab_start=ffffffff806c9d68, kernel.end=0xffffffff806c9d68 -- symtab=0xffffffff806c9d6c, maxaddr=0xffffffff806ca330 xc_dom_load_elf_symtab: h=20 symtab, size=0x4e4c8, maxaddr=0xffffffff807187f8 xc_dom_load_elf_symtab: h=21 strtab, size=0x32ff3, maxaddr=0xffffffff8074b7f0 xc_dom_parse_elf_kernel: xen-3.0-x86_64: 0xffffffff80000000 -> 0xffffffff8074b7f0 
xc_dom_mem_init: mem 256 MB, pages 0x10000 pages, 4k each
xc_dom_mem_init: 0x10000 pages
xc_dom_boot_mem_init: called
x86_compat: guest xen-3.0-x86_64, address size 64
xc_dom_malloc            : 512 kB
xc_dom_build_image: called
xc_dom_alloc_segment: kernel : 0xffffffff80000000 -> 0xffffffff8074c000 (pfn 0x0 + 0x74c pages) 
xc_dom_pfn_to_ptr: domU mapping: pfn 0x0+0x74c at 0x7f7ff1e87000
elf_load_binary: phdr 0 at 0x0x7f7ff1e87000 -> 0x0x7f7ff241a840
elf_load_bsdsyms: shdr 19 at 0x0x7f7ff2b7db7d -> 0x0x7f7ff24cf788
elf_load_bsdsyms: shdr 20 at 0x0x7f7ff2b7e220 -> 0x0x7f7ff24cf8a8
elf_load_bsdsyms: shdr 21 at 0x0x7f7ff2bcc6e8 -> 0x0x7f7ff251dd70
xc_dom_load_elf_symtab/load: bsd_symtab_start=ffffffff806c9d68, kernel.end=0xffffffff8074c000 -- symtab=0xffffffff806c9d6c, maxaddr=0xffffffff806ca330 xc_dom_load_elf_symtab: h=20 symtab, size=0x4e4c8, maxaddr=0xffffffff807187f8 xc_dom_load_elf_symtab: h=21 strtab, size=0x32ff3, maxaddr=0xffffffff8074b7f0 xc_dom_alloc_segment: phys2mach : 0xffffffff8074c000 -> 0xffffffff807cc000 (pfn 0x74c + 0x80 pages) 
xc_dom_pfn_to_ptr: domU mapping: pfn 0x74c+0x80 at 0x7f7ffdf5e000
xc_dom_alloc_page   :   start info   : 0xffffffff807cc000 (pfn 0x7cc)
xc_dom_alloc_page   :   xenstore     : 0xffffffff807cd000 (pfn 0x7cd)
xc_dom_alloc_page   :   console      : 0xffffffff807ce000 (pfn 0x7ce)
nr_page_tables: 0x0000ffffffffffff/48: 0xffff000000000000 -> 0xffffffffffffffff, 1 table(s) nr_page_tables: 0x0000007fffffffff/39: 0xffffff8000000000 -> 0xffffffffffffffff, 1 table(s) nr_page_tables: 0x000000003fffffff/30: 0xffffffff80000000 -> 0xffffffffbfffffff, 1 table(s) nr_page_tables: 0x00000000001fffff/21: 0xffffffff80000000 -> 0xffffffff80bfffff, 6 table(s) xc_dom_alloc_segment: page tables : 0xffffffff807cf000 -> 0xffffffff807d8000 (pfn 0x7cf + 0x9 pages) 
xc_dom_pfn_to_ptr: domU mapping: pfn 0x7cf+0x9 at 0x7f7ffdf55000
xc_dom_alloc_page   :   boot stack   : 0xffffffff807d8000 (pfn 0x7d8)
xc_dom_build_image  : virt_alloc_end : 0xffffffff807d9000
xc_dom_build_image  : virt_pgtab_end : 0xffffffff80c00000
xc_dom_boot_image: called
arch_setup_bootearly: doing nothing
xc_dom_compat_check: supported guest type: xen-3.0-x86_64 <= matches
xc_dom_compat_check: supported guest type: xen-3.0-x86_32p
xc_dom_update_guest_p2m: dst 64bit, pages 0x10000
clear_page: pfn 0x7ce, mfn 0x6b25e
clear_page: pfn 0x7cd, mfn 0x6b25f
xc_dom_pfn_to_ptr: domU mapping: pfn 0x7cc+0x1 at 0x7f7ffdf54000
start_info_x86_64: called
setup_hypercall_page: vaddr=0xffffffff80101000 pfn=0x101
domain builder memory footprint
   allocated
      malloc             : 562 kB
      anon mmap          : 0 bytes
   mapped
      file mmap          : 6321 kB
      domU mmap          : 8024 kB
arch_setup_bootlate: shared_info: pfn 0x0, mfn 0xdf09c
shared_info_x86_64: called
vcpu_x86_64: called
vcpu_x86_64: cr3: pfn 0x7cf mfn 0x6b25d
launch_vm: called, ctxt=0x7f7ff978e018
xc_dom_release: called
[2009-10-20 12:35:44 318] INFO (XendDomainInfo:1907) createDevice: vbd : {'uuid': '674fac1e-6196-e479-6674-ff4035e17f94', 'bootable': 1, 'driver': 'paravirtualised', 'dev': '0', 'uname': 'file:/usr/pkg/etc/xen/babylon-vnd.img', 'mode': 'w'} [2009-10-20 12:35:44 318] DEBUG (DevController:122) DevController: writing {'virtual-device': '0', 'device-type': 'disk', 'protocol': 'x86_64-abi', 'backend-id': '0', 'state': '1', 'backend': '/local/domain/0/backend/vbd/5/0'} to /local/domain/5/device/vbd/0. [2009-10-20 12:35:44 318] DEBUG (DevController:124) DevController: writing {'domain': 'babylon', 'frontend': '/local/domain/5/device/vbd/0', 'uuid': '674fac1e-6196-e479-6674-ff4035e17f94', 'dev': '0', 'state': '1', 'params': '/usr/pkg/etc/xen/babylon-vnd.img', 'mode': 'w', 'online': '1', 'frontend-id': '5', 'type': 'file'} to /local/domain/0/backend/vbd/5/0. [2009-10-20 12:35:44 318] INFO (XendDomainInfo:1907) createDevice: vif : {'bridge': 'bridge0', 'mac': '00:16:3e:61:0b:e3', 'uuid': 'bf555011-cb87-2472-b121-709d31216870'} [2009-10-20 12:35:44 318] DEBUG (DevController:122) DevController: writing {'mac': '00:16:3e:61:0b:e3', 'handle': '0', 'protocol': 'x86_64-abi', 'backend-id': '0', 'state': '1', 'backend': '/local/domain/0/backend/vif/5/0'} to /local/domain/5/device/vif/0. [2009-10-20 12:35:44 318] DEBUG (DevController:124) DevController: writing {'bridge': 'bridge0', 'domain': 'babylon', 'handle': '0', 'uuid': 'bf555011-cb87-2472-b121-709d31216870', 'script': '/usr/pkg/etc/xen/scripts/vif-bridge', 'mac': '00:16:3e:61:0b:e3', 'frontend-id': '5', 'state': '1', 'online': '1', 'frontend': '/local/domain/5/device/vif/0'} to /local/domain/0/backend/vif/5/0. [2009-10-20 12:35:44 318] DEBUG (XendDomainInfo:2738) Storing VM details: {'on_xend_stop': 'ignore', 'shadow_memory': '0', 'uuid': '779ec372-7ab0-a857-4f45-edb11a931615', 'on_reboot': 'restart', 'start_time': '1256002544.72', 'on_poweroff': 'destroy', 'bootloader_args': '', 'on_xend_start': 'ignore', 'on_crash': 'restart', 'xend/restart_count': '0', 'vcpus': '1', 'vcpu_avail': '1', 'bootloader': '', 'image': '(linux (kernel /domu-netbsd-amd64) (notes))', 'name': 'babylon'} [2009-10-20 12:35:44 318] DEBUG (XendDomainInfo:1360) Storing domain details: {'console/ring-ref': '438878', 'console/port': '2', 'name': 'babylon', 'console/limit': '1048576', 'vm': '/vm/779ec372-7ab0-a857-4f45-edb11a931615', 'domid': '5', 'store/port': '1', 'cpu/0/availability': 'online', 'memory/target': '262144', 'control/platform-feature-multiprocessor-suspend': '1', 'store/ring-ref': '438879', 'console/type': 'xenconsoled'} [2009-10-20 12:35:44 318] DEBUG (DevController:122) DevController: writing {'protocol': 'x86_64-abi', 'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/console/5/0'} to /local/domain/5/device/console/0. [2009-10-20 12:35:44 318] DEBUG (DevController:124) DevController: writing {'domain': 'babylon', 'frontend': '/local/domain/5/device/console/0', 'uuid': 'e7397e36-1de1-7ca2-886e-ec18446e1d80', 'frontend-id': '5', 'state': '1', 'location': '2', 'online': '1', 'protocol': 'vt100'} to /local/domain/0/backend/console/5/0. Failed to read /local/domain/0/backend/console/5/0/script (No such file or directory) [2009-10-20 12:35:44 318] DEBUG (XendDomainInfo:1450) XendDomainInfo.handleShutdownWatch 
[2009-10-20 12:35:45 318] DEBUG (DevController:166) Waiting for devices vif.
[2009-10-20 12:35:45 318] DEBUG (DevController:171) Waiting for 0.
[2009-10-20 12:35:45 318] DEBUG (DevController:656) hotplugStatusCallback /local/domain/0/backend/vif/5/0/hotplug-status. 

## Long delay ... then ##
[2009-10-20 12:37:25 318] DEBUG (XendDomainInfo:2409) XendDomainInfo.destroy: domid=5 [2009-10-20 12:37:25 318] DEBUG (XendDomainInfo:1946) Destroying device model 
[2009-10-20 12:37:25 318] DEBUG (XendDomainInfo:1953) Releasing devices
[2009-10-20 12:37:25 318] DEBUG (XendDomainInfo:1966) Removing vif/0
[2009-10-20 12:37:25 318] DEBUG (XendDomainInfo:910) XendDomainInfo.destroyDevice: deviceClass = vif, device = vif/0 [2009-10-20 12:37:25 318] ERROR (XendDomainInfo:1973) Device release failed: babylon; vif; vif/0 
Traceback (most recent call last):
File "usr/pkg/lib/python2.5/site-packages/xen/xend/XendDomainInfo.py", line 1967, in _releaseDevices 
    self.destroyDevice(true_devclass, dev, False);
File "usr/pkg/lib/python2.5/site-packages/xen/xend/XendDomainInfo.py", line 924, in destroyDevice 
    rc = self.getDeviceController(deviceClass).destroyDevice(devid, force)
File "/usr/pkg/lib/python2.5/site-packages/xen/xend/server/DevController.py", line 255, in destroyDevice 
    self.writeBackend(dev, 'online', "0")
File "/usr/pkg/lib/python2.5/site-packages/xen/xend/server/DevController.py", line 501, in writeBackend 
    raise VmError("Device %s not connected" % devid)
VmError: Device 0 not connected
[2009-10-20 12:37:25 318] DEBUG (XendDomainInfo:1966) Removing vbd/0
[2009-10-20 12:37:25 318] DEBUG (XendDomainInfo:910) XendDomainInfo.destroyDevice: deviceClass = vbd, device = vbd/0 
[2009-10-20 12:37:25 318] DEBUG (XendDomainInfo:1966) Removing console/0
[2009-10-20 12:37:25 318] DEBUG (XendDomainInfo:910) XendDomainInfo.destroyDevice: deviceClass = console, device = console/0 
xenstore-read: couldn't read path /local/domain/0/backend/console/5/0/type
[2009-10-20 12:37:25 318] DEBUG (XendDomainInfo:1951) No device model
[2009-10-20 12:37:25 318] DEBUG (XendDomainInfo:1953) Releasing devices
Error: Device 0 (vif) could not be connected. Hotplug scripts not working.

Sarton
Home | Main Index | Thread Index | Old Index