netbsd domU does not migrate properly from one xen host to another (continued)

[Pierre-Philipp Braun <pbraun%nethence.com@localhost>]

Hello,

It was somehow a known issue, but two years ago I filled-in a bug report two years ago about netbsd domUs not being able to migrate.  This was PV mode.

Subject: netbsd domU does not migrate properly from one xen host to another
https://gnats.netbsd.org/55207

I got a new full-blown XEN cluster up and running and ready to roll.  It's powered by DRBD v9, Slackware Linux and XEN v4.16.1.  I was able to start a NetBSD guest (current GENERIC kernel and 9.2 userland) in PVH mode.  So I gave it another shot: same thing happens in PVH.

trying to move guest `dnc3` from node `pmr1` to `pmr2` gives

# xl migrate dnc3 pmr2
migration target: Ready to receive domain.
Saving to migration stream new xl format (info 0x3/0x0/1434)
Loading new save file <incoming migration stream> (new xl fmt info 0x3/0x0/1434)
 Savefile contains xl domain config in JSON format
Parsing config from <saved>
xc: info: Saving domain 36, type x86 HVM
xc: info: Found x86 HVM domain from Xen 4.16
xc: info: Restoring domain
libxl: error: libxl_dom_suspend.c:372:suspend_common_wait_guest_timeout: Domain 36:guest did not suspend, timed out
xc: error: save callback suspend() failed: 0: Internal error
xc: error: Save failed (0 = Success): Internal error
libxl: error: libxl_stream_write.c:347:libxl__xc_domain_save_done: Domain 36:saving domain: domain responded to suspend request: Success
migration sender: libxl_domain_suspend failed (rc=-8)
xc: error: Failed to read Record Header from stream (0 = Success): Internal error
xc: error: Restore failed (0 = Success): Internal error
libxl: error: libxl_stream_read.c:850:libxl__xc_domain_restore_done: restoring domain: Success
libxl: error: libxl_create.c:1611:domcreate_rebuild_done: Domain 4:cannot (re-)build domain: -3
libxl: error: libxl_domain.c:1183:libxl__destroy_domid: Domain 4:Non-existant domain
libxl: error: libxl_domain.c:1137:domain_destroy_callback: Domain 4:Unable to destroy guest
libxl: error: libxl_domain.c:1064:domain_destroy_cb: Domain 4:Destruction of domain failed
migration target: Domain creation failed (code -3).
libxl: info: libxl_exec.c:117:libxl_report_child_exitstatus: migration transport process [21831] exited with error status 1
Migration failed, failed to suspend at sender.

I really wish I could offer netbsd guests in my little cloud.  Without those being able to migrate, I won't able to do that.  I would really appreciate some insights:

- is the issue becoming easier to solve with PVH?
- can I assume this issue going to be solved some day?
- is there any workarounds in mind?  disable something from the guest config?

Thanks & Best Regards
--
Pierre-Philipp Braun
SMTP Health Campaign: enforce STARTTLS and verify MX certificates
<https://nethence.com/smtp/>