Re: Instability issues with NetBSD-9, xen-4.11 and the xbdb backend driver

[Brian Buhrow <buhrow%nfbcal.org@localhost>]

	hello.  In pursuit of a solution to this issue, I tried installing the
xenkernel48 and xentools48 packages to see if the problem exists there.
The problem is less pronounced, but still exists.  Again, here's what
happens:

On the dom0 side we get errors like:

[   354.670121] xbd backend domain 1 handle 0x1 (1) using event channel 20, protocol x86_64-abi
[  2705.480121] xbdb1i1: unknown operation 21
[  2705.480121] xbd IO domain 1: error -1
[  2706.690055] xbd backend: detach device xendisks/viadev for domain 1
[  2706.750055] xvif1i0: disconnecting

	On the domu side, we get:
panic: biodone2 already

	Sometimes, we get the domu panic without any error messages from dom0,
but since these same domu's are very stable under xen-3.3 and FreeBSD-12
running as the dom0, I don't think the domu is the problem.  

	Given the error messages we're seeing from both the dom0 and domu
sides, I think there are two things going on, one a side effect of the
other.  

1.  Somehow, the requests coming from the domu are getting corrupted by the
time they make it to the dom0 backend driver.

2.  the dom0 driver returns a fail on the corrupted requests and moves on
to handle future requests.  The domu, however, doesn't know how to handle
the failed responses at the stage of the game where this is happening.  

	What ever the problem is, it exists for both Xen-4.11 and xen-4.8.

	As an additional data point, I'm going to try spinning this setup on
a different piece of hardware to rule out the possibility that I've just
got bad metal.  This will take me a few days to complete, so if anyone has
any other ideas, please send them along and I'll look at them when I get
back to this project.

-thanks
-Brian