Re: Problems booting NetBSD/Xen on Intel NUC 12 Extreme w/ i9-12900

["Mathew\, Cherry G.*" <c%bow.st@localhost>]

>>>>> "Brad" == Brad Spencer <brad%anduin.eldar.org@localhost> writes:

    Brad> Johan Stenstam <johani%johani.org@localhost> writes:
    >> Hi Manuel,
    >> 
    >>> On 15 May 2023, at 21:09, Manuel Bouyer <bouyer%antioche.eu.org@localhost> wrote:
    >>> 
    >>> On Mon, May 15, 2023 at 04:53:55PM +0200, Johan Stenstam wrote:
    >>>> 
    >>>> So, again, I was quite clearly wrong and this is, indeed, some
    >>>> sort of problem with my DOMU. Or at least I cannot find any
    >>>> other explanation. So I created a trivial Linux PV DOMU and
    >>>> replaced the pvgrub boot method with direct kernel booting like
    >>>> I use for NetBSD. Here are the xen configs for my NetBSD 9.2
    >>>> DOMU (not working) and the Linux DOMU (working fine):
    >>> 
    >>> Sorry I didn't have time to look closely at this, but there have
    >>> been pullups between 9.2 and 9.3 specifically to fix a crash
    >>> with Xen 4.15 So you should try a 9.3 kernel (or even daily
    >>> netbsd-9)
    >> 
    >> No worries. This has been a group effort with help from many
    >> different people. As I said in an earlier message tonight I now
    >> have running DOMUs and now I’m in the process of figuring out how
    >> much of my infrastructure I need to upgrade to get everything
    >> work again. At a minimum I need to upgrade to get around that
    >> need for the msg_relaxed=1 setting, which is a saving grace today
    >> (thanks Brian!)  but obviously not a long term solution. That
    >> will likely give me the pullups you mention.
    >> 
    >> The only remaining show stopper issue at the moment seems to be
    >> that I have yet to be able to bridge Internet connectivity into a
    >> DOMU from my 10_BETA DOM0 (yes, the ure0 that I use for access
    >> and the xvifNiM are connected to the same bridge, which is up,
    >> and, yes according to sysctl both v4 and v6 forwarding is
    >> active). And, yes, bridges work fine between DOMUs, and between
    >> DOMUs and DOM0, but for some reason not yet between DOMU xvif
    >> interfaces and physical ethernet. But this is less of a concern
    >> than the Xen related issues you and others helped me with.

    Brad> I am not suggesting that this is the problem you are having,
    Brad> as I am not 100% sure what sort of thing your ure0 is, but I
    Brad> have had trouble in the past when the external interface of
    Brad> the DOM0 is a 802.11 NIC.  In that case, I ended up having to
    Brad> use NAT at the DOM0 and redo the networking on the DOMUs.  The
    Brad> main issue in that case is likely this:

    Brad> https://wiki.xenproject.org/wiki/Xen_in_WiFi_networks

    Brad> NetBSD doesn't have the solution offered in that page,
    Brad> however.


I think this is not Xen specific - when you bridge multiple VMs to the
same physical NIC, you need a sort of L2 NAT (I believe on Linux this is
called macvlan) - the ipfilter rules (which my brain froze over) seems
to be intended for that purpose, afaict (please correct me if I'm
wrong).

For NetBSD, this means, we need either a plugin for npf(7) that is able
to do this, or some other abstraction, say over vether(4). I'd be
interested to hear what people think is the right solution for this.

Best,
-- 
~cherry