port-arm/60655: port-arm/: vmt(4) probes the VMware backdoor unconditionally on aarch64 and panics

["Showta Ishizaki via gnats" <gnats-admin%NetBSD.org@localhost>]

>Number:         60655
>Category:       port-arm
>Synopsis:       vmt(4) probes the VMware backdoor unconditionally on aarch64 and panics the kernel on hypervisors that do not implement it
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    port-arm-maintainer
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Aug 27 02:00:00 +0000 2026
>Originator:     Showta Ishizaki
>Release:        NetBSD 11.99.7 (also NetBSD 11.0)
>Organization:
>Environment:
System: NetBSD 11.99.7 (GENERIC64) evbarm/aarch64, built from -current
Architecture: aarch64
Machine: evbarm
Host: qemu-system-aarch64 -M virt -accel hvf -cpu host, Apple M4, macOS
Also reproduced with the released NetBSD 11.0 evbarm-aarch64 arm64.img.

>Description:

sys/arch/aarch64/aarch64/vmt.c attaches at cpufeaturebus and its match
routine calls vmt_probe() with no guard:

	static int
	vmt_match(device_t parent, cfdata_t match, void *aux)
	{
		/* vmt should not attach to more than a single CPU. */
		if (vmt_attached)
			return 0;

		return vmt_probe();
	}

vmt_probe() reaches BACKDOOR_OP_AARCH64 in sys/dev/vmt/vmtvar.h, whose
operative instruction is

	mrs xzr, mdccsr_el0

On VMware this access is intercepted by the hypervisor and becomes the
backdoor call.  Elsewhere it is an ordinary read of a real system
register, and a hypervisor that neither implements nor emulates it
delivers an undefined instruction exception instead.  Apple's
Hypervisor.framework does exactly that, and since the access happens at
EL1, trap_el1h_sync() falls through to its default case and panics before
the root file system is mounted:

	[   1.0000000] cpu0 at acpi0: unknown CPU (ID = 0x610f0000), id 0x0
	[   1.0000000] cpu0: package 0, core 0, smt 0, numa 0
	[   1.0000000] panic: Trap: fatal Unknown Reason (Illegal Instruction):
	               pc=ffffc000003fa318 sp=ffffc000010262e0 esr=02000000
	[   1.0000000] fp ffffc00001026880 config_found_acquire() at ...
	[   1.0000000] fp ffffc00001026910 acpi_fdt_attach() at ...
	[   1.0000000] cpu0: End traceback...
	Stopped in pid 0.0 (system) at netbsd:cpu_Debugger+0xc

pc resolves to vm_cmd+0x28.

The same unconditional probe on x86 is harmless because the backdoor there
is an inl from an I/O port, and reading an unimplemented port returns
0xffffffff rather than faulting.  aarch64 offers no equivalent guarantee,
so the same shape of code is not safe here.

This is not specific to Apple's hypervisor.  Any aarch64 hypervisor that
does not implement the VMware backdoor will panic NetBSD at boot the same
way.

>How-To-Repeat:

On an Apple silicon host with qemu installed:

	ftp https://cdn.NetBSD.org/pub/NetBSD/NetBSD-11.0/evbarm-aarch64/binary/gzimg/arm64.img.gz
	gunzip arm64.img.gz
	qemu-img resize -f raw arm64.img 40G
	cp /opt/homebrew/share/qemu/edk2-aarch64-code.fd .
	qemu-img create -f raw varstore.img 64M

	qemu-system-aarch64 -M virt -accel hvf -cpu host -smp 4 -m 4G \
	  -drive if=pflash,format=raw,readonly=on,file=edk2-aarch64-code.fd \
	  -drive if=pflash,format=raw,file=varstore.img \
	  -drive if=none,file=arm64.img,format=raw,id=hd0 \
	  -device virtio-blk-pci,drive=hd0 -nographic

The kernel panics as above.  -M virt,acpi=off panics in the same place.

The same image with -accel tcg -cpu cortex-a72 boots to a login prompt.
That is the control: the image and qemu are fine, and the only difference
is the hypervisor.

>Fix:

Two pieces.  NetBSD already has the machinery for "poke something that may
not be there": cpu_set_onfault() and the faultbuf that bus_space_peek()
uses, and trap_el1h_error() already consults it for an SError.  It was
simply never wired up for an undefined instruction.

The first hunk does that wiring; the second has the probe use it.  Built
from -current and verified under -accel hvf on an Apple M4: the kernel now
walks past vmt and reaches a login prompt.

--- sys/arch/aarch64/aarch64/trap.c
+++ sys/arch/aarch64/aarch64/trap.c
@@ -245,6 +245,24 @@
 #endif
 		break;
 
+	case ESR_EC_UNKNOWN: {
+		/*
+		 * An undefined instruction in the kernel is normally fatal,
+		 * but a driver may be probing for an instruction that only
+		 * some machines implement, in the same way bus_space_peek()
+		 * probes for a device that may not be there.  Let such a
+		 * probe unwind through its faultbuf, as trap_el1h_error()
+		 * already does for an SError.
+		 */
+		struct faultbuf * const fb = cpu_disable_onfault();
+
+		if (fb != NULL) {
+			cpu_jump_onfault(tf, fb, EFAULT);
+			return;
+		}
+		goto fatal;
+	}
+
 	case ESR_EC_FP_ACCESS:
 		if ((curlwp->l_flag & (LW_SYSTEM|LW_SYSTEM_FPU)) ==
 		    (LW_SYSTEM|LW_SYSTEM_FPU)) {
@@ -258,6 +276,7 @@
 	case ESR_EC_ILL_STATE:
 	case ESR_EC_BTE_A64:
 	default:
+	fatal:
 		panic("Trap: fatal %s: pc=%016" PRIx64 " sp=%016" PRIx64
 		    " esr=%08x", eclass_trapname(eclass), tf->tf_pc, tf->tf_sp,
 		    esr);

--- sys/arch/aarch64/aarch64/vmt.c
+++ sys/arch/aarch64/aarch64/vmt.c
@@ -33,6 +33,8 @@
 #include <sys/device.h>
 #include <sys/module.h>
 
+#include <aarch64/machdep.h>
+
 #include <dev/vmt/vmtreg.h>
 #include <dev/vmt/vmtvar.h>
 
@@ -48,11 +50,25 @@
 static int
 vmt_match(device_t parent, cfdata_t match, void *aux)
 {
+	struct faultbuf fb;
+	int rv;
+
 	/* vmt should not attach to more than a single CPU. */
 	if (vmt_attached)
 		return 0;
 
-	return vmt_probe();
+	/*
+	 * The backdoor VMware listens on is an access to a real system
+	 * register.  A hypervisor that does not implement it delivers an
+	 * undefined instruction exception instead, so run the probe under a
+	 * faultbuf and read a fault as "not VMware".
+	 */
+	if (cpu_set_onfault(&fb) != 0)
+		return 0;
+	rv = vmt_probe();
+	cpu_unset_onfault();
+
+	return rv;
 }