kern/57878: i915 calls agp_i810_chipset_flush with spin lock held but agp_i810_chipset_flush waits for xcall

[campbell+netbsd%mumble.net@localhost]

>Number:         57878
>Category:       kern
>Synopsis:       i915 calls agp_i810_chipset_flush with spin lock held but agp_i810_chipset_flush waits for xcall
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Jan 25 23:15:00 +0000 2024
>Originator:     Taylor R Campbell
>Release:        current, 10
>Organization:
The NetAGP Foundation
>Environment:
>Description:
[     1.051264] agp0 at pchb0: i855-family chipset
[     1.051264] agp0: detected 32636k stolen memory
[     1.051264] agp0: aperture at 0xf0000000, size 0x8000000
[     1.051264] Intel 82855GM GMCH Memory Controller (miscellaneous system, revision 0x02) at pci0 dev 0 function 1 not configured
[     1.051264] Intel 82855GM GMCH Configuration Process (miscellaneous system, revision 0x02) at pci0 dev 0 function 3 not configured
[     1.051264] i915drmkms0 at pci0 dev 2 function 0: Intel 82855GM GMCH Integrated Graphics Device (rev. 0x02)
[     1.051264] Intel 82855GM GMCH Integrated Graphics Device (miscellaneous display, revision 0x02) at pci0 dev 2 function 1 not configured
...
[    93.755186] panic: kernel diagnostic assertion "ci->ci_mtx_count == -1" failed: file "/usr/src/sys/kern/kern_synch.c", line 762 mi_switch: cpu0: ci_mtx_count (-2) != -1 (block with spin-mutex held)
[    93.755186] cpu0: Begin traceback...
[    93.755186] vpanic(c13ec9a4,db9ddb1c,db9ddb64,c0d8486b,c13ec9a4,c12d733f,c13ecc68,c13ec7e4,2fa,c12b9380) at netbsd:vpanic+0x184
[    93.755186] kern_assert(c13ec9a4,c12d733f,c13ecc68,c13ec7e4,2fa,c12b9380,0,fffffffe,c0d7fc5b,c32ad400) at netbsd:kern_assert+0x23
[    93.755186] mi_switch(c32ad400,c0d80241,c2c55640,c166dec0,0,c13f4c9f,c1673d84,c1673d84,c1673d80,0) at netbsd:mi_switch+0x81a
[    93.755186] sleepq_block(0,0,c12b86e0,0,0,c166dec0,0,33,c1673d80,db9ddbec) at netbsd:sleepq_block+0x22e
[    93.755186] cv_wait(c1673d84,c1673d80,c4f68104,c4e76964,33,c1673d84,c2c7d960,c2f1b104,20000,db9ddc0c) at netbsd:cv_wait+0xe4
[    93.755186] xc_wait(33,0,0,0,c2c7d960,c2c7b530,db9ddc28,c06756f9,c2f1b104,2) at netbsd:xc_wait+0x5e
[    93.755186] agp_i810_chipset_flush(c2f1b104,2,0,c4e05200,0,db9ddc44,c0729312,c4e0526c,18,c4e05200) at netbsd:agp_i810_chipset_flush+0x85
[    93.755186] intel_gtt_chipset_flush(c4e0526c,18,c4e05200,c3154ac0,c3154b64,db9ddc78,c07137f4,c2c7d960,2cc00,cc00) at netbsd:intel_gtt_chipset_flush+0x1a
[    93.755186] intel_gt_flush_ggtt_writes(c2c7d960,2cc00,cc00,c4e05200,0,c4e052ac,c4e052b0,c071aa7a,c4e05200,0) at netbsd:intel_gt_flush_ggtt_writes+0x87
[    93.755186] i915_gem_object_flush_write_domain(c4e05200,fffffffe,3,c4f68158,c078a8c5,c4e05318,ffff0000,c2c7b004,c4f68104,c4e05200) at netbsd:i915_gem_object_flush_write_domain+0x171
[    93.755186] i915_gem_object_prepare_write(c4e05200,db9ddd24,0,0,0,0,d,0,7,c4f13340) at netbsd:i915_gem_object_prepare_write+0x180
[    93.755186] i915_gem_pwrite_ioctl(c2c7b004,db9ddeac,c4dcc244,c4354e5c,c4e5aa80,0,db9dddc4,20,1d,db9dddc4) at netbsd:i915_gem_pwrite_ioctl+0x6fb
[    93.755186] drm_ioctl(c4234180,8020645d,db9ddeac,db9ddf38,c0dcdbce,c4234180,8020645d,db9ddeac,db9ddeb0,c0d8ad5f) at netbsd:drm_ioctl+0x246
[    93.755186] drm_ioctl_shim(c4234180,8020645d,db9ddeac,db9ddeb0,c0d8ad5f,c161c8e0,ffffffff,0,c4234180,0) at netbsd:drm_ioctl_shim+0x43
[    93.755186] sys_ioctl(c32ad400,db9ddf68,db9ddf60,c4c05a68,0,c32ad400,c1651038,db9ddf68,0,0) at netbsd:sys_ioctl+0x35b
[    93.755186] syscall() at netbsd:syscall+0x194

This happens because:

/* i915_gem_object_flush_write_domain */
...
		spin_lock(&obj->vma.lock);
		for_each_ggtt_vma(vma, obj) {
			if (i915_vma_unset_ggtt_write(vma))
				intel_gt_flush_ggtt_writes(vma->vm->gt);
		}
		spin_unlock(&obj->vma.lock);

/* agp_i810_chipset_flush */
...
			xc_wait(xc_broadcast(0, &agp_flush_cache_xc,
				NULL, NULL));

>How-To-Repeat:
launch X with a DIAGNOSTIC kernel on a machine with an affected Intel graphics chipset
>Fix:
change agp_i810_chipset_flush from xcall(9) to ipi(9)