kern/56982: mutex error (locking against myself) in wdc(4) NOIRQ case?

To: kern-bug-people%netbsd.org@localhost,gnats-admin%netbsd.org@localhost,netbsd-bugs%netbsd.org@localhost
Subject: kern/56982: mutex error (locking against myself) in wdc(4) NOIRQ case?
From: Izumi Tsutsui <tsutsui%ceres.dti.ne.jp@localhost>
Date: Sat, 27 Aug 2022 20:20:01 +0000 (UTC)

>Number:         56982
>Category:       kern
>Synopsis:       mutex error (locking against myself) in wdc(4) NOIRQ case?
>Confidential:   no
>Severity:       critical
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sat Aug 27 20:20:01 +0000 2022
>Originator:     Izumi Tsutsui
>Release:        NetBSD 9.3 (patched per PR/56403)
>Organization:
>Environment:
Architecture: m68k, but maybe all related devices
Machine: mac68k, but maybe all related devices
>Description:
As noted in port-mac68k/56973, wdc(4) on NetBSD 9.3 has a problem
around NOIRQ case.

After applied fixes for it from PR/56403, I've got the following
mutex panic:

---
[   1.0000000] Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
[   1.0000000]     2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017,
[   1.0000000]     2018, 2019, 2020, 2021, 2022
[   1.0000000]     The NetBSD Foundation, Inc.  All rights reserved.
[   1.0000000] Copyright (c) 1982, 1986, 1989, 1991, 1993
[   1.0000000]     The Regents of the University of California.  All rights reserved.

[   1.0000000] NetBSD 9.3 (INSTALL) #1: Sun Aug 28 03:42:18 JST 2022
[   1.0000000] 	tsutsui@mirage:/s/netbsd-9/src/sys/arch/mac68k/compile/obj.mac68k/INSTALL
[   1.0000000] Apple Macintosh Quadra 630  (68040)
[   1.0000000] cpu: delay factor 1059
[   1.0000000] fpu: emulator
[   1.0000000] total memory = 36864 KB
[   1.0000000] avail memory = 30168 KB
[   1.0000000] mrg: 'Quadra630 ROMs' ROM glue, tracing off, debug off, silent traps
[   1.0000000] mrg: I/O map kludge for ROMs that use hardware addresses directly.
[   1.0000000] mainbus0 (root)
[   1.0000000] obio0 at mainbus0
[   1.0000000] esp0 at obio0 addr 0 (quick): address 0x5a8000: NCR53C96, 16MHz, SCSI ID 7
[   1.0000000] scsibus0 at esp0: 8 targets, 8 luns per target
[   1.0000000] wdc0 at obio0 (Quadra/Performa series IDE interface)
[   1.0000000] atabus0 at wdc0 channel 0
[   1.0000000] adb0 at obio0
[   1.0000000] asc0 at obio0: Apple Sound Chip
[   1.0000000] intvid0 at obio0 @ f9001000: Valkyrie video subsystem
[   1.0000000] intvid0: 832 x 624, 256 color
[   1.0000000] macfb0 at intvid0
[   1.0000000] wsdisplay0 at macfb0 (kbdmux ignored)
[   1.0000000] sn0 at obio0: integrated SONIC Ethernet adapter
[   1.0000000] sn0: Ethernet address 08:00:07:9f:07:c6
[   1.0000000] iwm0 at obio0: Apple GCR floppy disk controller
[   1.0000000] iwm: Chip revision not supported (-77)
[   1.0000000] zsc0 at obio0 chip type 0 
[   1.0000000] zsc0 channel 0: d_speed   9600 DCD clk 0 CTS clk 0
[   1.0000000] zstty0 at zsc0 channel 0 (console i/o)
[   1.0000000] zsc0 channel 1: d_speed   9600 DCD clk 0 CTS clk 0
[   1.0000000] zstty1 at zsc0 channel 1
[   1.0000000] nubus0 at mainbus0
[   1.0083991] scsibus0: waiting 2 seconds for devices to settle...
[   1.0853573] adb0 (direct, Cuda): 2 targets
[   1.1437621] aed0 at adb0 addr 0: ADB Event device
[   1.2012555] akbd0 at adb0 addr 2: keyboard II (Japanese layout)
[   1.2729173] wskbd0 at akbd0 (mux ignored)
[   1.3225433] ams0 at adb0 addr 3: 1-button, 100 dpi mouse
[   1.4127240] wsmouse0 at ams0 (mux ignored)
[   3.1517508] sd0 at scsibus0 target 0 lun 0: <Logitec, LHD-U32H/E, 105S> disk fixed
[   3.2517469] sd0: 30533 MB, 62037 cyl, 16 head, 63 sec, 512 bytes/sect x 62533296 sectors
[   3.3685872] sd0: async, 8-bit transfers
[   3.8685298] cd0 at scsibus0 target 3 lun 0: <MATSHITA, CD-ROM CR-8008, 8.0e> cdrom removable
[   3.9736653] cd0: sync (248.00ns offset 15), 8-bit (4.032MB/s) transfers
[   7.1520444] wd0 at atabus0 drive 0
[   7.2187327] wd0: <IBM-DALA-3360>
[   7.2688515] wd0: 348 MB, 929 cyl, 16 head, 48 sec, 512 bytes/sect x 713472 sectors
[   7.2850956] boot device: sd0
[   7.3351658] root on md0a dumps on md0b
[   7.3896797] root file system type: ffs
[   7.4423464] kern.module.path=/stand/mac68k/9.3/modules
[   7.5184557] PRAM time does not appear to have been read correctly.
[   7.5964773] PRAM: 0x83da4f80, macos_boottime: 0xa6f18d0f.
Terminal type? [vt220] 
Erase set to backspace.
erase ^?, werase ^W, kill ^U, intr ^C

 NetBSD/mac68k 9.3

 This menu-driven tool is designed to help you install NetBSD to a hard disk,
 or upgrade an existing NetBSD system, with a minimum of work.
 In the following menus type the reference letter (a, b, c, ...) to select an
 item, or type CTRL+N/CTRL+P to select the next/previous item.
 The arrow keys and Page-up/Page-down may also work.
 Activate the current selection from the menu by typing the enter key.
 
 
 Thank you for using NetBSD!

[snip]

 Ok, let's upgrade NetBSD on your hard disk.  As always, this will change
 information on your hard disk.  You should have made a full backup before
 this procedure!  Do you really want to upgrade NetBSD?  (This is your last  
 warning before this procedure starts modifying your disks.)  


                               +---------------+
                               | Yes or no?    |
                               |               |
                               | a: No         |
                               |[  26.1879870] Mutex error: mutex_vector_enter,4
84: locking against myself     +---------------+

[  26.1879870] lock address : 0x0000000000a97148
[  26.1879870] current cpu  :                  0
[  26.1879870] current lwp  : 0x0000000000a2ee20
[  26.1879870] owner field  : 0x0000000000058000 wait/spin:                0/1

[  26.1879870] panic: lock error: Mutex: mutex_vector_enter,484: locking against
 myself: lock 0xa97148 cpu 0 lwp 0xa2ee20
[  26.1879870] cpu0: Begin traceback...
[  26.1879870] ?(?)
[  26.1879870] db_panic(a97148,1f6c14,1e4,4e0054,51758d8) at 0
[  26.1879870] vpanic(1fca5a,51758e4,5175924,131414,1fca5a) + 13a
[  26.1879870] panic(1fca5a,1f6bd0,1f6c14,1e4,1f6b23) + c
[  26.1879870] lockdebug_abort(1f6c14,1e4,a97148,4e0054,1f6b23) + 84
[  26.1879870] mutex_abort(?)
[  26.1879870] eventswitch(1f6c14,1e4,a97148,1f6b23) + 2c
[  26.1879870] mutex_spin_enter(a97148,4e92b8,a99f30,0,3ea16) + 6e
[  26.1879870] wddone(a32688,a99f30,98a0a0,a99f30) + 256
[  26.1879870] wdc_ata_bio_done(?)
[  26.1879870] wdcprobe1(98a098,a99f30,98a0a0) + 62
[  26.1879870] wdc_ata_bio_intr(?)
[  26.1879870] wdcprobe1(98a098,a99f30,0,1,98a098) + 92
[  26.1879870] wdc_ata_bio_poll(98a098,a99f30,0,98a008,1c1094) + 1c
[  26.1879870] ata_xfer_start(a99f30) + 114
[  26.1879870] atastart(?)
[  26.1879870] ata_exec_xfer(98a098,a99f30,a99f30,a97008,5175ae8) + 374
[  26.1879870] wdc_ata_bio(9e5008,a99f30) + 62
[  26.1879870] wdstart1(?)
[  26.1879870] smc91cxx_mii_writereg(a97008,a9cc20,a99f30,a970a0,12b462) + 208
[  26.1879870] wd_diskstart(a32688,a9cc20,a970a0,9ece28,c) + ce
[  26.1879870] dk_start(?)
[  26.1879870] dk_strategy(a97008,a9cc20) + d6
[  26.1879870] wdstrategy(a9cc20,4200,3102,3102,0) + 4c
[  26.1879870] readdisklabel(0,3102,399a8,9c2e08,996158) + 7e
[  26.1879870] dk_getdisklabel(a97008,0,3102) + 7e
[  26.1879870] dk_open(a97008,0,3102,1,2000,a2ee20) + dc
[  26.1879870] wdopen(?)
[  26.1879870] cdev_open(0,3102,1,2000,a2ee20) + 8e
[  26.1879870] spec_open(5175d34) + 118
[  26.1879870] VOP_OPEN(a30bd8,1,9d5ee0) + 46
[  26.1879870] vn_open(5175e64,1,0) + 18e
[  26.1879870] do_open(a2ee20,0,992c68,0,0,5175f00) + 96
[  26.1879870] do_sys_openat(?)
[  26.1879870] fifo_open(a2ee20,ffffff9c,ffff7f64,0,0,5175f00) + ee
[  26.1879870] sys_open(a2ee20,5175f38,5175f30,45,0) + 1e
[  26.1879870] syscall_plain(5,a2ee20,5175fb4,ffff7f64,0) + 82
[  26.1879870] syscall(5) + 70
[  26.1879870] trap0() + e
[  26.1879870] cpu0: End traceback...
Stopped in pid 9.1 (sysinst) at netbsd:cpu_Debugger+0x6:        unlk    a6
db> 

---

>How-To-Repeat:
As above, boot NetBSD/mac68k INSTALL kernel (that has options DIAGONSTIC)
on LC630 and other IDE based 68k Macs.

Maybe other NOIRQ wdc(4) variants are affected?
https://github.com/NetBSD/src/commit/d034868f5308ed012da088cd2a6112443061753a

Note installation works with INSTALL kernel built without options
DIAGNOSTIC. (not sure why mac68k INSTALL keeps DIAGNOSTIC even on releases)

>Fix:
Check mutex around wdc_ata_bio_done() or wddone()?

Follow-Ups:
- Re: kern/56982: mutex error (locking against myself) in wdc(4) NOIRQ case?
  - From: Rin Okuyama

Prev by Date: NetBSD Nightly Trouble Ticket Report
Next by Date: bin/56983: "SMALL test, no fallback usage" by test(1) looks annoying
Previous by Thread: kern/56981: Kernel panics with nouveau0: error: unknown chipset (ffffffff)
Next by Thread: Re: kern/56982: mutex error (locking against myself) in wdc(4) NOIRQ case?
Indexes:

Home | Main Index | Thread Index | Old Index