port-m68k/55990: kernel stack leak in m68k cpu_setmcontext() and reenter_syscall()

To: port-m68k-maintainer%netbsd.org@localhost,gnats-admin%netbsd.org@localhost,netbsd-bugs%netbsd.org@localhost
Subject: port-m68k/55990: kernel stack leak in m68k cpu_setmcontext() and reenter_syscall()
From: Izumi Tsutsui <tsutsui%ceres.dti.ne.jp@localhost>
Date: Fri, 12 Feb 2021 12:05:00 +0000 (UTC)
>Number:         55990
>Category:       port-m68k
>Synopsis:       kernel stack leak in m68k cpu_setmcontext() and reenter_syscall()
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    port-m68k-maintainer
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Fri Feb 12 12:05:00 +0000 2021
>Originator:     Izumi Tsutsui
>Release:        NetBSD 9.1
>Organization:
>Environment:
System: NetBSD ferrari 9.1 NetBSD 9.1 (GENERIC) #13: Fri Feb 12 18:53:22 JST 2021  tsutsui@mirage:/s/netbsd-9/src/sys/arch/sun3/compile/GENERIC sun3
Architecture: m68k
Machine: confirmed on x68k and sun3, but maybe affects all m68k ports
>Description:
During investigation of panics on NetBSD/sun3 (on 3/60) and NetBSD/x68k,
I've found there are kernel stack leaks in cpu_setmcontext() in
m68k/sig_machdep.c, and it seems to cause random Address error or
MMU fault panics, especially on running Xservers etc.

With the following debug printfs, it shows stack address is moved
to forward on each reenter_syscall() calls:

---

Index: sig_machdep.c
===================================================================
RCS file: /cvsroot/src/sys/arch/m68k/m68k/sig_machdep.c,v
retrieving revision 1.50
diff -u -p -d -r1.50 sig_machdep.c
--- sig_machdep.c	27 Nov 2018 14:09:54 -0000	1.50
+++ sig_machdep.c	12 Feb 2021 11:46:01 -0000
@@ -287,6 +287,8 @@ cpu_setmcontext(struct lwp *l, const mco
 	unsigned int format = mcp->__mc_pad.__mc_frame.__mcf_format;
 	int sz, error;
 
+printf("%s: stack (&sz) = %p", __func__, &sz);
+
 	/* Validate the supplied context */
 	if ((flags & _UC_CPU) != 0) {
 		error = cpu_mcontext_validate(l, mcp);
@@ -301,8 +303,10 @@ cpu_setmcontext(struct lwp *l, const mco
 		sz = exframesize[format];
 		if (sz < 0)
 			return (EINVAL);
+printf(" restore frame (format=%d, sz=%d)", format, sz);
 
 		if (frame->f_stackadj == 0) {
+printf(" ->reenter_syscall\n");
 			reenter_syscall(frame, sz);
 			/* NOTREACHED */
 		}
@@ -411,5 +415,6 @@ cpu_setmcontext(struct lwp *l, const mco
 		l->l_sigstk.ss_flags &= ~SS_ONSTACK;
 	mutex_exit(l->l_proc->p_lock);
 
+printf(" ->return\n");
 	return 0;
 }

---

On NetBSD/x68k 9.1 (12MB X68030 on XM6i emulator)
---
4) ->return
cpu_setmcontext: stack (&sz) = 0x39c2bec restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c2b98 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c2b98 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c2b44 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c2b44 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c2af0 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c2af0 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c2a9c restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c2a9c restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c2a48 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c2a48 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c29f4 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c29f4 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c29a0 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c29a0 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c294c restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c294c restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c28f8 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c28f8 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c28a4 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c28a4 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c2850 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c2850 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c27fc restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c27fc restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c27a8 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c27a8 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c2754 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c2754 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c2700 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c2700 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c26ac restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c26ac restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c2658 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c2658 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c2604 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c2604 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c25b0 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c25b0 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c255c restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c255c restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c2508 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c2508 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c24b4 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c24b4 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c2460 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c2460 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c240c restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c240c restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c23b8 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c23b8 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c2364 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c2364 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c2310 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c2310 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c22bc restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c22bc restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c2268 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c2268 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c2214 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c2214 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c21c0 restore frame (format=11, sz=84) ->return
cpu_setmcontext: stack (&sz) = 0x39c21c0 restore frame (format=11, sz=84) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0x39c216c restore frame (format=11, sz=84)uvm_fault(0x2d20c0, 0x7fff0000, 0x1) -> 0xe
  type 8, code [mmu,,ssw]: 4015166
trap type 8, code = 0x4015166, v = 0x7fff0000
kernel program counter = 0x7fff0000
kernel: MMU fault trap
pid = 1112, lid = 1, pc = 7FFF0000, ps = 2008, sfc = 1, dfc = 1
Registers:
             0        1        2        3        4        5        6        7
dreg: FFFFFFFF 0000FFFF 000A000D 0022BAC8 00000054 00000000 00000000 0016CB00
areg: FFFFFFFF 039C2178 039C21E0 039C2000 008BB660 00182C6C 039C2170 FFEFF190

Kernel stack (039C1F3C):
9C1F3C: 000035C4 039C2094 00000080 000A000D 0022BAC8 00000054 00000000 00000000
9C1F5C: 0016CB00 039C21E0 039C2000 008BB660 00182C6C 0087E458 039C1F9C 00000001
9C1F7C: 002D20C0 7FFF0000 00000D00 0082F3C0 00000000 002323D8 00000000 00000D00
9C1F9C: 039C1FD0 0000B310 0082F3C0 00000009 00000000 00000D00 00000029 0000000A
9C1FBC: FFFFFFFC 00181A24 00000000 00000D00 002D6B29 039C1FF4 00007668 00000000
9C1FDC: 00000D00 00000029 00000029 00000005 00000000 00000D00 039C2078 00183084
9C1FFC: 00000029 
panic: MMU fault
cpu0: Begin traceback...
?(?)
db_panic(2000,8,10,182c6c,39c1f3c) at 0
cpu0: End traceback...

dumping to dev 4,1 offset 239549
dump 

---

On NetBSD/sun3 9.1 (24M real sun3/60)
---
0220] cpu_setmcontext: stack (&sz) = 0xf7b84e8 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b84e8 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b84d0 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b84d0 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b84b8 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b84b8 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b84a0 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b84a0 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b8488 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b8488 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b8470 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b8470 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b8458 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b8458 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b8440 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b8440 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b8428 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b8428 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b8410 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b8410 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b83f8 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b83f8 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b83e0 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b83e0 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b83c8 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b83c8 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b83b0 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b83b0 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b8398 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b8398 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b8380 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b8380 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b8368 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b8368 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b8350 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b8350 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b8338 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b8338 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b8320 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b8320 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b8308 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b8308 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b82f0 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b82f0 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b82d8 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b82d8 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b82c0 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b82c0 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b82a8 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b82a8 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b8290 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b8290 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b8278 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b8278 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b8260 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b8260 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b8248 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b8248 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b8230 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b8230 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b8218 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b8218 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b8200 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b8200 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b81e8 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b81e8 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b81d0 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b81d0 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b81b8 restore frame (format=10, sz=24) ->return
cpu_setmcontext: stack (&sz) = 0xf7b81b8 restore frame (format=10, sz=24) ->reenter_syscall
cpu_setmcontext: stack (&sz) = 0xf7b81a0 restore frame (format=10, sz=24)trap type=0x1, code=0xa045, v=0xffffffff
kernel: Address error trap
pid = 549, lid = 1, pc = 0E152C0E, ps = 2008, sfc = 1, dfc = 1
Registers:
             0        1        2        3        4        5        6        7
dreg: FFFFFFFF 0000FFFF 000A000D 0E152B94 00000004 00000000 00000027 00004000
areg: FFFFFFFF 0F7B8178 0F7B8210 0F7B8000 0E301920 0E0E277C 0F7B81A4 0DFFF364

Kernel stack (0F7B7FE4):
7B7FE4: 0E008D3E 0F7B80D4 00000080 000A000D 0E152B94 00000004 00000000 
panic: Address error
cpu0: Begin traceback...
?(?)
db_panic(1,2000,f7b8000,e0e29b6,f7b7fe4) at 0
cpu0: End traceback...

dumping to dev 7,1 offset 213232

---


>How-To-Repeat:
See above.

It would be triggered by instruction address errors by page faults,
i.e. memory shortage, so only happens on lower (<32MB) memory machines?

>Fix:
It looks m68k/reenter_syscall.s adjusts stack pointer to prepare
"moved stack frame by stkadj bytes" but doesn't restore %sp
after syscall() is returned?
(I'm not sure how reenter_syscall() was designed though)

---
Izumi Tsutsui
Prev by Date: Re: kern/55884: viadrmums* at vga* does not add new line in dmesg
Next by Date: Re: bin/55979 (sh single quotes removes nul characters)
Previous by Thread: Re: kern/55982 (Subject: kernel crash booting Dell T430 with Radeon video card)
Next by Thread: Re: port-m68k/55990: kernel stack leak in m68k cpu_setmcontext() and reenter_syscall()
Indexes:
Home | Main Index | Thread Index | Old Index