Re: kern/59339: heartbeat watchdog fires since 10.99.14

["Thomas Klausner via gnats" <gnats-admin%NetBSD.org@localhost>]

The following reply was made to PR kern/59339; it has been noted by GNATS.

From: Thomas Klausner <wiz%NetBSD.org@localhost>
To: Taylor R Campbell <riastradh%NetBSD.org@localhost>
Cc: NetBSD bugtracking <gnats-bugs%NetBSD.org@localhost>
Subject: Re: kern/59339: heartbeat watchdog fires since 10.99.14
Date: Fri, 9 May 2025 14:59:02 +0200

 On Fri, May 09, 2025 at 11:55:27AM +0000, Taylor R Campbell wrote:
 > > Date: Fri, 9 May 2025 10:55:03 +0200
 > > From: Thomas Klausner <wiz%NetBSD.org@localhost>
 > > 
 > > A kernel with this patch paniced twice in 10 minutes.  I didn't even
 > > have time to start mutt and send an email after the first panic.
 > > 
 > > crash says
 > > 
 > > System panicked: kernel diagnostic assertion "ticks > 0" failed: file "/usr/src/sys/kern/kern_time.c", line 837 [0] it->it_time.it_value=1746779983.758601200 ticks=0
 > > 
 > > and
 > > 
 > > System panicked: kernel diagnostic assertion "ticks > 0" failed: file "/usr/src/sys/kern/kern_time.c", line 837 [0] it->it_time.it_value=1746780327.140798025 ticks=0
 > > 
 > > Let me know what further information you need, or if you want me to
 > > upload the crash dumps & debug kernels somewhere.
 > 
 > Can you get a stack trace from those?  Meanwhile I will get an updated
 > patch either for more diagnostics or a candidate fix.
 
 Seems to have been the same problem both times:
 
 (gdb) bt
 #0  0xffffffff80239b85 in cpu_reboot (howto=howto@entry=260, bootstr=bootstr@entry=0x0) at /usr/src/sys/arch/amd64/amd64/machdep.c:709
 #1  0xffffffff80de3ab5 in kern_reboot (howto=howto@entry=260, bootstr=bootstr@entry=0x0) at /usr/src/sys/kern/kern_reboot.c:91
 #2  0xffffffff80e29873 in vpanic (fmt=0xffffffff814f1ea0 "kernel %sassertion \"%s\" failed: file \"%s\", line %d [%u] it->it_time.it_value=%lld.%09ld ticks=%d", ap=ap@entry=0xffff8ba48411de08)
     at /usr/src/sys/kern/subr_prf.c:288
 #3  0xffffffff80ff803c in kern_assert (fmt=fmt@entry=0xffffffff814f1ea0 "kernel %sassertion \"%s\" failed: file \"%s\", line %d [%u] it->it_time.it_value=%lld.%09ld ticks=%d")
     at /usr/src/sys/lib/libkern/kern_assert.c:51
 #4  0xffffffff80dff166 in itimer_arm_real (it=0xffff89c5007f1e80) at /usr/src/sys/kern/kern_time.c:837
 #5  0xffffffff80e007eb in itimer_settime (it=it@entry=0xffff89c5007f1e80) at /usr/src/sys/kern/kern_time.c:946
 #6  0xffffffff80e019de in dotimer_settime (timerid=4, value=value@entry=0xffff8ba48411df40, ovalue=ovalue@entry=0x0, flags=1, p=<optimized out>) at /usr/src/sys/kern/kern_time.c:1463
 #7  0xffffffff80e01be0 in sys___timer_settime50 (l=0xffff89c4872ebc00, uap=0xffff8ba48411e000, retval=<optimized out>) at /usr/src/sys/kern/kern_time.c:1391
 #8  0xffffffff805b7cc1 in sy_call (rval=0xffff8ba48411dfb0, uap=0xffff8ba48411e000, l=0xffff89c4872ebc00, sy=0xffffffff8188a550 <sysent+10704>) at /usr/src/sys/sys/syscallvar.h:65
 #9  sy_invoke (code=446, rval=0xffff8ba48411dfb0, uap=0xffff8ba48411e000, l=0xffff89c4872ebc00, sy=0xffffffff8188a550 <sysent+10704>) at /usr/src/sys/sys/syscallvar.h:94
 #10 syscall (frame=0xffff8ba48411e000) at /usr/src/sys/arch/x86/x86/syscall.c:137
 #11 0xffffffff8021025d in handle_syscall ()
 (gdb) fr 4
 #4  0xffffffff80dff166 in itimer_arm_real (it=0xffffbb4807e73bc0) at /disk/storage-202410/archive/foreign/src/sys/kern/kern_time.c:837
 837             KASSERTMSG(ticks > 0, "[%u] it->it_time.it_value=%lld.%09ld ticks=%d",
 (gdb) p *it
 $1 = {{it_real = {it_ch = {_c_store = {0xffffbb485b69c1c0, 0xffffffff819837e0 <cache_stat_callout>, 0xffffffff80dff1ff <itimer_callout>, 0xffffbb4807e73bc0, 0xffffffff818b7840 <callout_cpu0>, 0x10000007530, 0x11deeba1,
           0x0, 0x0, 0x0}}, it_rtchgq = {le_next = 0x0, le_prev = 0x0}}, it_virtual = {it_vlist = 0xffffbb485b69c1c0, it_list = {le_next = 0xffffffff819837e0 <cache_stat_callout>,
         le_prev = 0xffffffff80dff1ff <itimer_callout>}, it_active = 192}}, it_ops = 0xffffffff8138a470 <ptimer_itimer_ops>, it_time = {it_interval = {tv_sec = 0, tv_nsec = 0}, it_value = {tv_sec = 1746780327,
       tv_nsec = 140798025}}, it_clockid = 0, it_overruns = 0, it_dying = false}
 (gdb) p ticks
 $2 = 0
 (gdb) p it->it_clockid
 $3 = 0
 # so not CLOCK_MONOTONIC, which is 3
 
 (gdb) p it->it_time.it_value
 $4 = {tv_sec = 1746780327, tv_nsec = 140798025}
 
 Anything else I can provide?
  Thomas