Re: "ro->_ro_rt ==NULL || ro->_ro_rt->rt_refcnt > 0" failed

[Ryota Ozaki <ozaki-r%netbsd.org@localhost>]

Hi,

I've committed a fix for rt_refcnt. Could you try again
with -current? (though I'm not sure the fix is related to
the issue...)

Thanks,
  ozaki-r

On Fri, Aug 28, 2015 at 3:55 PM, Ryota Ozaki <ozaki-r%netbsd.org@localhost> wrote:
> Hi,
>
> Thank you for sending backtraces. I'm investigating now...
>
>   ozaki-r
>
>
> On Fri, Aug 28, 2015 at 1:46 PM, Paul Goyette <paul%vps1.whooppee.com@localhost> wrote:
>> On Fri, 28 Aug 2015, Paul Goyette wrote:
>>
>>> On Fri, 28 Aug 2015, Paul Goyette wrote:
>>>
>>>> On Fri, 28 Aug 2015, Jun Ebihara wrote:
>>>>
>>>>> On: i386 kernel from nyftp.
>>>>> 7.99.21 NetBSD 7.99.21 (GENERIC.201508271450Z) #0: Thu Aug 27 17:23:37
>>>>> UTC 2015
>>>>> builds%b47.netbsd.org@localhost:/home/builds/ab/HEAD/i386/201508271450Z-obj/home/source/ab/HEAD/src/sys/arch/i386/compile/GENERIC
>>>>> i386
>>>>>
>>>>> sometimes panic around route.h.
>>>>>
>>>>> savecore: reboot after panic: panic: kernel diagnostic assertion
>>>>> "ro->_ro_rt ==NULL || ro->_ro_rt->rt_refcnt > 0" failed: file
>>>>> "/home/source/ab/HEAD/src/sys/net/route.h", line 433
>>>>>
>>>>
>>>> I had one of these crashes yesterday.  It happened when I was stopping a
>>>> pkgsrc/net/openvpn tunnel via "/etc/rc.d/openvpn onestop"
>>>>
>>>>
>>>> I don't bring my tunnel up very often, so I'm not sure if it is
>>>> reproducible...
>>>
>>>
>>> I don't know if this is the same crash as yesterday, but I just got
>>> another one!
>>>
>>> I started openvpn, then stopped it.  No crash.  so I repeated this a
>>> couple more times.  Still no crash.
>>>
>>> Then I brought the tunnel up, and did a 'ping6 ftp.netbsd.org' and it
>>> crashed almost immediately.
>>
>>
>> Here's the backtrace again, this time with symbol table loaded!
>>
>> (gdb) bt
>> #0  0xffffffff802f5ae5 in cpu_reboot (howto=howto@entry=260,
>>     bootstr=bootstr@entry=0x0)
>>     at /build/netbsd-local/src/sys/arch/amd64/amd64/machdep.c:671
>> #1  0xffffffff803742d4 in vpanic (
>>     fmt=0xffffffff804b6e38 "kernel %sassertion \"%s\" failed: file \"%s\",
>> line %d ", ap=ap@entry=0xfffffe817d50eb70)
>>     at /build/netbsd-local/src/sys/kern/subr_prf.c:340
>> #2  0xffffffff80467583 in kern_assert (
>>     fmt=fmt@entry=0xffffffff804b6e38 "kernel %sassertion \"%s\" failed: file
>> \"%s\", line %d ")
>>     at /build/netbsd-local/src/sys/lib/libkern/kern_assert.c:51
>> #3  0xffffffff8033a0e3 in rtfree (rt=0xfffffe815325aab0)
>>     at /build/netbsd-local/src/sys/net/route.c:417
>> #4  0xffffffff8033a5fd in rtcache_clear (ro=ro@entry=0xfffffe821db78060)
>>     at /build/netbsd-local/src/sys/net/route.c:1473
>> #5  0xffffffff8033a681 in rtcache_free (ro=ro@entry=0xfffffe821db78060)
>>     at /build/netbsd-local/src/sys/net/route.c:1518
>> #6  0xffffffff8020a65a in in6_pcbdetach (in6p=0xfffffe821db78000)
>>     at /build/netbsd-local/src/sys/netinet6/in6_pcb.c:618
>> #7  0xffffffff80332bec in rip6_detach (so=<optimized out>)
>>     at /build/netbsd-local/src/sys/netinet6/raw_ip6.c:644
>> #8  0xffffffff80332ccc in rip6_detach_wrapper (a=0xfffffe817c792498)
>>     at /build/netbsd-local/src/sys/netinet6/raw_ip6.c:964
>> #9  0xffffffff803cea10 in soclose (so=0xfffffe817c792498)
>>     at /build/netbsd-local/src/sys/kern/uipc_socket.c:762
>> #10 0xffffffff803860a1 in soo_close (fp=0xfffffe81ef653980)
>>     at /build/netbsd-local/src/sys/kern/sys_socket.c:255
>> #11 0xffffffff802b0de4 in closef (fp=0xfffffe81ef653980)
>>     at /build/netbsd-local/src/sys/kern/kern_descrip.c:831
>> #12 0xffffffff802b39e3 in fd_free ()
>>     at /build/netbsd-local/src/sys/kern/kern_descrip.c:1561
>> #13 0xffffffff802badd5 in exit1 (l=l@entry=0xfffffe815a2249a0,
>>     rv=rv@entry=2)
>>     at /build/netbsd-local/src/sys/kern/kern_exit.c:275
>> #14 0xffffffff802db146 in sigexit (l=l@entry=0xfffffe815a2249a0,
>>     signo=signo@entry=2)
>>     at /build/netbsd-local/src/sys/kern/kern_sig.c:2048
>> #15 0xffffffff802db46b in postsig (signo=2)
>>     at /build/netbsd-local/src/sys/kern/kern_sig.c:1848
>> #16 0xffffffff802c4999 in lwp_userret (l=l@entry=0xfffffe815a2249a0)
>>     at /build/netbsd-local/src/sys/kern/kern_lwp.c:1530
>> #17 0xffffffff803866b4 in mi_userret (l=0xfffffe815a2249a0)
>>     at /build/netbsd-local/src/sys/sys/userret.h:94
>> #18 userret (l=0xfffffe815a2249a0) at ./machine/userret.h:82
>> #19 syscall (frame=0xfffffe817d50ef00)
>>     at /build/netbsd-local/src/sys/arch/x86/x86/syscall.c:184
>> #20 0xffffffff80100691 in Xsyscall ()
>> (gdb) frame 3
>> #3  0xffffffff8033a0e3 in rtfree (rt=0xfffffe815325aab0)
>>     at /build/netbsd-local/src/sys/net/route.c:417
>> warning: Source file is more recent than executable.
>> (gdb) print rt
>> $1 = (struct rtentry *) 0xfffffe815325aab0
>> (gdb) print *rt
>> $2 = {rt_nodes = {{rn_mklist = 0xfffffe81f18b9558, rn_p =
>> 0xfffffe821dbb4bb8,
>>       rn_b = -1, rn_bmask = 0 '\000', rn_flags = 4 '\004', rn_u = {rn_leaf =
>> {
>>           rn_Key = 0xfffffe821ddcc048 "\034\030",
>>           rn_Mask = 0xfffffe810e9882b8 "", rn_Dupedkey = 0x0}, rn_node = {
>>           rn_Off = 501006408, rn_L = 0xfffffe810e9882b8, rn_R = 0x0}}}, {
>>       rn_mklist = 0x0, rn_p = 0x0, rn_b = 0, rn_bmask = 0 '\000',
>>       rn_flags = 0 '\000', rn_u = {rn_leaf = {rn_Key = 0x0, rn_Mask = 0x0,
>>           rn_Dupedkey = 0x0}, rn_node = {rn_Off = 0, rn_L = 0x0,
>>           rn_R = 0x0}}}}, rt_gateway = 0xfffffe821e1cdeb8, rt_flags = 2051,
>>   rt_refcnt = 0, rt_use = 17, rt_ifp = 0xfffffe8138031810,
>>   ^^^^^^^^^^^^^^
>>   rt_ifa = 0xfffffe81ee677010, rt_ifa_seqno = 0, rt_llinfo = 0x0, rt_rmx = {
>>     rmx_locks = 0, rmx_mtu = 0, rmx_hopcount = 0, rmx_recvpipe = 0,
>>     rmx_sendpipe = 0, rmx_ssthresh = 0, rmx_rtt = 0, rmx_rttvar = 0,
>>     rmx_expire = 0, rmx_pksent = 0}, rt_gwroute = 0xfffffe815325a780,
>>   rt_timer = {lh_first = 0x0}, rt_parent = 0x0, _rt_key =
>> 0xfffffe821ddcc048,
>>   rt_tag = 0x0}
>>
>> (gdb)
>>
>>
>>
>>
>> -------------------------------------------------------------------------
>> | Paul Goyette     | PGP Key fingerprint:     | E-mail addresses:       |
>> | (Retired)        | FA29 0E3B 35AF E8AE 6651 | paul at whooppee.com    |
>> | Kernel Developer | 0786 F758 55DE 53BA 7731 | pgoyette at netbsd.org  |
>> -------------------------------------------------------------------------