Re: "ro->_ro_rt ==NULL || ro->_ro_rt->rt_refcnt > 0" failed

[Paul Goyette <paul%vps1.whooppee.com@localhost>]

On Fri, 28 Aug 2015, Paul Goyette wrote:

> On Fri, 28 Aug 2015, Paul Goyette wrote:
>
> > On Fri, 28 Aug 2015, Jun Ebihara wrote:
> >
> > > On: i386 kernel from nyftp.
> > > 7.99.21 NetBSD 7.99.21 (GENERIC.201508271450Z) #0: Thu Aug 27 17:23:37 UTC 
> > > 2015 
> > > builds%b47.netbsd.org@localhost:/home/builds/ab/HEAD/i386/201508271450Z-obj/home/source/ab/HEAD/src/sys/arch/i386/compile/GENERIC 
> > > i386
> > >
> > > sometimes panic around route.h.
> > >
> > > savecore: reboot after panic: panic: kernel diagnostic assertion 
> > > "ro->_ro_rt ==NULL || ro->_ro_rt->rt_refcnt > 0" failed: file 
> > > "/home/source/ab/HEAD/src/sys/net/route.h", line 433
> >
> > I had one of these crashes yesterday.  It happened when I was stopping a 
> > pkgsrc/net/openvpn tunnel via "/etc/rc.d/openvpn onestop"
> >
> >
> > I don't bring my tunnel up very often, so I'm not sure if it is 
> > reproducible...
>
> I don't know if this is the same crash as yesterday, but I just got another 
> one!
>
> I started openvpn, then stopped it.  No crash.  so I repeated this a couple 
> more times.  Still no crash.
>
> Then I brought the tunnel up, and did a 'ping6 ftp.netbsd.org' and it crashed 
> almost immediately.

Here's the backtrace again, this time with symbol table loaded!

(gdb) bt
#0  0xffffffff802f5ae5 in cpu_reboot (howto=howto@entry=260,
    bootstr=bootstr@entry=0x0)
    at /build/netbsd-local/src/sys/arch/amd64/amd64/machdep.c:671
#1  0xffffffff803742d4 in vpanic (
    fmt=0xffffffff804b6e38 "kernel %sassertion \"%s\" failed: file \"%s\", line %d ", ap=ap@entry=0xfffffe817d50eb70)
    at /build/netbsd-local/src/sys/kern/subr_prf.c:340
#2  0xffffffff80467583 in kern_assert (
    fmt=fmt@entry=0xffffffff804b6e38 "kernel %sassertion \"%s\" failed: file \"%s\", line %d ")
    at /build/netbsd-local/src/sys/lib/libkern/kern_assert.c:51
#3  0xffffffff8033a0e3 in rtfree (rt=0xfffffe815325aab0)
    at /build/netbsd-local/src/sys/net/route.c:417
#4  0xffffffff8033a5fd in rtcache_clear (ro=ro@entry=0xfffffe821db78060)
    at /build/netbsd-local/src/sys/net/route.c:1473
#5  0xffffffff8033a681 in rtcache_free (ro=ro@entry=0xfffffe821db78060)
    at /build/netbsd-local/src/sys/net/route.c:1518
#6  0xffffffff8020a65a in in6_pcbdetach (in6p=0xfffffe821db78000)
    at /build/netbsd-local/src/sys/netinet6/in6_pcb.c:618
#7  0xffffffff80332bec in rip6_detach (so=<optimized out>)
    at /build/netbsd-local/src/sys/netinet6/raw_ip6.c:644
#8  0xffffffff80332ccc in rip6_detach_wrapper (a=0xfffffe817c792498)
    at /build/netbsd-local/src/sys/netinet6/raw_ip6.c:964
#9  0xffffffff803cea10 in soclose (so=0xfffffe817c792498)
    at /build/netbsd-local/src/sys/kern/uipc_socket.c:762
#10 0xffffffff803860a1 in soo_close (fp=0xfffffe81ef653980)
    at /build/netbsd-local/src/sys/kern/sys_socket.c:255
#11 0xffffffff802b0de4 in closef (fp=0xfffffe81ef653980)
    at /build/netbsd-local/src/sys/kern/kern_descrip.c:831
#12 0xffffffff802b39e3 in fd_free ()
    at /build/netbsd-local/src/sys/kern/kern_descrip.c:1561
#13 0xffffffff802badd5 in exit1 (l=l@entry=0xfffffe815a2249a0,
    rv=rv@entry=2)
    at /build/netbsd-local/src/sys/kern/kern_exit.c:275
#14 0xffffffff802db146 in sigexit (l=l@entry=0xfffffe815a2249a0,
    signo=signo@entry=2)
    at /build/netbsd-local/src/sys/kern/kern_sig.c:2048
#15 0xffffffff802db46b in postsig (signo=2)
    at /build/netbsd-local/src/sys/kern/kern_sig.c:1848
#16 0xffffffff802c4999 in lwp_userret (l=l@entry=0xfffffe815a2249a0)
    at /build/netbsd-local/src/sys/kern/kern_lwp.c:1530
#17 0xffffffff803866b4 in mi_userret (l=0xfffffe815a2249a0)
    at /build/netbsd-local/src/sys/sys/userret.h:94
#18 userret (l=0xfffffe815a2249a0) at ./machine/userret.h:82
#19 syscall (frame=0xfffffe817d50ef00)
    at /build/netbsd-local/src/sys/arch/x86/x86/syscall.c:184
#20 0xffffffff80100691 in Xsyscall ()
(gdb) frame 3
#3  0xffffffff8033a0e3 in rtfree (rt=0xfffffe815325aab0)
    at /build/netbsd-local/src/sys/net/route.c:417
warning: Source file is more recent than executable.
(gdb) print rt
$1 = (struct rtentry *) 0xfffffe815325aab0
(gdb) print *rt
$2 = {rt_nodes = {{rn_mklist = 0xfffffe81f18b9558, rn_p = 
0xfffffe821dbb4bb8,
      rn_b = -1, rn_bmask = 0 '\000', rn_flags = 4 '\004', rn_u = {rn_leaf = {
          rn_Key = 0xfffffe821ddcc048 "\034\030",
          rn_Mask = 0xfffffe810e9882b8 "", rn_Dupedkey = 0x0}, rn_node = {
          rn_Off = 501006408, rn_L = 0xfffffe810e9882b8, rn_R = 0x0}}}, {
      rn_mklist = 0x0, rn_p = 0x0, rn_b = 0, rn_bmask = 0 '\000',
      rn_flags = 0 '\000', rn_u = {rn_leaf = {rn_Key = 0x0, rn_Mask = 0x0,
          rn_Dupedkey = 0x0}, rn_node = {rn_Off = 0, rn_L = 0x0,
          rn_R = 0x0}}}}, rt_gateway = 0xfffffe821e1cdeb8, rt_flags = 2051,
  rt_refcnt = 0, rt_use = 17, rt_ifp = 0xfffffe8138031810,
  ^^^^^^^^^^^^^^
  rt_ifa = 0xfffffe81ee677010, rt_ifa_seqno = 0, rt_llinfo = 0x0, rt_rmx = {
    rmx_locks = 0, rmx_mtu = 0, rmx_hopcount = 0, rmx_recvpipe = 0,
    rmx_sendpipe = 0, rmx_ssthresh = 0, rmx_rtt = 0, rmx_rttvar = 0,
    rmx_expire = 0, rmx_pksent = 0}, rt_gwroute = 0xfffffe815325a780,
  rt_timer = {lh_first = 0x0}, rt_parent = 0x0, _rt_key = 0xfffffe821ddcc048,
  rt_tag = 0x0}
(gdb)




-------------------------------------------------------------------------
| Paul Goyette     | PGP Key fingerprint:     | E-mail addresses:       |
| (Retired)        | FA29 0E3B 35AF E8AE 6651 | paul at whooppee.com    |
| Kernel Developer | 0786 F758 55DE 53BA 7731 | pgoyette at netbsd.org  |
-------------------------------------------------------------------------