Re: Fwd: FreeRADIUS instability

["Pawel S. Veselov" <pawel.veselov%gmail.com@localhost>]

On 9/16/21 11:31 AM, Dima Veselov wrote:

I'm trying to help Dima with this problem.

> > I do not know if this is NetBSD-related, but I suffer from FreeRADIUS
> > instability on NetBSD for a long time and do not know how to debug this.
> >
> > Symptoms are: RADIUS server randomly (once a day or once a week) can stop
> > answering and this is not connected to the actual load. While in that state
> > it can be killed with -9 only, other signals do nothing

> Well, it seems that the signals are blocked and this does not have to
> do with kevent (probably FreeRADIUS does it explicitly).

There is no issue with the kevent() call being responsive. kevent() 
obviously cycles at least on signals.

> So perhaps the handler does something and does not exit? 

The signal handlers respond just fine. The code isn't stuck in a signal 
handler, and I can see that kevent() exits on a signal, but the app just 
re-enters it shortly thereafter.

> Yes, the question is what happened to fd#3 (presumably the kqueue).
> If you can get into the debugger (gdb <radiusd> <pid>) and look at
> queue call and see what fd is passed to it? 

It's still fd#3



I suspect that this is an application problem and that either the filter 
is nulled out (or configured to filter events it doesn't care about), or 
there is a bug in the app's event processing.

The debugging proves to be a bit problematic though, and I don't quite 
understand what is going on.

When I try to set up breakpoints, they either don't stick, or fail with 
a memory access violation.

[root@almaz ~]$ gdb -p 18452
(gdb) break kevent
Breakpoint 1 at 0x74a6cf479479
(gdb) break src/lib/event.c:628
Breakpoint 2 at 0x74a6d2035ac4: file src/lib/event.c, line 628.
(gdb) cont
.... time passes, I send signals, can see GDB stops on signals, etc ...
(gdb) info breakpoints
Num     Type           Disp Enb Address            What
1       breakpoint     keep y   <PENDING>          kevent
2       breakpoint     keep y   <PENDING>          src/lib/event.c:628

So breakpoints never fire. Or:

(gdb) break process.c:5019
Breakpoint 3 at 0x449977: file src/main/process.c, line 5019.
(gdb) break process.c:5026
Breakpoint 4 at 0x4499a0: file src/main/process.c, line 5026.
(gdb) continue
Continuing.
Warning:
Cannot insert breakpoint 3.
Cannot access memory at address 0x449977
Cannot insert breakpoint 4.
Cannot access memory at address 0x4499a0

Command aborted.
(gdb) info breakpoints
Num     Type           Disp Enb Address            What
3       breakpoint     keep y   0x0000000000449977 in event_status at 
src/main/process.c:5019
4       breakpoint     keep y   0x00000000004499a0 in event_status at 
src/main/process.c:5026

I'm not new to gdb, but have 0 NetBSD experience, so I'd appreciate if 
you could point me in the right direction.