Re: Re: kern/58422: kernel crash when using the iscsi initiator

[6bone%6bone.informatik.uni-leipzig.de@localhost]

On Sat, 13 Jul 2024, Michael van Elst wrote:

> Date: Sat, 13 Jul 2024 08:00:03 +0000 (UTC)
> From: Michael van Elst <mlelstv%serpens.de@localhost>
> Reply-To: gnats-bugs%netbsd.org@localhost
> To: kern-bug-people%netbsd.org@localhost, gnats-admin%netbsd.org@localhost,
>     netbsd-bugs%netbsd.org@localhost, 6bone%6bone.informatik.uni-leipzig.de@localhost
> Subject: [Extern] Re: kern/58422: kernel crash when using the iscsi initiator
>
> The following reply was made to PR kern/58422; it has been noted by GNATS.
>
> From: mlelstv%serpens.de@localhost (Michael van Elst)
> To: gnats-bugs%netbsd.org@localhost
> Cc:
> Subject: Re: kern/58422: kernel crash when using the iscsi initiator
> Date: Sat, 13 Jul 2024 07:59:10 -0000 (UTC)
>
> 6bone%6bone.informatik.uni-leipzig.de@localhost writes:
>
> >NetBSD 6bone.informatik.uni-leipzig.de 10.0_STABLE NetBSD 10.0_STABLE (MYCONF10.gdb) #1: Mon Jul  8 15:34:51 CEST 2024  root%6bone.informatik.uni-leipzig.de@localhost:/usr/obj/sys/arch/amd64/compile/MYCONF10.gdb amd64
>
> >[ 85126.220101] cpu2: Begin traceback...
> >[ 85126.220101] vpanic() at netbsd:vpanic+0x183
> >[ 85126.220101] panic() at netbsd:panic+0x3c
> >[ 85126.220101] trap() at netbsd:trap+0xb24
> >[ 85126.220101] --- trap (number 6) ---
> >[ 85126.230191] mutex_enter() at netbsd:mutex_enter+0x3e
> >[ 85126.230191] wake_ccb() at iscsi:wake_ccb+0x8e
> >[ 85126.230191] iscsi_cleanup_thread() at iscsi:iscsi_cleanup_thread+0x351
> >[ 85126.230191] cpu2: End traceback...
>
>
> That's a use-after-free, when a connection is shut down.
>
> The cleanup thread handles timed out CCBs that have already
> been freed with the closing connection.

I don't know if it's relevant, but connecting the iscsi initiator occurs 
without a noticeable message in the dmesg.

After you start using the iscsi device, the messages "S3C1: freeing UNUSED 
pdu" appear regularly in the dmesg.

This goes on for a few days until the crash.

> Can you try to recreate the crash with a DIAGNOSTIC kernel and/or
> set the sysctl hw.iscsi.debug to 9 ?

The dump from the bug report already comes from a kernel with DIAGNOSTIC 
enabled. I will try to cause the crash with hw.iscsi.debug=9.

> It might be easier to recreate if you know how the connection
> is shut down. Maybe some reconfig or reboot of the iscsi target
> or some network issue.

The iscsi connection was only started once during boot. Then it ran until 
it crashed. The iscsi target is a NetApp system that runs very stable. I 
think a network problem is unlikely.


Thank you for your efforts

Regards
Uwe