port-sparc64/57785: unable to use iscsi kernel initiator on sparc64

[2857%gmx.de@localhost]

>Number:         57785
>Category:       port-sparc64
>Synopsis:       unable to use iscsi kernel initiator on sparc64
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    port-sparc64-maintainer
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Dec 19 03:50:00 +0000 2023
>Originator:     zip100
>Release:        NetBSD 9.3
>Organization:
>Environment:
target: NetBSD netbsdvm 9.3 NetBSD 9.3 (GENERIC) #0: Thu Aug  4 15:30:37 UTC 2022  mkrepro%mkrepro.NetBSD.org@localhost:/usr/src/sys/arch/amd64/compile/GENERIC amd64

initiator: 
NetBSD sunfish 9.3 NetBSD 9.3 (GENERIC) #0: Thu Aug  4 15:30:37 UTC 2022  mkrepro%mkrepro.NetBSD.org@localhost:/usr/src/sys/arch/sparc64/compile/GENERIC sparc64
>Description:
Despite initiator being a sparc64 machine, I do believe it's not related to endianness, though I might be wrong...anyway, just trying to follow up the example from https://man.netbsd.org/iscsictl.8 . 

---------------------------------------- 
HOSTNAME     ADDRESS           ROLE
sunfish      192.168.33.64.    initiator
netbsdvm     192.168.33.199    target
----------------------------------------

I've setup a 256M test disk:
netbsdvm # cat /etc/iscsi/targets
# extent        file or device          start           length
extent0         /tmp/iscsi-target0      0               256MB
# target        flags   storage         netmask
target0         rw      extent0         0.0.0.0/0

netbsdvm # grep iscsi /etc/rc.conf
iscsi_target=YES


Kernel messages:

<snip>

Dec 19 03:17:03 netbsdvm iscsi-target: > iSCSI Discovery login  successful from iqn.1994-04.org.netbsd:iscsi.sunfish:2201371340 on 192.168.33.64 disk -1, ISID 70368764559360, TSIH 1
Dec 19 03:17:03 netbsdvm iscsi-target: pid 474:/usr/src/external/bsd/iscsi/lib/../dist/src/lib/target.c:1223: ***ERROR*** CmdSN
Dec 19 03:17:03 netbsdvm iscsi-target: pid 474:/usr/src/external/bsd/iscsi/lib/../dist/src/lib/target.c:1388: ***ERROR*** logout_command_t() failed 
Dec 19 03:17:03 netbsdvm iscsi-target: pid 474:/usr/src/external/bsd/iscsi/lib/../dist/src/lib/target.c:1502: ***ERROR*** execute_t() failed 
^^^^ output after refresh_targets



Dec 19 03:17:52 netbsdvm iscsi-target: pid 474:/usr/src/external/bsd/iscsi/lib/../dist/src/lib/target.c:1108: ***ERROR*** cmd.tsih 0 not found 
Dec 19 03:17:52 netbsdvm iscsi-target: > iSCSI Normal login  successful from iqn.1994-04.org.netbsd:iscsi.sunfish:2201371340 on 192.168.33.64 disk 0, ISID 70368764559360, TSIH 2
Dec 19 03:17:52 netbsdvm iscsi-target: pid 474:/usr/src/external/bsd/iscsi/lib/../dist/src/lib/disk.c:1364: ***ERROR*** UNKNOWN OPCODE 0xa3 
^^^ output after login



Kernel log on client after running final newfs command:
[ 1655.6300054] sd0(iscsi0:0:0:0): QUEUE FULL resulted in 0 openings
[ 1655.7027396] sd0(iscsi0:0:0:0): QUEUE FULL resulted in 0 openings
[ 1655.7754745] sd0(iscsi0:0:0:0): QUEUE FULL resulted in 0 openings
[ 1655.8482093] sd0(iscsi0:0:0:0): QUEUE FULL resulted in 0 openings
[ 1655.9209439] sd0(iscsi0:0:0:0): QUEUE FULL resulted in 0 openings
[ 1655.9936783] sd0(iscsi0:0:0:0): QUEUE FULL resulted in 0 openings
[ 1656.0664131] sd0(iscsi0:0:0:0): QUEUE FULL resulted in 0 openings

(spamming endlessly on the console, machine is unresponsive via network)
>How-To-Repeat:
Let's follow along the example from https://man.netbsd.org/iscsictl.8 :

sunfish # iscsictl add_send_target -a 192.168.33.199
Added Send Target 1

sunfish # iscsictl refresh_targets
OK

sunfish # iscsictl list_targets
     1: iqn.1994-04.org.netbsd.iscsi-target:target0
        2: 192.168.33.199:3260,1

sunfish # iscsictl login -P 1 
Created Session 2, Connection 1

sunfish # iscsictl list_sessions
Session 2: Target iqn.1994-04.org.netbsd.iscsi-target:target0

sunfish # newfs /dev/rsd0a
newfs: /dev/rsd0a: open for read: Device not configured

sunfish # newfs /dev/rsd0 
newfs: Unable to determine file system size

sunfish # newfs /dev/rsd0
newfs: Unable to determine file system size

sunfish # disklabel sd0
# /dev/rsd0:
type: SCSI
disk: NetBSD iSCSI    
label: fictitious
flags:
bytes/sector: 512
sectors/track: 32
tracks/cylinder: 64
sectors/cylinder: 2048
cylinders: 256
total sectors: 524288
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0           # microseconds
track-to-track seek: 0  # microseconds
drivedata: 0 

3 partitions:
#        size    offset     fstype [fsize bsize cpg/sgs]
 c:    524288         0     unused      0     0        # (Cyl.      0 -    255)
disklabel: boot block size 0
disklabel: super block size 0


----------------------------------------------------------------------

Trying to fix the situation with fdisk:

sunfish # fdisk -0ua sd0
fdisk: primary partition table invalid, no magic in sector 0
fdisk: Cannot determine the number of heads
Disk: /dev/rsd0
NetBSD disklabel disk geometry:
cylinders: 256, heads: 64, sectors/track: 32 (2048 sectors/cylinder)
total sectors: 524288, bytes/sector: 512

BIOS disk geometry:
cylinders: 256, heads: 64, sectors/track: 32 (2048 sectors/cylinder)
total sectors: 524288

Partitions aligned to 2048 sector boundaries, offset 32

Do you want to change our idea of what BIOS thinks? [n]  

Partition 0:
<UNUSED>
The data for partition 0 is:
<UNUSED>
sysid: [0..255 default: 169] 
start: [0..256cyl default: 32, 0cyl, 0MB] 
size: [0..256cyl default: 524256, 256cyl, 256MB] 
Do you want to change the active partition? [n] y
Choosing 4 will make no partition active.
active partition: [0..4 default: 0] 
Are you happy with this choice? [n] y

We haven't written the MBR back to disk yet.  This is your last chance.
Partition table:
0: NetBSD (sysid 169)
    start 32, size 524256 (256 MB, Cyls 0-255), Active
        PBR is not bootable: All bytes are identical (0x00)
1: <UNUSED>
2: <UNUSED>
3: <UNUSED>
First active partition: 0
Drive serial number: 0 (0x00000000)
Should we write new partition table? [n] y

^^^^^ could serial number of 000000000 be the problem? also, where did 3 empty partitions come from...


Running disklabel, apparently this should help...

sunfish # disklabel -i sd0
Enter '?' for help
partition>P
3 partitions:
#        size    offset     fstype [fsize bsize cpg/sgs]
 c:    524288         0     unused      0     0        # (Cyl.      0 -    255)
partition>e
Filesystem type [unused]: 4.2BSD
Start offset ('x' to start after partition 'x') [0c, 0s, 0M]: 
Partition size ('$' for all remaining) [0c, 0s, 0M]: $
 e:    524288         0     4.2BSD      0     0     0  # (Cyl.      0 -    255)
partition>W
Label disk [n]?y
Label written
partition>Q



And this completely locks up the SSH session, spamming QUEUE FULL on the console:
sunfish # newfs /dev/rsd0a


>Fix: