bin/60086: cgdconfig may not always find ZFS device labels

["andrew%pmk1.net@localhost via gnats" <gnats-admin%NetBSD.org@localhost>]

>Number:         60086
>Category:       bin
>Synopsis:       cgdconfig may not always find ZFS device labels
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    bin-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Mar 16 12:30:00 +0000 2026
>Originator:     Andrew Parker
>Release:        NetBSD 11.0_RC2 (and probably others)
>Organization:
none
>Environment:
Architecture: x86_64       
Machine: amd64
>Description:
When using ZFS on top of cgd devices, the verify_zfs function in cgdconfig may not always find the last two labels.  This appears to depend on the size of the device.  In my case, I'm using named GPT wedges.

The verify_zfs function in cgdconfig calculates the offset of the last two disklabels using the device size.  ZFS zdb function (for example) uses an aligned value for determining the offset.  Because of this, cgdconfig's calculated offset may not be where the actual vdev label exists on the device which results in the verify function not working on boot.
>How-To-Repeat:
This is repeatable using a GPT named disk wedge on a USB device:

# gpt show sd0
     start      size  index  contents
         0         1         PMBR
         1         1         Pri GPT header
         2        32         Pri GPT table
        34  31129533      1  GPT part - NetBSD Cryptographic Disk
  31129567        32         Sec GPT table
  31129599         1         Sec GPT header

Followed the guide to get a working cgd device on top of the named wedge and create a zpool.  Verified the labels using zdb:

# zdb -l /dev/rcgd1d 
--------------------------------------------
LABEL 0
--------------------------------------------
    version: 5000
    name: 'sata'
    state: 0
    txg: 4
    pool_guid: 4175716765529626646
    hostname: 'claire-netbsd.local'
    top_guid: 10667087423283623277
    guid: 10667087423283623277
    vdev_children: 1
    vdev_tree:
        type: 'disk'
        id: 0
        guid: 10667087423283623277
        path: '/dev/cgd0d'
        whole_disk: 0
        metaslab_array: 37
        metaslab_shift: 32
        ashift: 12
        asize: 512105381888
        is_log: 0
        create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
--------------------------------------------
LABEL 1
--------------------------------------------
    version: 5000
    name: 'sata'
    state: 0
    txg: 4
    pool_guid: 4175716765529626646
    hostname: 'claire-netbsd.local'
    top_guid: 10667087423283623277
    guid: 10667087423283623277
    vdev_children: 1
    vdev_tree:
        type: 'disk'
        id: 0
        guid: 10667087423283623277
        path: '/dev/cgd0d'
        whole_disk: 0
        metaslab_array: 37
        metaslab_shift: 32
        ashift: 12
        asize: 512105381888
        is_log: 0
        create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
--------------------------------------------
LABEL 2
--------------------------------------------
    version: 5000
    name: 'usb'
    state: 2
    txg: 36
    pool_guid: 6389698929858570228
    hostname: ''
    top_guid: 4200771912375277257
    guid: 4200771912375277257
    vdev_children: 1
    vdev_tree:
        type: 'disk'
        id: 0
        guid: 4200771912375277257
        path: '/dev/cgd1d'
        whole_disk: 0
        metaslab_array: 37
        metaslab_shift: 27
        ashift: 9
        asize: 15933374464
        is_log: 0
        create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
--------------------------------------------
LABEL 3
--------------------------------------------
    version: 5000
    name: 'usb'
    state: 2
    txg: 36
    pool_guid: 6389698929858570228
    hostname: ''
    top_guid: 4200771912375277257
    guid: 4200771912375277257
    vdev_children: 1
    vdev_tree:
        type: 'disk'
        id: 0
        guid: 4200771912375277257
        path: '/dev/cgd1d'
        whole_disk: 0
        metaslab_array: 37
        metaslab_shift: 27
        ashift: 9
        asize: 15933374464
        is_log: 0
        create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data

# echo "1234" |sudo cgdconfig -p -v -Vzfs cgd1  /dev/dk4 # This is the correct p/w for testing
    with alg aes-cbc keylen 256 blocksize 18446744073709551615 ivmethod encblkno1
cgdconfig: verification failed permanently

After adding some extra logging to both cgdconfig and zdb I could see there were calculating different offsets for the last two labels.

>Fix:
Updating cgdconfig to use the same alignment function as zdb (external/cddl/osnet/dist/cmd/zdb/zdb.c) worked for me:

--- a/sbin/cgdconfig/cgdconfig.c
+++ b/sbin/cgdconfig/cgdconfig.c
@@ -1213,6 +1213,8 @@ verify_zfs(int fd)
                return rv;
        }
 
+       vdev_size = P2ALIGN(vdev_size, (uint64_t)sizeof (vdev_label_t));
+
        vdev_phys_t *vdev_phys = emalloc(sizeof(*vdev_phys));
        for (size_t i = 0; i < VDEV_LABELS; i++) {
                off_t vdev_phys_off = (i < VDEV_LABELS / 2 ?