Re: gzimg hashes

To: "David H. Gutteridge" <david%gutteridge.ca@localhost>
Subject: Re: gzimg hashes
From: Brook Milligan <brook%biology.nmsu.edu@localhost>
Date: Tue, 7 Jul 2026 10:52:43 -0600
> On Jul 6, 2026, at 10:49, Brook Milligan <brook%biology.nmsu.edu@localhost> wrote:
> 
> 
>> On Jul 5, 2026, at 20:12, David H. Gutteridge <david%gutteridge.ca@localhost> wrote:
>> 
>> On Sun, 5 Jul 2026 at 17:42:17 -0600, Brook Milligan wrote:
>>> I notice that the hashes generated for release files in the gzimg
>>> directory cover the uncompressed files, not the compressed files
>>> actually distributed.  This seems to be a mistake.
>>> 
>>> The fix is clearly to reorder the hash generation relative to the
>>> compression, but I’m checking here to make sure I’m not missing
>>> something.
>>> 
>>> Cheers,
>>> Brook
>> 
>> Yes, that's one of the issues raised in PR 59195 (and its duplicate,
>> PR 60302), e.g.,
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail-index.netbsd.org%2Fnetbsd-bugs%2F2025%2F04%2F03%2Fmsg088095.html&data=05%7C02%7Cbrook%40biology.nmsu.edu%7Cce313e1520f249b728b708dedb04114e%7Ca3ec87a89fb84158ba8ff11bace1ebaa%7C1%7C0%7C639189007694159583%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C60000%7C%7C%7C&sdata=%2Bv5UsD4R9GqmbVitcRxAG%2F81hAsh4KAIIb0ILBfSbhc%3D&reserved=0
> 
> OK, thanks.  I have a patch, see below, that solves most of this.  

Here is (I believe) a better patch + the referenced mkhashes script.  It does the following:

- reverts the commit to src/distrib/utils/embedded/mkimage (rev 1.91) so mkimage is no longer responsible for hashes

- creates the script src/distrib/utils/embedded/mkimage, which makes a standard set of hashes.  The boilerplate is based upon mkimage, so I hope that is OK.

- uses mkhashes to generate hashes for all (that I know of) users, i.e., in src/etc/etc.*/Makefile.inc

Note that only a single hash for any file will be retained and multiple hashes will accumulate in the files.

I think this solves the issues mentioned in PR 59195 and PR 60302.

Comments are welcome.

Thanks a lot.

Cheers,
Brook


Index: distrib/utils/embedded/mkimage
===================================================================
RCS file: /cvsroot/src/distrib/utils/embedded/mkimage,v
retrieving revision 1.97
diff -u -r1.97 mkimage
--- distrib/utils/embedded/mkimage	22 May 2026 14:44:16 -0000	1.97
+++ distrib/utils/embedded/mkimage	7 Jul 2026 16:36:35 -0000
@@ -425,8 +425,4 @@
 	image="${image}.gz"
 fi
 
-cd "${IMAGEDIR}"
-${CKSUM} -a MD5 "$(basename "${image}")" > MD5
-${CKSUM} -a SHA512 "$(basename "${image}")" > SHA512
-
 echo ${bar} Image is ${image} ${bar}
Index: etc/etc.evbarm/Makefile.inc
===================================================================
RCS file: /cvsroot/src/etc/etc.evbarm/Makefile.inc,v
retrieving revision 1.138
diff -u -r1.138 Makefile.inc
--- etc/etc.evbarm/Makefile.inc	20 Feb 2026 07:20:28 -0000	1.138
+++ etc/etc.evbarm/Makefile.inc	7 Jul 2026 16:36:35 -0000
@@ -3,6 +3,7 @@
 #	etc.evbarm/Makefile.inc -- evbarm-specific etc Makefile targets
 #
 
+MKHASHES=${NETBSDSRCDIR}/distrib/utils/embedded/mkhashes
 MKIMAGE= ${NETBSDSRCDIR}/distrib/utils/embedded/mkimage
 
 # If you change the list of distributed kernels, don't forget
@@ -96,8 +97,6 @@
 	MACHINE=${MACHINE} \
 	MKDTB=${MKDTB} \
 	MKPOSTFIX=${MKPOSTFIX} \
-	CKSUM=${CKSUM} \
-	IMAGEDIR=${IMAGE.dir} \
 	${HOST_SH} ${MKIMAGE} -h ${.TARGET:S/smp_//} -D ${DESTDIR} \
 	    -S ${NETBSDSRCDIR} -B ${IMAGEENDIAN} ${MKI_OPTS.${.TARGET}} \
 	    ${IMAGE.dir}/${.TARGET:S/smp_//}.img
@@ -107,10 +106,12 @@
 	@${TOOL_INSTALLBOOT} -m ${MACHINE} -o board=${f} \
 			${IMAGE.dir}/${.TARGET:S/smp_//}-${f}.img \
 		&& ${TOOL_GZIP} -f ${IMAGE.dir}/${.TARGET:S/smp_//}-${f}.img \
+		&& ${MKHASHES} -c ${CKSUM} ${IMAGE.dir}/${.TARGET:S/smp_//}-${f}.img.gz \
 		|| rm ${IMAGE.dir}/${.TARGET:S/smp_//}-${f}.img
 .endfor
 	@echo "===> Compressing image ${IMAGE.dir}/${.TARGET:S/smp_//}.img"
 	${TOOL_GZIP} -f ${GZIP_N_FLAG} -9 ${IMAGE.dir}/${.TARGET:S/smp_//}.img
+	${MKHASHES} -c ${CKSUM} ${IMAGE.dir}/${.TARGET:S/smp_//}.img.gz 
 
 ARCHES.earmv4=		armv4
 ARCHES.earmv4eb=	armv4
Index: etc/etc.evbmips/Makefile.inc
===================================================================
RCS file: /cvsroot/src/etc/etc.evbmips/Makefile.inc,v
retrieving revision 1.32
diff -u -r1.32 Makefile.inc
--- etc/etc.evbmips/Makefile.inc	22 Mar 2025 09:25:14 -0000	1.32
+++ etc/etc.evbmips/Makefile.inc	7 Jul 2026 16:36:35 -0000
@@ -52,6 +52,9 @@
 IMAGEENDIAN=	le
 .endif
 
+# Support for mkhashes
+MKHASHES= 	${NETBSDSRCDIR}/distrib/utils/embedded/mkhashes
+
 __mkimage: .USE
 	TOOL_MAKE=${MAKE} \
 	TOOL_MAKEFS=${TOOL_MAKEFS} \
@@ -70,11 +73,10 @@
 	MACHINE=${MACHINE} \
 	MKDTB=${MKDTB} \
 	MKPOSTFIX=${MKPOSTFIX} \
-	CKSUM=${CKSUM} \
-	IMAGEDIR=${IMAGE.dir} \
 	${HOST_SH} ${MKIMAGE} -h ${.TARGET:S/smp_//} -D ${DESTDIR} \
             -S ${NETBSDSRCDIR} -B ${IMAGEENDIAN} ${MKI_OPTS.${.TARGET}} \
             ${IMAGE.dir}/${.TARGET:S/smp_//}.img.gz
+	${MKHASHES} -c ${CKSUM} ${IMAGE.dir}/${.TARGET:S/smp_//}.img.gz
 
 .if !empty(MACHINE_ARCH:Mmips*64eb)
 smp_octeon: __mkimage
Index: etc/etc.evbppc/Makefile.inc
===================================================================
RCS file: /cvsroot/src/etc/etc.evbppc/Makefile.inc,v
retrieving revision 1.22
diff -u -r1.22 Makefile.inc
--- etc/etc.evbppc/Makefile.inc	10 Jan 2026 06:56:58 -0000	1.22
+++ etc/etc.evbppc/Makefile.inc	7 Jul 2026 16:36:35 -0000
@@ -39,6 +39,9 @@
 IMAGE.kern=     ${IMAGE.rel}/binary/kernel
 IMAGEENDIAN=	be
 
+# Support for mkhashes
+MKHASHES= 	${NETBSDSRCDIR}/distrib/utils/embedded/mkhashes
+
 .if ${MACHINE_ARCH} == "powerpc"
 __mkimage: .USE
 	TOOL_MAKE=${MAKE} \
@@ -58,11 +61,10 @@
 	MACHINE=${MACHINE} \
 	MKDTB=${MKDTB} \
 	MKPOSTFIX=${MKPOSTFIX} \
-	CKSUM=${CKSUM} \
-	IMAGEDIR=${IMAGE.dir} \
 	${HOST_SH} -x ${MKIMAGE} -h ${.TARGET:S/smp_//} -D ${DESTDIR} \
             -S ${NETBSDSRCDIR} -B ${IMAGEENDIAN} ${MKI_OPTS.${.TARGET}} \
             ${IMAGE.dir}/${.TARGET:S/smp_//}.img.gz
+	${MKHASHES} -c ${CKSUM} ${IMAGE.dir}/${.TARGET:S/smp_//}.img.gz
 
 smp_nintendo: __mkimage
 	cd ${KERNSRCDIR}/arch/evbppc/stand/wii && ${MAKE} release
Index: etc/etc.riscv/Makefile.inc
===================================================================
RCS file: /cvsroot/src/etc/etc.riscv/Makefile.inc,v
retrieving revision 1.7
diff -u -r1.7 Makefile.inc
--- etc/etc.riscv/Makefile.inc	22 Mar 2025 09:25:14 -0000	1.7
+++ etc/etc.riscv/Makefile.inc	7 Jul 2026 16:36:35 -0000
@@ -15,6 +15,9 @@
 IMAGE.dir=      ${IMAGE.rel}/binary/gzimg
 IMAGE.kern=     ${IMAGE.rel}/binary/kernel
 
+# Support for mkhashes
+MKHASHES= 	${NETBSDSRCDIR}/distrib/utils/embedded/mkhashes
+
 #	TOOL_MKUBOOTIMAGE=${TOOL_MKUBOOTIMAGE} \
 
 __mkimage: .USE
@@ -34,12 +37,10 @@
 	MACHINE=${MACHINE} \
 	MKDTB=${MKDTB} \
 	MKPOSTFIX=${MKPOSTFIX} \
-	CKSUM=${CKSUM} \
-	IMAGEDIR=${IMAGE.dir} \
 	${HOST_SH} ${MKIMAGE} -h ${.TARGET:S/smp_//} -D ${DESTDIR} \
             -S ${NETBSDSRCDIR} ${MKI_OPTS.${.TARGET}} \
             ${IMAGE.dir}/${.TARGET:S/smp_//}.img.gz
-
+	${MKHASHES} -c ${CKSUM} ${IMAGE.dir}/${.TARGET:S/smp_//}.img.gz
 
 .if ${MACHINE_ARCH} == "riscv64"
 .if ${HAVE_UEFI} == "yes"



#!/bin/sh
# $NetBSD$
#
# Copyright (c) 2026 The NetBSD Foundation, Inc.
# All rights reserved.
#
# This code is derived from software contributed to The NetBSD Foundation
# by Brook Milligan.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# 1. Redistributions of source code must retain the above copyright
#    notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
#    notice, this list of conditions and the following disclaimer in the
#    documentation and/or other materials provided with the distribution.
# 3. Neither the name of The NetBSD Foundation nor the names of its
#    contributors may be used to endorse or promote products derived
#    from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
# ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
#

#
# Makes a set of hashes for a file.
#

set -e

PROG="$(basename "$0")"

cksum=${CKSUM:-cksum}
hashes=${HASHES:-"MD5 SHA512"}
imagedir=${IMAGEDIR}

bar="==="

tmp="$(mktemp -d "${TMPDIR:-/tmp}/$PROG.XXXXXX")"

trap "cleanup" 0 1 2 3 15

cleanup() {
	case "$tmp" in
	"${TMPDIR:-/tmp}/$PROG."*)	rm -fr "$tmp";;
	esac
}

usage() {
	cat << EOF 1>&2
Usage: $PROG [-H <hashes>] [-c <cksum>] [-d <imagedir>] <image>

-H	Hash algorithms [${hashes}]
-c	Checksum program [${cksum}]
-d	Image directory [dirname \${image}]
EOF
	exit 1
}

OPTS="H:c:d:"
while getopts "$OPTS" f
do
	case $f in
	H)	hashes="$OPTARG";;
	c)	cksum="$OPTARG";;
	d)	imagedir="$OPTARG";;
	*)	usage;;
	esac
done

shift $(( $OPTIND - 1 ))
if [ -n "$1" ]; then
	# take the next argument as being the image name
	image="$1"
	shift
fi

if [ -z "${image}" ]; then
	echo "${PROG}: image undefined"
	usage
fi

if [ -z "${imagedir}" ]; then
	imagedir="$(dirname "${image}")"
fi

image="$(basename "${image}")"

cd "${imagedir}"

if [ ! -f ${image} ]; then
	echo "${PROG}: ${imagedir}/${image} does not exist"
	usage
fi

for h in ${hashes}; do
	echo "${bar} ${h} hash: ${imagedir}/${image} ${bar}"
	if [ -f ${h} ]; then
		grep -v "${image}" ${h} > ${tmp}/${h} || true
	else
		cp /dev/null ${tmp}/${h}
	fi
	${cksum} -a ${h} "${image}" >> ${tmp}/${h}
	mv ${tmp}/${h} ${h}
done
References:
- Re: gzimg hashes
  - From: David H. Gutteridge
- Re: gzimg hashes
  - From: Brook Milligan
Prev by Date: Re: ldd core files
Next by Date: mpt(4) integrated RAID event message improvements
Previous by Thread: Re: gzimg hashes
Next by Thread: ahcisata0: reset failed
Indexes:
Home | Main Index | Thread Index | Old Index