Re: Supporting compression-less release builds

To: tech-userlevel%netbsd.org@localhost
Subject: Re: Supporting compression-less release builds
From: Julio Merino <jmmv%NetBSD.org@localhost>
Date: Sat, 18 Jan 2025 16:40:10 +0000

On Sun, Jan 12, 2025 at 02:49:41PM +0000, Julio Merino wrote:
>
> Does anyone see a problem with this plan?

OK, I read through the comments in the sibling messages.

Unifying everything into one compression method might not be the best
idea, at least for now, because there might be reasons for kernels to be
gzipped only and because changing this would be too intrusive (requiring
changes to the docs and the like in some parameterized manner).

So let's take changes one at a time, and possibly keep the tunables for
sets / images / kernels separate. That said, I still think doing a code
cleanup to unify where flags are specified (so that compression levels
are consistently defined) is worthwhile.

I'm attaching just the first patch that I had in mind, which implements
the COMPRESS_SETS change only.  It works, but I haven't done sufficient
validation yet.

What do you think?

-- 
Julio Merino

Index: src.cvs/BUILDING
===================================================================
--- src.cvs.orig/BUILDING
+++ src.cvs/BUILDING
@@ -249,34 +249,35 @@ CONFIGURATION
      The supported mk.conf(5) make variables are:
 
            BSDOBJDIR, BSDSRCDIR, BUILD, BUILDID, BUILDINFO, BUILDSEED,
-           CDEXTRA, CONFIGOPTS, COPTS, CPUFLAGS, DESTDIR, EXTERNAL_TOOLCHAIN,
-           INSTALLBOOT_BOARDS, INSTALLWORLDDIR, KERNARCHDIR, KERNCONFDIR,
-           KERNEL_DIR, KERNOBJDIR, KERNSRCDIR, LOCALTIME, MAKEVERBOSE,
-           MKAMDGPUFIRMWARE, MKARGON2, MKARZERO, MKATF, MKBINUTILS, MKBSDGREP,
-           MKBSDTAR, MKCATPAGES, MKCLEANSRC, MKCLEANVERIFY, MKCOMPAT,
-           MKCOMPATMODULES, MKCOMPATTESTS, MKCOMPATX11, MKCOMPLEX, MKCROSSGDB,
-           MKCTF, MKCVS, MKCXX, MKDEBUG, MKDEBUGKERNEL, MKDEBUGLIB,
-           MKDEBUGTOOLS, MKDEPINCLUDES, MKDOC, MKDTB, MKDTC, MKDTRACE,
-           MKDYNAMICROOT, MKFIRMWARE, MKGCC, MKGCCCMDS, MKGDB, MKGROFF,
-           MKGROFFHTMLDOC, MKHESIOD, MKHOSTOBJ, MKHTML, MKIEEEFP, MKINET6,
-           MKINFO, MKIPFILTER, MKISCSI, MKKERBEROS, MKKMOD, MKKYUA, MKLDAP,
-           MKLIBCSANITIZER, MKLIBCXX, MKLIBSTDCXX, MKLINKLIB, MKLINT, MKLLVM,
-           MKLLVMRT, MKLVM, MKMAKEMANDB, MKMAN, MKMANDOC, MKMANZ, MKMDNS,
-           MKNLS, MKNOUVEAUFIRMWARE, MKNPF, MKNSD, MKOBJ, MKOBJDIRS, MKPAM,
-           MKPCC, MKPF, MKPIC, MKPICINSTALL, MKPICLIB, MKPIE, MKPIGZGZIP,
-           MKPOSTFIX, MKPROFILE, MKRADEONFIRMWARE, MKRELRO, MKREPRO,
-           MKREPRO_TIMESTAMP, MKRUMP, MKSANITIZER, MKSHARE, MKSKEY, MKSLJIT,
-           MKSOFTFLOAT, MKSTATICLIB, MKSTATICPIE, MKSTRIPIDENT, MKSTRIPSYM,
-           MKTEGRAFIRMWARE, MKTPM, MKUNBOUND, MKUNPRIVED, MKUPDATE, MKX11,
-           MKX11FONTS, MKX11MOTIF, MKXORG_SERVER, MKYP, MKZFS, NETBSDSRCDIR,
-           NETBSD_OFFICIAL_RELEASE, NOCLEANDIR, NODISTRIBDIRS, NOINCLUDES,
-           OBJMACHINE, RELEASEDIR, RUMPUSER_THREADS, RUMP_CURLWP, RUMP_DEBUG,
-           RUMP_DIAGNOSTIC, RUMP_KTRACE, RUMP_LOCKDEBUG, RUMP_LOCKS_UP,
-           RUMP_NBCOMPAT, RUMP_VIRTIF, RUMP_VNODE_LOCKDEBUG,
-           TOOLCHAIN_MISSING, TOOLDIR, USETOOLS, USE_FORT, USE_HESIOD,
-           USE_INET6, USE_JEMALLOC, USE_KERBEROS, USE_LDAP, USE_LIBCSANITIZER,
-           USE_PAM, USE_PIGZGZIP, USE_SANITIZER, USE_SKEY, USE_SSP,
-           USE_XZ_SETS, USE_YP, X11MOTIFPATH, X11SRCDIR.
+           CDEXTRA, COMPRESS_SETS, CONFIGOPTS, COPTS, CPUFLAGS, DESTDIR,
+           EXTERNAL_TOOLCHAIN, INSTALLBOOT_BOARDS, INSTALLWORLDDIR,
+           KERNARCHDIR, KERNCONFDIR, KERNEL_DIR, KERNOBJDIR, KERNSRCDIR,
+           LOCALTIME, MAKEVERBOSE, MKAMDGPUFIRMWARE, MKARGON2, MKARZERO,
+           MKATF, MKBINUTILS, MKBSDGREP, MKBSDTAR, MKCATPAGES, MKCLEANSRC,
+           MKCLEANVERIFY, MKCOMPAT, MKCOMPATMODULES, MKCOMPATTESTS,
+           MKCOMPATX11, MKCOMPLEX, MKCROSSGDB, MKCTF, MKCVS, MKCXX, MKDEBUG,
+           MKDEBUGKERNEL, MKDEBUGLIB, MKDEBUGTOOLS, MKDEPINCLUDES, MKDOC,
+           MKDTB, MKDTC, MKDTRACE, MKDYNAMICROOT, MKFIRMWARE, MKGCC,
+           MKGCCCMDS, MKGDB, MKGROFF, MKGROFFHTMLDOC, MKHESIOD, MKHOSTOBJ,
+           MKHTML, MKIEEEFP, MKINET6, MKINFO, MKIPFILTER, MKISCSI, MKKERBEROS,
+           MKKMOD, MKKYUA, MKLDAP, MKLIBCSANITIZER, MKLIBCXX, MKLIBSTDCXX,
+           MKLINKLIB, MKLINT, MKLLVM, MKLLVMRT, MKLVM, MKMAKEMANDB, MKMAN,
+           MKMANDOC, MKMANZ, MKMDNS, MKNLS, MKNOUVEAUFIRMWARE, MKNPF, MKNSD,
+           MKOBJ, MKOBJDIRS, MKPAM, MKPCC, MKPF, MKPIC, MKPICINSTALL,
+           MKPICLIB, MKPIE, MKPIGZGZIP, MKPOSTFIX, MKPROFILE,
+           MKRADEONFIRMWARE, MKRELRO, MKREPRO, MKREPRO_TIMESTAMP, MKRUMP,
+           MKSANITIZER, MKSHARE, MKSKEY, MKSLJIT, MKSOFTFLOAT, MKSTATICLIB,
+           MKSTATICPIE, MKSTRIPIDENT, MKSTRIPSYM, MKTEGRAFIRMWARE, MKTPM,
+           MKUNBOUND, MKUNPRIVED, MKUPDATE, MKX11, MKX11FONTS, MKX11MOTIF,
+           MKXORG_SERVER, MKYP, MKZFS, NETBSDSRCDIR, NETBSD_OFFICIAL_RELEASE,
+           NOCLEANDIR, NODISTRIBDIRS, NOINCLUDES, OBJMACHINE, RELEASEDIR,
+           RUMPUSER_THREADS, RUMP_CURLWP, RUMP_DEBUG, RUMP_DIAGNOSTIC,
+           RUMP_KTRACE, RUMP_LOCKDEBUG, RUMP_LOCKS_UP, RUMP_NBCOMPAT,
+           RUMP_VIRTIF, RUMP_VNODE_LOCKDEBUG, TOOLCHAIN_MISSING, TOOLDIR,
+           USETOOLS, USE_FORT, USE_HESIOD, USE_INET6, USE_JEMALLOC,
+           USE_KERBEROS, USE_LDAP, USE_LIBCSANITIZER, USE_PAM, USE_PIGZGZIP,
+           USE_SANITIZER, USE_SKEY, USE_SSP, USE_XZ_SETS, USE_YP,
+           X11MOTIFPATH, X11SRCDIR.
 
      The obsolete mk.conf(5) make variables are:
 
@@ -880,4 +881,4 @@ CAVEATS
      in object directories.  Instead, one may have to manually remove the
      files.  Consult the UPDATING file for notices concerning this.
 
-NetBSD                           July 21, 2023                          NetBSD
+NetBSD                         January 11, 2025                         NetBSD
Index: src.cvs/distrib/amd64/installimage/installimage.mk
===================================================================
--- src.cvs.orig/distrib/amd64/installimage/installimage.mk
+++ src.cvs/distrib/amd64/installimage/installimage.mk
@@ -3,10 +3,14 @@
 # common code between distrib/amd64/installimage/Makefile and
 # distrib/amd64/installimage-bios/Makefile.
 
-.if ${USE_XZ_SETS:Uno} != "no"
+.if ${COMPRESS_SETS} == "no"
+INSTIMAGEMB?=	6000			# for all installation binaries
+.elif ${COMPRESS_SETS} == "gzip"
 INSTIMAGEMB?=	2500			# for all installation binaries
-.else
+.elif ${COMPRESS_SETS} == "xz"
 INSTIMAGEMB?=	3800			# for all installation binaries
+.else
+.error "Unsupported COMPRESS_SETS=${COMPRESS_SETS}"
 .endif
 
 PRIMARY_BOOT=		bootxx_ffsv1
Index: src.cvs/distrib/common/bootimage/Makefile.bootimage
===================================================================
--- src.cvs.orig/distrib/common/bootimage/Makefile.bootimage
+++ src.cvs/distrib/common/bootimage/Makefile.bootimage
@@ -279,8 +279,6 @@ WORKFSTAB?=	work.fstab
 WORKRCCONF?=	work.rc.conf
 WORKFS?=        work.rootfs
 TARGETFS?=      imgroot.fs
-COMPRESS_PROGRAM=${"${USE_XZ_SETS:Uno}"!="no":?${TOOL_XZ}:${TOOL_GZIP}}
-TAR_SUFF=	${"${USE_XZ_SETS:Uno}"!="no":?tar.xz:tgz}
 
 CLEANFILES+=	${WORKSPEC} ${WORKFSTAB} ${WORKRCCONF} ${WORKFS}
 CLEANFILES+=	${TARGETFS}
@@ -363,14 +361,14 @@ ${TARGETFS}: prepare_md_post ${WORKFSTAB
 	fi;
 	@${MKDIR} ${MKDIRPERM} ${WORKDIR}
 .for set in ${IMG_SETS}
-	@if [ ! -f ${SETS_DIR}/${set}.${TAR_SUFF} ]; then 		\
-		echo "Missing ${SETS_DIR}/${set}.${TAR_SUFF}, aborting";\
+	@if [ ! -f ${SETS_DIR}/${set}.${SETS_TAR_SUFF} ]; then 		\
+		echo "Missing ${SETS_DIR}/${set}.${SETS_TAR_SUFF}, aborting";\
 		false; 							\
 	fi
-	@echo Extracting ${set}.${TAR_SUFF} ...
+	@echo Extracting ${set}.${SETS_TAR_SUFF} ...
 	@(cd ${WORKDIR}; ${TOOL_PAX} ${PAX_TIMESTAMP} -rn \
-		--use-compress-program=${COMPRESS_PROGRAM:Q} \
-		-f ${SETS_DIR}/${set}.${TAR_SUFF} .)
+		${SETS_COMPRESS_PROGRAM_FLAG:Q} \
+		-f ${SETS_DIR}/${set}.${SETS_TAR_SUFF} .)
 .endfor
 .if defined(SECONDARY_BOOT)
 	@echo Copying secondary boot...
Index: src.cvs/distrib/notes/Makefile.inc
===================================================================
--- src.cvs.orig/distrib/notes/Makefile.inc
+++ src.cvs/distrib/notes/Makefile.inc
@@ -34,13 +34,16 @@ whatis: .OPTIONAL
 
 PRESET=	${GFLAGS} -U -dM=${M} -dV=${DISTRIBVER} -d.CURDIR=${.CURDIR} -r${M}=1
 PRESET+=	-rmajor=${DISTRIBVER:C/\..*$//}
-.if ${USE_XZ_SETS:Uno} == "yes"
-PRESET+=	-dsetsuffix=tar.xz
+PRESET+=	-dsetsuffix=${SETS_TAR_SUFF}
+.if ${COMPRESS_SETS} == "no"
+.elif ${COMPRESS_SETS} == "gzip"
+PRESET+=	-dcompressor=gzip
+.elif ${COMPRESS_SETS} == "xz"
 PRESET+=	-dcompressor=xz
 .else
-PRESET+=	-dsetsuffix=tgz
-PRESET+=	-dcompressor=gzip
+.error "Unsupported COMPRESS_SETS=${COMPRESS_SETS}"
 .endif
+
 .if ${NETBSD_OFFICIAL_RELEASE:Uno} == "yes"
 PRESET+=	-rFOR_RELEASE=1
 .else
Index: src.cvs/distrib/sets/Makefile
===================================================================
--- src.cvs.orig/distrib/sets/Makefile
+++ src.cvs/distrib/sets/Makefile
@@ -15,8 +15,6 @@ all:
 .include <bsd.own.mk>
 .include "${NETBSDSRCDIR}/distrib/common/Makefile.distrib"
 
-COMPRESS_PROGRAM=${"${USE_XZ_SETS:Uno}"!="no":?${TOOL_XZ}:${TOOL_GZIP}}
-TAR_SUFF=${"${USE_XZ_SETS:Uno}"!="no":?tar.xz:tgz}
 XZ_OPT?=		-9
 GZIP_FLAGS=	${GZIP_N_FLAG}
 UMASK=		umask 022
@@ -33,10 +31,10 @@ SETSENV=	DESTDIR=${DESTDIR:Q} \
 		MKTEMP=${TOOL_MKTEMP:Q} \
 		MTREE=${TOOL_MTREE:Q} \
 		PAX=${TOOL_PAX:Q} \
-		COMPRESS_PROGRAM=${COMPRESS_PROGRAM:Q} \
+		COMPRESS_PROGRAM_FLAG=${SETS_COMPRESS_PROGRAM_FLAG:Q} \
 		GZIP=${GZIP_FLAGS:Q} \
 		XZ_OPT=${XZ_OPT:Q} \
-		TAR_SUFF=${TAR_SUFF:Q} \
+		TAR_SUFF=${SETS_TAR_SUFF:Q} \
 		PKG_CREATE=${TOOL_PKG_CREATE:Q} \
 		SED=${TOOL_SED:Q} \
 		TSORT=${TSORT:Q}
@@ -235,11 +233,11 @@ makesetfiles: .PHONY sanitise_METALOG
 
 .for tar in ${MAKETARSETS}
 do-${tar}: .PHONY sanitise_METALOG
-	${_MKMSG_CREATE} "${tar}.${TAR_SUFF}"
+	${_MKMSG_CREATE} "${tar}.${SETS_TAR_SUFF}"
 	${SETSCMD} ./maketars -d ${DESTDIR:S,^$,/,} ${MAKETARS_FLAGS} \
 	    ${METALOG.unpriv} \
 	    -N ${NETBSDSRCDIR}/etc -t ${TARDIR} ${tar} \
-	|| { rm -f ${TARDIR}/${tar}.${TAR_SUFF}; false; }
+	|| { rm -f ${TARDIR}/${tar}.${SETS_TAR_SUFF}; false; }
 .endfor
 
 
@@ -260,7 +258,7 @@ makesums: .PRECIOUS .PHONY check_RELEASE
 .for tar in ${MAKETARSETS}
 do-sum-${tar}: .PHONY do-${tar}
 	${_MKMSG_CREATE} "${tar} checksums"
-	${MAKESUMS} -t ${TARDIR} ${tar}.${TAR_SUFF}
+	${MAKESUMS} -t ${TARDIR} ${tar}.${SETS_TAR_SUFF}
 	${UMASK} && for i in MD5 SHA512; do \
 		${TOOL_CAT} ${TARDIR}/$$i >> ${TARDIR}/$$i.tmp; \
 	done
Index: src.cvs/distrib/sets/maketars
===================================================================
--- src.cvs.orig/distrib/sets/maketars
+++ src.cvs/distrib/sets/maketars
@@ -240,8 +240,8 @@ for setname in ${lists:-${nlists}}; do
 		msg "Creating ${out}"
 		rm -f "${tars}/${out}"
 		TMPFILES="${TMPFILES} ${tars}/${out}.tmp"
-		runpax "${setname}" -z --use-compress-program \
-		    ${COMPRESS_PROGRAM} > "${tars}/${out}.tmp" &&
+		runpax "${setname}" \
+		    ${COMPRESS_PROGRAM_FLAG} > "${tars}/${out}.tmp" &&
 		mv "${tars}/${out}.tmp" "${tars}/${out}"
 	fi
 	es=$((${es} + $?))
Index: src.cvs/distrib/utils/x_gzip/Makefile
===================================================================
--- src.cvs.orig/distrib/utils/x_gzip/Makefile
+++ src.cvs/distrib/utils/x_gzip/Makefile
@@ -19,7 +19,7 @@ CPPFLAGS+=	-DNO_LZ_SUPPORT
 DPADD=		${LIBZ}
 LDADD=		-lz
 
-.if ${USE_XZ_SETS:Uno} != "no"
+.if ${COMPRESS_SETS} == "xz"
 DPADD+=		${LIBLZMA}
 LDADD+=		-llzma
 .else
Index: src.cvs/doc/BUILDING.mdoc
===================================================================
--- src.cvs.orig/doc/BUILDING.mdoc
+++ src.cvs/doc/BUILDING.mdoc
@@ -33,7 +33,7 @@
 .\" Toolchain prefix for commands
 .ds toolprefix nb
 .
-.Dd July 21, 2023
+.Dd January 11, 2025
 .Dt BUILDING 8
 .Os NetBSD
 .\" turn off hyphenation
@@ -569,6 +569,7 @@ make variables are:
 .Sy BUILDINFO ,
 .Sy BUILDSEED ,
 .Sy CDEXTRA ,
+.Sy COMPRESS_SETS ,
 .Sy CONFIGOPTS ,
 .Sy COPTS ,
 .Sy CPUFLAGS ,
Index: src.cvs/etc/Makefile
===================================================================
--- src.cvs.orig/etc/Makefile
+++ src.cvs/etc/Makefile
@@ -410,9 +410,7 @@ install-obsolete-lists: .PHONY .MAKE
 distrib-dirs: .PHONY check_DESTDIR
 	cd ${NETBSDSRCDIR}/etc/mtree && ${MAKE} distrib-dirs
 
-COMPRESS_PROGRAM=${"${USE_XZ_SETS:Uno}"!="no":?${TOOL_XZ}:${TOOL_GZIP}}
 XZ_OPT?=-9
-TAR_SUFF=${"${USE_XZ_SETS:Uno}"!="no":?tar.xz:tgz}
 
 # release, snapshot --
 #	Build a full distribution including kernels & install media.
@@ -420,7 +418,7 @@ TAR_SUFF=${"${USE_XZ_SETS:Uno}"!="no":?t
 release snapshot: .PHONY .MAKE check_DESTDIR check_RELEASEDIR snap_md_post
 	${MAKEDIRTARGET} ${NETBSDSRCDIR}/distrib/sets sets
 	${MAKESUMS} -A -t ${RELEASEDIR}/${RELEASEMACHINEDIR}/binary/sets \
-		${KERNEL_SETS:@.SETS.@kern-${.SETS.}.${TAR_SUFF}@}
+		${KERNEL_SETS:@.SETS.@kern-${.SETS.}.${SETS_TAR_SUFF}@}
 	${MAKESUMS} -t ${RELEASEDIR}/${RELEASEMACHINEDIR}/binary/kernel '*.gz'
 
 
@@ -601,8 +599,8 @@ kernset-${configfile}: .PHONY build_kern
 	@kernlist=$$(${GETKERNELAWK} ${KERNCONFDIR}/${configfile}); \
 	kerndir=${KERNOBJDIR}/${configfile:C/.*\///}; \
 	kernsuffixes="${KERNEL_SUFFIXES:S/^/./}"; \
-	kern_tgz=${RELEASEDIR}/${RELEASEMACHINEDIR}/binary/sets/kern-${configfile}.${TAR_SUFF}; \
-	pax_cmd="GZIP=${GZIP_FLAGS:Q} XZ_OPT=${XZ_OPT:Q} ${TOOL_PAX} ${PAX_TIMESTAMP} --use-compress-program ${COMPRESS_PROGRAM:Q} -O -w -M -N ${NETBSDSRCDIR}/etc -f $${kern_tgz}"; \
+	kern_tgz=${RELEASEDIR}/${RELEASEMACHINEDIR}/binary/sets/kern-${configfile}.${SETS_TAR_SUFF}; \
+	pax_cmd="GZIP=${GZIP_FLAGS:Q} XZ_OPT=${XZ_OPT:Q} ${TOOL_PAX} ${PAX_TIMESTAMP} ${SETS_COMPRESS_PROGRAM_FLAG:Q} -O -w -M -N ${NETBSDSRCDIR}/etc -f $${kern_tgz}"; \
 	umask 022 && \
 	cd $${kerndir} && { \
 		kernels=; newest=; \
Index: src.cvs/share/man/man5/mk.conf.5
===================================================================
--- src.cvs.orig/share/man/man5/mk.conf.5
+++ src.cvs/share/man/man5/mk.conf.5
@@ -27,7 +27,7 @@
 .\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
 .\" POSSIBILITY OF SUCH DAMAGE.
 .\"
-.Dd September 7, 2024
+.Dd January 11, 2025
 .Dt MK.CONF 5
 .Os
 .\" turn off hyphenation
@@ -203,6 +203,28 @@ options, or via a single option whose ar
 space-separated paths.
 .DFLTu
 .
+.It Sy COMPRESS_SETS
+Indicates whether the distribution tar files are to be compressed or not,
+and in which format.
+Can be one of
+.Sy no ,
+.Sy gzip
+or
+.Sy xz .
+Forced to
+.Sy gzip
+if
+.Sy USE_PIGZGZIP=yes .
+.DFLT
+.Dq xz
+on
+.Sy aarch64 ,
+.Sy amd64 ,
+and
+.Sy sparc64 ,
+.Dq gzip
+on other platforms.
+.
 .It Sy CONFIGOPTS
 Additional options to
 .Xr config 1
@@ -2364,26 +2386,6 @@ on other platforms if
 otherwise
 .Dq no .
 .
-.It Sy USE_XZ_SETS
-.YorN
-Indicates whether the distribution tar files are to be compressed
-with
-.Xr xz 1
-instead of
-.Xr gzip 1
-or
-.Xr pigz 1 .
-.NOVAR USE_PIGZGZIP=yes
-.DFLT
-.Dq yes
-on
-.Sy aarch64 ,
-.Sy amd64 ,
-and
-.Sy sparc64 ,
-.Dq no
-on other platforms.
-.
 .It Sy USE_YP
 .YorN
 Indicates whether YP (NIS) support is
@@ -2482,6 +2484,10 @@ Obsolete.
 The new toolchain is now the default.
 To disable, use
 .Sy TOOLCHAIN_MISSING=yes .
+.It Sy USE_XZ_SETS
+Obsolete.
+Use
+.Sy COMPRESS_SETS=xz .
 .
 .El
 .
Index: src.cvs/share/mk/bsd.own.mk
===================================================================
--- src.cvs.orig/share/mk/bsd.own.mk
+++ src.cvs/share/mk/bsd.own.mk
@@ -1670,16 +1670,16 @@ ${var}?= yes
 ${var}?= no
 .endfor
 
-# Default to USE_XZ_SETS on some 64bit architectures where decompressor
+# Default to xz-compressed sets on some 64bit architectures where decompressor
 # memory will likely not be in short supply.
 # Since pigz can not create .xz format files currently, disable .xz
 # format if USE_PIGZGZIP is enabled.
 .if ${USE_PIGZGZIP} == "no" && \
     (${MACHINE} == "amd64" || \
      ${MACHINE_ARCH:Maarch64*})
-USE_XZ_SETS?= yes
+COMPRESS_SETS?= xz
 .else
-USE_XZ_SETS?= no
+COMPRESS_SETS?= gzip
 .endif
 
 #
@@ -1699,6 +1699,22 @@ GZIP_N_FLAG?=		-n
 TOOL_GZIP_N=		${TOOL_GZIP} ${GZIP_N_FLAG}
 
 #
+# Set common settings to handle compress sets.
+#
+.if ${COMPRESS_SETS} == "no"
+SETS_TAR_SUFF=tar
+SETS_COMPRESS_PROGRAM_FLAG=
+.elif ${COMPRESS_SETS} == "gzip"
+SETS_COMPRESS_PROGRAM_FLAG=--use-compress-program=${TOOL_GZIP}
+SETS_TAR_SUFF=tgz
+.elif ${COMPRESS_SETS} == "xz"
+SETS_COMPRESS_PROGRAM_FLAG=--use-compress-program=${TOOL_XZ}
+SETS_TAR_SUFF=tar.xz
+.else
+.error "Unsupported COMPRESS_SETS=${COMPRESS_SETS}"
+.endif
+
+#
 # Where X11 sources are and where it is installed to.
 #
 .if !defined(X11SRCDIR)
Index: src.cvs/share/mk/bsd.README
===================================================================
--- src.cvs.orig/share/mk/bsd.README
+++ src.cvs/share/mk/bsd.README
@@ -108,40 +108,40 @@ the make(1) command line.
 The supported mk.conf(5) make variables are:
 
 	BSDOBJDIR, BSDSRCDIR, BUILD, BUILDID, BUILDINFO, BUILDSEED,
-	CDEXTRA, CONFIGOPTS, COPTS, CPUFLAGS, DESTDIR, EXTERNAL_TOOLCHAIN,
-	INSTALLBOOT_BOARDS, INSTALLWORLDDIR, KERNARCHDIR, KERNCONFDIR,
-	KERNEL_DIR, KERNOBJDIR, KERNSRCDIR, LOCALTIME, MAKEVERBOSE,
-	MKAMDGPUFIRMWARE, MKARGON2, MKARZERO, MKATF, MKBINUTILS, MKBSDGREP,
-	MKBSDTAR, MKCATPAGES, MKCLEANSRC, MKCLEANVERIFY, MKCOMPAT,
-	MKCOMPATMODULES, MKCOMPATTESTS, MKCOMPATX11, MKCOMPLEX, MKCROSSGDB,
-	MKCTF, MKCVS, MKCXX, MKDEBUG, MKDEBUGKERNEL, MKDEBUGLIB,
-	MKDEBUGTOOLS, MKDEPINCLUDES, MKDOC, MKDTB, MKDTC, MKDTRACE,
-	MKDYNAMICROOT, MKFIRMWARE, MKGCC, MKGCCCMDS, MKGDB, MKGROFF,
-	MKGROFFHTMLDOC, MKHESIOD, MKHOSTOBJ, MKHTML, MKIEEEFP, MKINET6,
-	MKINFO, MKIPFILTER, MKISCSI, MKKERBEROS, MKKMOD, MKKYUA, MKLDAP,
-	MKLIBCSANITIZER, MKLIBCXX, MKLIBSTDCXX, MKLINKLIB, MKLINT, MKLLVM,
-	MKLLVMRT, MKLVM, MKMAKEMANDB, MKMAN, MKMANDOC, MKMANZ, MKMDNS,
-	MKNLS, MKNOUVEAUFIRMWARE, MKNPF, MKNSD, MKOBJ, MKOBJDIRS, MKPAM,
-	MKPCC, MKPF, MKPIC, MKPICINSTALL, MKPICLIB, MKPIE, MKPIGZGZIP,
-	MKPOSTFIX, MKPROFILE, MKRADEONFIRMWARE, MKRELRO, MKREPRO,
-	MKREPRO_TIMESTAMP, MKRUMP, MKSANITIZER, MKSHARE, MKSKEY, MKSLJIT,
-	MKSOFTFLOAT, MKSTATICLIB, MKSTATICPIE, MKSTRIPIDENT, MKSTRIPSYM,
-	MKTEGRAFIRMWARE, MKTPM, MKUNBOUND, MKUNPRIVED, MKUPDATE, MKX11,
-	MKX11FONTS, MKX11MOTIF, MKXORG_SERVER, MKYP, MKZFS, NETBSDSRCDIR,
-	NETBSD_OFFICIAL_RELEASE, NOCLEANDIR, NODISTRIBDIRS, NOINCLUDES,
-	OBJMACHINE, RELEASEDIR, RUMPUSER_THREADS, RUMP_CURLWP, RUMP_DEBUG,
-	RUMP_DIAGNOSTIC, RUMP_KTRACE, RUMP_LOCKDEBUG, RUMP_LOCKS_UP,
-	RUMP_NBCOMPAT, RUMP_VIRTIF, RUMP_VNODE_LOCKDEBUG,
-	TOOLCHAIN_MISSING, TOOLDIR, USETOOLS, USE_FORT, USE_HESIOD,
-	USE_INET6, USE_JEMALLOC, USE_KERBEROS, USE_LDAP, USE_LIBCSANITIZER,
-	USE_PAM, USE_PIGZGZIP, USE_SANITIZER, USE_SKEY, USE_SSP,
-	USE_XZ_SETS, USE_YP, X11MOTIFPATH, X11SRCDIR.
+	CDEXTRA, COMPRESS_SETS, CONFIGOPTS, COPTS, CPUFLAGS, DESTDIR,
+	EXTERNAL_TOOLCHAIN, INSTALLBOOT_BOARDS, INSTALLWORLDDIR,
+	KERNARCHDIR, KERNCONFDIR, KERNEL_DIR, KERNOBJDIR, KERNSRCDIR,
+	LOCALTIME, MAKEVERBOSE, MKAMDGPUFIRMWARE, MKARGON2, MKARZERO,
+	MKATF, MKBINUTILS, MKBSDGREP, MKBSDTAR, MKCATPAGES, MKCLEANSRC,
+	MKCLEANVERIFY, MKCOMPAT, MKCOMPATMODULES, MKCOMPATTESTS,
+	MKCOMPATX11, MKCOMPLEX, MKCROSSGDB, MKCTF, MKCVS, MKCXX, MKDEBUG,
+	MKDEBUGKERNEL, MKDEBUGLIB, MKDEBUGTOOLS, MKDEPINCLUDES, MKDOC,
+	MKDTB, MKDTC, MKDTRACE, MKDYNAMICROOT, MKFIRMWARE, MKGCC,
+	MKGCCCMDS, MKGDB, MKGROFF, MKGROFFHTMLDOC, MKHESIOD, MKHOSTOBJ,
+	MKHTML, MKIEEEFP, MKINET6, MKINFO, MKIPFILTER, MKISCSI, MKKERBEROS,
+	MKKMOD, MKKYUA, MKLDAP, MKLIBCSANITIZER, MKLIBCXX, MKLIBSTDCXX,
+	MKLINKLIB, MKLINT, MKLLVM, MKLLVMRT, MKLVM, MKMAKEMANDB, MKMAN,
+	MKMANDOC, MKMANZ, MKMDNS, MKNLS, MKNOUVEAUFIRMWARE, MKNPF, MKNSD,
+	MKOBJ, MKOBJDIRS, MKPAM, MKPCC, MKPF, MKPIC, MKPICINSTALL,
+	MKPICLIB, MKPIE, MKPIGZGZIP, MKPOSTFIX, MKPROFILE,
+	MKRADEONFIRMWARE, MKRELRO, MKREPRO, MKREPRO_TIMESTAMP, MKRUMP,
+	MKSANITIZER, MKSHARE, MKSKEY, MKSLJIT, MKSOFTFLOAT, MKSTATICLIB,
+	MKSTATICPIE, MKSTRIPIDENT, MKSTRIPSYM, MKTEGRAFIRMWARE, MKTPM,
+	MKUNBOUND, MKUNPRIVED, MKUPDATE, MKX11, MKX11FONTS, MKX11MOTIF,
+	MKXORG_SERVER, MKYP, MKZFS, NETBSDSRCDIR, NETBSD_OFFICIAL_RELEASE,
+	NOCLEANDIR, NODISTRIBDIRS, NOINCLUDES, OBJMACHINE, RELEASEDIR,
+	RUMPUSER_THREADS, RUMP_CURLWP, RUMP_DEBUG, RUMP_DIAGNOSTIC,
+	RUMP_KTRACE, RUMP_LOCKDEBUG, RUMP_LOCKS_UP, RUMP_NBCOMPAT,
+	RUMP_VIRTIF, RUMP_VNODE_LOCKDEBUG, TOOLCHAIN_MISSING, TOOLDIR,
+	USETOOLS, USE_FORT, USE_HESIOD, USE_INET6, USE_JEMALLOC,
+	USE_KERBEROS, USE_LDAP, USE_LIBCSANITIZER, USE_PAM, USE_PIGZGZIP,
+	USE_SANITIZER, USE_SKEY, USE_SSP, USE_YP, X11MOTIFPATH, X11SRCDIR.
 
 The obsolete mk.conf(5) make variables are:
 
 	EXTSRCSRCDIR, MKBFD, MKCRYPTO, MKEXTSRC, MKKDEBUG, MKKERBEROS4,
 	MKLLD, MKLLDB, MKMCLINKER, MKPERFUSE, MKTOOLSDEBUG, NBUILDJOBS,
-	SHAREDSTRINGS, USE_COMBINE, USE_NEW_TOOLCHAIN.
+	SHAREDSTRINGS, USE_COMBINE, USE_NEW_TOOLCHAIN, USE_XZ_SETS.
 
 Notable variables documented in mk.conf(5) and duplicated here:
 
Index: src.cvs/usr.sbin/sysinst/Makefile.inc
===================================================================
--- src.cvs.orig/usr.sbin/sysinst/Makefile.inc
+++ src.cvs/usr.sbin/sysinst/Makefile.inc
@@ -71,8 +71,6 @@ UNIF_AWK=	${.CURDIR}/../../unif.awk
 MSG_XLAT_SH=	${.CURDIR}/../../msg_xlat.sh
 MSG_CMP_SH=	${.CURDIR}/../../msg_cmp.sh
 
-SETS_TAR_SUFF=${"${USE_XZ_SETS:Uno}"!="no":?"tar.xz":"tgz"}
-
 .if ${MKBSDTAR:Uno} == "no"
 CPPFLAGS+=	-DUSING_PAXASTAR
 .endif
@@ -106,13 +104,13 @@ CPPFLAGS+=	-DHAVE_MODULES
 CATALOGDIR=	/usr/share/sysinst/catalog
 CPPFLAGS+=	-I. -I${.CURDIR}/../.. -I${.CURDIR} \
 		-I${.CURDIR}/../../../../sbin/fsck \
-		-DSETS_TAR_SUFF=${SETS_TAR_SUFF:Q} \
+		-DSETS_TAR_SUFF=\"${SETS_TAR_SUFF:Q}\" \
 		-DREL=\"${DISTRIBVER}\" -DMACH=\"${MACHINE}\" \
 		-DMACH_${MACHINE} -DARCH_${MACHINE_ARCH} \
 		${NODISKLABEL:D-DNO_DISKLABEL} \
 		${NOPARTMAN:D-DNO_PARTMAN}
 
-.if ${USE_XZ_SETS:Uno} != "no"
+.if ${COMPRESS_SETS} == "xz"
 CPPFLAGS+=	-DEXTRACT_NEEDS_BIG_RAM
 .endif

References:
- Supporting compression-less release builds
  - From: Julio Merino

Prev by Date: Re: Supporting compression-less release builds
Next by Date: Re: Supporting compression-less release builds
Previous by Thread: Re: Supporting compression-less release builds
Next by Thread: [PATCH] Transitive LIBDPLIBS, PROGDPLIBS
Indexes:

Home | Main Index | Thread Index | Old Index