introduce mk/openmp.mk, common choice of pulling in paralle/openmp or not (lots of biology)

["Dr. Thomas Orgis" <thomas.orgis%uni-hamburg.de@localhost>]

Dear all,

as briefly discussed in a smaller circle, I added mk/openmp.mk
recently, which contains this logic:

.if !empty(PKGSRC_COMPILER:M*clang*)
.include "../../parallel/openmp/buildlink3.mk"
.endif

I added usage of this to a number of packges that either

a) contained that exact logic
b) contained an unconditional include of parallel/openmp bl3.

This should be uncontroversial: It just reduces code duplication and
prevents unnecessary builds of parallel/openmp on platforms that ship
the OpenMP library as part of GCC. I does not break anything that does
not use it, just reduces the occasions of unnecessary parallel/openmp
builds (which mostly will remain unused after being built as fake
dependency with gcc toolchain).

What remains are a number of packages that have specific code for
Darwin. We might want to clean that up and perhaps include some snippet
for that in openmp.mk — or is it already fine as Darwin compiler ==
clang? As I do not test that platform, I did not touch those
occurences. My application always comes with openmp in the gcc
toolchain.

There for sure are packges that do not have explicit OpenMP use yet,
where it is implicitly used. Those should be made explicit using the
common logic.

The remaining places of bare inclusion of paralle/openmp:

$ find . -type f |xargs grep -C 10 openmp/buildlink3.mk
[mk/openmp.mk of course]
./biology/rna-star/Makefile-# Xcode supports -Xpreprocessor -fopenmp, but does not supply OpenMP libs
./biology/rna-star/Makefile-# parallel/openmp installs an entire llvm
./biology/rna-star/Makefile-# Maybe a libomp package would suffice?
./biology/rna-star/Makefile-.if ${OPSYS} == Darwin
./biology/rna-star/Makefile-SUBST_CLASSES+=		openmp
./biology/rna-star/Makefile-SUBST_SED.openmp=	-e 's|-fopenmp|-Xpreprocessor &|g'
./biology/rna-star/Makefile-SUBST_STAGE.openmp=	pre-configure
./biology/rna-star/Makefile-SUBST_FILES.openmp=	Makefile
./biology/rna-star/Makefile-LDFLAGS+= 		-L${PREFIX}/lib -lomp -lm
./biology/rna-star/Makefile:.include "../../parallel/openmp/buildlink3.mk"
./biology/rna-star/Makefile-.endif

I wonder if some of that SUBST logic could be centralized, at least the
option names?

./biology/fasttree/Makefile-.include "../../mk/bsd.prefs.mk"
./biology/fasttree/Makefile-
./biology/fasttree/Makefile-# Xcode supports -Xpreprocessor -fopenmp, but does not supply OpenMP libs
./biology/fasttree/Makefile-.if ${OPSYS} == Darwin
./biology/fasttree/Makefile-SUBST_CLASSES+=		openmp
./biology/fasttree/Makefile-SUBST_SED.openmp=	-e 's|-fopenmp|-Xpreprocessor &|g'
./biology/fasttree/Makefile-SUBST_STAGE.openmp=	pre-configure
./biology/fasttree/Makefile-SUBST_FILES.openmp=	Makefile
./biology/fasttree/Makefile-LDFLAGS+= 		-L${PREFIX}/lib -lomp -lm
./biology/fasttree/Makefile:.include "../../parallel/openmp/buildlink3.mk"
./biology/fasttree/Makefile-.endif

This looks very similar to the above. Would like deduplicaton.

./biology/canu/Makefile-do-install:
./biology/canu/Makefile-	cd ${TMP_INST}/bin && ${PAX} -wr * ${DESTDIR}${PREFIX}/bin
./biology/canu/Makefile-	cd ${TMP_INST}/lib/perl5 && ${PAX} -wr * ${DESTDIR}${PREFIX}/lib/perl5
./biology/canu/Makefile-	cd ${TMP_INST}/share && ${PAX} -wr * ${DESTDIR}${PREFIX}/share
./biology/canu/Makefile-
./biology/canu/Makefile-# Xcode supports -Xpreprocessor -fopenmp, but does not supply OpenMP libs
./biology/canu/Makefile-# parallel/openmp installs an entire llvm
./biology/canu/Makefile-# Maybe a libomp package would suffice?
./biology/canu/Makefile-.if ${OPSYS} == Darwin
./biology/canu/Makefile:.include "../../parallel/openmp/buildlink3.mk"
./biology/canu/Makefile-.endif

Would that already be even more correct if checking for clang instead of Darwin?

./biology/stacks/Makefile-
./biology/stacks/Makefile-.if ${OPSYS} == Darwin
./biology/stacks/Makefile-# pkgsrc sed wrapper lacks -i
./biology/stacks/Makefile-SUBST_CLASSES+=		sed
./biology/stacks/Makefile-SUBST_STAGE.sed=	pre-configure
./biology/stacks/Makefile-SUBST_FILES.sed=	Makefile.in
./biology/stacks/Makefile-SUBST_SED.sed=		-e 's|sed -i.bkp|/usr/bin/sed -i .bkp|'
./biology/stacks/Makefile-CFLAGS+=		-Xpreprocessor -fopenmp
./biology/stacks/Makefile-
./biology/stacks/Makefile:.include "../../parallel/openmp/buildlink3.mk"
./biology/stacks/Makefile-.else
./biology/stacks/Makefile-# macOS clang lacks OpenMP
./biology/stacks/Makefile-CFLAGS+=		-fopenmp
./biology/stacks/Makefile-.endif

Here we got a variant where adding -fopenmp to CFLAGS seems to be
necessary. Projects I know do that themselves. But maybe openmp.mk
should provide OPENMP_CFLAGS to that effect?


And then we got a number of wip packages that are offenders …

./wip/m4ri/Makefile-CONFIGURE_ARGS+=	--enable-openmp
./wip/m4ri/Makefile-
./wip/m4ri/Makefile-
./wip/m4ri/Makefile:.include "../../parallel/openmp/buildlink3.mk"
./wip/m4ri/Makefile-.include "../../mk/bsd.pkg.mk"

./wip/m4ri/buildlink3.mk:.include "../../parallel/openmp/buildlink3.mk"


Oh. Also in buildlink3.mk … something to consider.

--
./wip/suitesparse/Makefile

[fixed in main, should be deleted]

./wip/efl/options.mk-# Use clang to build efl
./wip/efl/options.mk-.if !empty(PKG_OPTIONS:Mclang)
./wip/efl/options.mk:.include "../../parallel/openmp/buildlink3.mk"
./wip/efl/options.mk-.endif

./wip/efl/buildlink3.mk-.if ${PKG_BUILD_OPTIONS.efl:Mclang}
./wip/efl/buildlink3.mk:.include "../../parallel/openmp/buildlink3.mk"
./wip/efl/buildlink3.mk-.endif

That's another variant. I wonder if one could fold that into openmk.mk,
too. Perhaps with some variable to set for 'I was built with clang, use
that and not the current compiler'. Though mixing compilers and runtimes
like that might be funky. Wha

./wip/raze/Makefile:# .include "../../parallel/openmp/buildlink3.mk"

Somebody being not sure yet.

./wip/libflame_openmp/Makefile-# $NetBSD$
./wip/libflame_openmp/Makefile-LIBFLAME_VARIANT=libflame_openmp
./wip/libflame_openmp/Makefile-.include "../../wip/libflame/Makefile.common"
./wip/libflame_openmp/Makefile:.include "../../wip/blis_openmp/buildlink3.mk"
./wip/libflame_openmp/Makefile-.include "../../mk/bsd.pkg.mk"

Oh, someone is working on libflame … Matthew: I guess we should get in
touch regarding integrating that into the mk/blas.buildlink3.mk scheme,
in case you did not work on that already. Could be nice for AMD
systems. I wonder, though: Isn't there matching blis_openmpi?

The clang openmp dependency seems to be missing from the blis/flame
packages.

[skipping some biology packages also in base]

./wip/gzdoom/Makefile:# .include "../../parallel/openmp/buildlink3.mk"

Again someone who is unsure.

./wip/GenericRepeatFinder/Makefile:.include "../../parallel/openmp/buildlink3.mk"

Clear offender.


I think the fruit is low-hanging enough to stop the malpractice of
unconditional dependencies on parallel/openmp before the freeze, and
possibly to improve the logic for Darwin. I guess the number of
affected packages will grow when one looks more closely at various
(scientific) software. Off the top of my head I notice that the omp
option of math/fftw does not mention the openmp dependency for clang
toolchains at all. People might want to have that fixed.


Alrighty then,

Thomas

-- 
Dr. Thomas Orgis
HPC @ Universität Hamburg