Am Mon, 12 Mar 2018 21:13:21 -0400 schrieb Greg Troxel <gdt%lexort.com@localhost>: > I may be being pedantic, but if you have to recompile the ABI is not > actually the same. Same ABI means you can swap out the library and > it's always ok, by definition. Yes. There is the grey area of added API that is not part of the standard. If you only use BLAS+LAPACK symbols in your program, you can LD_PRELOAD any blas and lapack lib and the functions from that will override whatever is linked in and it should work. That is how I tested openblas vs. netlib for my other post. You are right, adding a function to openblas to tune the number of threads strictly breaks ABI. But if you limit the exported symbols to the official ones (which I think is what the Debian libblas.so and liblapack.so wrappers do), you get an interchangeable library with a fixed ABI. The only remaining complexities are around parallel BLAS and runtime interference. > As far as having -lblas hardcoded, there > really needs to be a scheme to use blas.pc which is brought into > visisbility by the bl3 file, and have that drive the names and paths. Hm, so you say that instead of hacking library flags into packages' build systems, we should hack .pc file usage into them? I guess --with-blas=$(pkgconfig --libs blas) is what you mean, then, instead of putting in a variable BLAS_LIBS or so? This also means we will have to add .pc files for implementations that do not ship one, but they are easy enough to write. That reminds me of my local scripts to generate .pc files for NetCDF and HDF5. I should push that upstream, and into pkgsrc before that, perhaps. > You say library names not conflicting, but I wonder if you can > assert that this is always true and always will be true for any pair > of blas-like things. forks tend to only sometimes change names. I am pretty much sure. You need to understand that BLAS implementations are not forks of Netlib in the usual sense. They often do include parts of Netlib code, yes, but they never try to replace the official Netlib. See the list Jason linked to: https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms#Implementations Each variant has its unique name. The closest are things like libcublas for the CUDA-optimized BLAS. Of course, there could be a fork of OpenBLAS in the future that also installs libopenblas, but that would be a bog-standard conflict with the original and we would not support concurrent installation. In case one BLAS package does install a libblas.so (usually a symlink), we can prevent/rename that in pkgsrc. We already excercise freedom in library naming. That can continue. > It also allows you to put in > /usr/pkg/fooblas/lib/pkgconfig/blas.pc and force that for every one, > and to have the logic to find blas use that, and patch all depending > packages to find the pc file. That might be a separate reason to have prefixes, for separate blas.pc files. But blas.pc is not established in use. People use -lblas / -lopenblas directly. This would be an intentional change in pkgsrc to patch packages to refer to some .pc file for the build. We might decide to use or not to use blas.pc in future, but I see that the namespacing decision needs to be done up front. As the library names do not conflict in the real world, I tend to leave it at not enforcing .pc files. With multiple openblas libraries from one build, we should decide if we shall provide multiple .pc files, too, as OpenBLAS does indeed ship one by default. > With the same symbols, you really cannot link different blas > implementations in the same program. So you are in trouble if there > are libraries that use blas, and other packages use those. Is that > going to be a problem? Yes, that is an issue. In practice, things will still work, as the symbols do the same operations at heart, but performance will be unpredictable for the user. > > Yes. This is the mess I want to avoid. I would like all pkgsrc > > packages using one BLAS, but still see value in providing the > > others to users to > > presumably you mean in any one consistent build. Yes, all things built inside pkgsrc shall use one BLAS by default, but users building their own software on top can choose their BLAS from the ones installed by pkgsrc (possibly getting an inconsistent binary as a result). > LD_PRELOAD only works if the ABI really is the same. If you linked with a BLAS that has additional functions, those functions will still be served by the correct lib, but the common symbols are overridden by LD_PRELOAD. You get such fun effect like happily calling openblas_set_num_threads(8), but still having serial operation as the preloaded BLAS routines are not affected by that. So … it _does_ work. It might be exactly what the user specifying LD_PRELOAD intended;-) > Well I meeant not bug that they exist at all, but bug that for a given > system there isn't a straightforward choice of a single right answer. Yes. There never will be a straightforward choice given the parallelization options at least. Well, unless there is a platform with one and only way of doing parallelization and no possible runtime conflict of approaches. > OK, but that sounds like an argument for choosing a BLAS tune to the > arch, system wide. For x86, it's a safe choice to have serial OpenBLAS as default. I intend to work towards that. There may be something better for a certain CPU model, though. Alrighty then, Thomas -- Dr. Thomas Orgis Universität Hamburg RRZ / Basis-Infrastruktur / HPC Schlüterstr. 70 20146 Hamburg Tel.: 040/42838 8826 Fax: 040/428 38 6270
Attachment:
smime.p7s
Description: S/MIME cryptographic signature