pkgsrc-Users archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: [changed subject to] pkgsrc in scientific computing

Am Tue, 07 Jul 2015 15:28:49 -0500
schrieb Jason Bacon <>:

> On our HPC clusters, I simply bootstrap a whole new tree about every 6 
> to 12 months to make newer software versions available.

Ah, so we're indeed on the same page there. We are deploying our first
big setup based on pkgsrc for common software, but always thinking
about other stuff on top.

One tricky thing is how to handle differing compilers, especially since
C++ and Fortran modules are not compatible between them. One solution
is simply not to use those and write wrappers over C in your own code,
but do you happen to deal with getting stuff like HDF5 from pkgsrc with
intel/pgi compilers?

The we got differing MPI implementations. Various commercial software
on top. We want to offer the whole deal and are having endless debates
on how to do it best. Perhaps we should at some point have a longer
discussion with you, too. Now, we're really busy getting a fresh system up
and running, of course with an elaborate structure of environment

> Older trees are 
> left in place so researchers can finish up projects using the same 
> version of a package, but eventually deprecated.

We will never delete user software for the lifetime of the system (unless
there is a _really_ nasty security risk from just having it around).
But well, we won't carry all old versions onto the next setup.

> I have a lot of scientific packages in wip and more coming, but too 
> little time to devote to it.

Ah, so you helped us getting some of tha geography stuff going? ;-)

Btw.: I wonder if it makes sense having thousands of TexLive packages
in pkgsrc. It's such a huge collection of packages that actually comes
with its own package manager. It lends itself well to installation in a
separate prefix anyway. In our world, there are various separate
packages in addition to pkgsrc anyway. Pkgsrc takes the place of the
normal GNU/Linux userspace, on top of which specialist software is

>  One of my colleagues here is learning to 
> package and may join pkgsrc-wip soon.

Yes, if this really works out for us in the long term, I might start
contributing packages, too. Though, there probably always will be
standalone packages we build in-house. I see the need for pkgsrc with
the wildly interdependent stuff.

>  There's a lot of work to be done 
> in categories like math and biology, though. Fortran support needs some 
> work as well.

Do you have R built with proper BLAS (perhaps even Intel MKL?), and
possibly MPI from pkgsrc? Folks are using this software more and more,
as the field of application of HPC clusters widens.

But, well, let's continue that on a separate thread perhaps, in some
weeks when I can breathe again (*preparing yet another compute node

Alrighty then,


Dr. Thomas Orgis
Universität Hamburg
RRZ / Zentrale Dienste / HPC
Schlüterstr. 70
20146 Hamburg
Tel.: 040/42838 8826
Fax: 040/428 38 6270

Attachment: smime.p7s
Description: S/MIME cryptographic signature

Home | Main Index | Thread Index | Old Index