tech-pkg archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: pbulk parallel builds



On 06/04/17 02:20, Jonathan Perkin wrote:
* On 2017-06-04 at 01:20 BST, Jason Bacon wrote:

Is it possible for pbulk to build multiple packages simultaneously
on the same machine?
Yes!  I've been doing this for many years but it requires a few
patches that aren't yet in pkgsrc.  I proposed a change that will
hopefully be accepted upstream for people to test a while back:

   https://mail-index.netbsd.org/pkgsrc-users/2017/02/14/msg024455.html

I still use my previous patches for all my bulk builds, but would
appreciate testing of the above so we can make this easier for
everyone.

Thanks, Jonathan! Unfortunately, I've had to move away from mksandbox. The problem is that on RHEL/CentOS, Yum installs things into /usr/bin, /usr/lib, etc., which has caused occasional leakage issues (pkgsrc picking up a tool or library installed by Yum instead of its own). If I'm going to utilize our cluster nodes, which have a lot of Yum packages installed to support commercial software, I have to use a clean chroot image from a fresh install rather than mksandbox, which imports much of the chroot env from the host.

My approach is documented at the bottom of this page:

http://uwm.edu/hpc/software-management/

Also, I don't see a need to distribute my builds across multiple systems, given that our compute nodes have 24 cores, a lot of RAM, and fast RAIDs.

I'm hoping I can build multiple packages simultaneously in a single chroot, but if that's impossible for reasons I'm not yet aware of, I'll figure out an approach similar to what you've done with mksandbox, using multiple chroots.

--
Earth is a beta site.



Home | Main Index | Thread Index | Old Index