pkgsrc-Users archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: Conceptual talk about distributed pbulk on exotic port

Thanks, I'll check it out.  

@Jonathan when you say "Spin up as many processes as your hosts will take" correct me if i'm wrong but qemu-system-sparc is not the same as a kvm-qemu. qemu-system-sparc rely on dynamic translation and will emulate cpu architecture as well as cpu speed. Even if I had a 6Ghz proc with bazillion cores, it will still be capped with one 170Mhz cpu per "Emulated VM" ?

@Greg I came to a setup which is not as advanced as you want but nevertheless interesting. A pbulk executed on a sparc host with 3 "cross-distcc host" ( distcc compiling code for sparc on amd64 ). Unfortunely, communication between sparc host and distcc host were not by ssh. Still I can check my note and share the information if interested.

On Thu, Jul 5, 2018 at 1:03 PM Greg Troxel <> wrote:

(no disagreement with Jonathan's advice)

Agreed that true full cross compiling doesn't work in general.  Note
that part of the issue is things wrong in pkgsrc, and part of it is that
many upstream programs do not crossbuild like they should.  I remember
someone going on rampage in the late 90s to enable compiling for sun2 on
sparc and fixing up a lot of packges.

It would be a useful contribution to figure out distcc and cross in a
general way.  I have been heading to this for a while and not gotten to
it.  Basically, I want to do this such that:

  distcc uses ssh, rather than distccd and
  cleartext/no-auth/no-authentication-of-results.  (Note that ssh with
  ControlMaster and a background connection is very fast.)

  the build host can have sevaral compilers, for various OS/version/cpu,
  and is able to switch to the right complier based on something set by
  the client or looked up by address.

It may be that running a cross compiler as distcc is faster than running
a native compiler under qemu.  Of cousre, you still need native cycles
for many things, so one native and many cross distcc is surely not

Home | Main Index | Thread Index | Old Index