Re: Rust, pkgsrc

[Chavdar Ivanov <ci4ic4%gmail.com@localhost>]

On 11/06/18 18:50, Havard Eidnes wrote:
> > Yes. I had a similar problem.  The build would fill up the
> > /tmp/ directory and die from exhausted resources.  I had /tmp/
> > created with tmpfs and had a constraint of 64M.  The answer for
> > me was to create /tmp in /etc/fstab with tmpfs and no size
> > constraint.  Then Rust would build, but it still took a long
> > time.
> Yes, rust is an absolute Pig resource-wise.  Not only does it
> carry a copy of llvm inside itself (it most probably has to), it
> also carries a nubmer of other packages inside.  Plus, the build
> builds most parts at least twice if I've been able to observe
> correctly.
>
> When building on NetBSD/amd64 8.0, I noticed that the work/
> directory after a "make" consumes in the order of 10G disk space,
> possibly more when cross-building (my current work/ is 13G, which
> is an unfinished cross-build because I hit build issues...).
>
> It is conceivable that the storage could be reduced somewhat (but
> probably not by much?) by tweaking src/bootstrap/boostrap.py to
> say -Cdebuginfo=0 instead of 2 for RUSTFLAGS.
>
> So that you have an idea what to expect:
>
> My amd64 build host has 8G real memory, and a 2G tmpfs /tmp, and
> ... it didn't run out of space anywhere :) On this particular
> host (i7 3rd-gen, 4 real cores, 8 w/HT, pkgsrc and system on SSD)
> the build of 1.29.2 completed in a little over 2 hours wall-clock
> time, csh's "time" report at the end of the build was
>
> 40468.007u 1958.277s 2:04:42.18 567.0%  0+0k 10302+100556io 129329pf+0w
On my 16GB HP Envy, 4c/8t, I got overnight:
---
Build completed successfully in 4:22:05
make  14672.75s user 36893488148207.93s system -2147483648% cpu 
4:43:37.28 total
/usr/pkgsrc/lang/rust
----
with /tmp just a standard directory, tmpfs as usual on /var/shm. This is 
more than twice as slow compared to the last few rust builds, which also 
took just over two hours.

I wonder if something was changed in the build process, previous builds 
were highly parallelized, with all the threads being seen busy during 
various stages, whereas this time I didn't see anything like this.

All my builds were off unmodified pkgsrc lang/rust directory with 
nothing in /etc/mk.conf that I can see to affect this.

I suspect the ridiculous times for system and cpu are from zsh 'time'.

> So, yes, the build makes fairly good use of the multiple cores;
> notice the 567.0%, which, if I've understood correctly, indicates
> approx. 5.7* parallelism on average.
>
> The last version I managed to build on one of my NetBSD/macppc
> 8.0 machines (a single-core 1.5GHz G4 Mac Mini, 1GB memory) was
> 1.29.2, the build took nearly 29 hours wallclock time.  This one
> doesn't have a tmpfs, and has a single file system with ~40G
> free, so it also didn't run into any barriers on the /tmp front
> either.
>
> Regards,
>
> - Håvard