Re: Profiling a build

[void <void%f-m.fm@localhost>]

On Wed, Dec 10, 2025 at 10:57:59AM +0000, Sad Clouds wrote:
> On Wed, 10 Dec 2025 06:11:02 -0000 (UTC)
> mlelstv%serpens.de@localhost (Michael van Elst) wrote:
>
> > Saying that, NFS is not ideal for building, there is lots of latency
> > added to even traversing file paths, and with many files (like in
> > a pbulk scenario), the filesystem caches won't be effective.
>
> I have two Raspberry Pi 4 SOCs and each has Samsung T7 SSD attached to
> USB3 port. For sequential writes using 64 KiB block size I get these
> metrics:
>
> NetBSD-10 local UFS file system:
>  5717.50 msec, 179.10 MiB/sec, 2865.59 Blocks/sec, 348.97 usec/Block
> NetBSD-10 NFSv3 network share:
>  10923.26 msec, 93.74 MiB/sec, 1499.92 Blocks/sec, 666.70 usec/Block
>
> I did have some issues with NFS file locking when running NetBSD
> build.sh from a 16-core Linux client, but I worked around those by
> mounting NFS shares with nolock,local_lock=all options. After that
> building on NFS shares (both source and object files kept there) works
> pretty well.

I should better explain the experiences that caused the decision
and conclusion I arrived at. (I didn't consider iSCSI at all though)

For a while in a freebsd context I struggled with e.g. git updates on
mips64 & armv6 (when those platfors were supported) armv7 and arm64.aarch64 on usb2
because it was as if the reads and writes to and from the usb2 mounted media
*combined with* (for example) memory pressure would stack up and cause a crash.
A sync(2) would take an age to return. Even worse on mmcsd.

In essence, it's not *just* speed of reads/writes. It's how busy the machine is.
Although, on paper, usb2 should have been sufficient, it wasn't. Offloading
reads and writes to nfs client meant I could (for example) buildworld on a 
pi2 with -j2 and still have interactive access, 5x ssh sessions, and top running.

Then there's the issue of RAM. All these machines have less than 1GB RAM.
Once memory pressure causes swap to be hit, if swap resides on usb2 or mmcsd,
in my experience the compile will likely fall over. -j2 is about the largest -j I
could get away wiht for a buildworld. -j1 for installworld (cos it involved 
writing to the mmcsd)

--