MJ a écrit : > > > On 28/02/2026 10:25 pm, BERTRAND Joël wrote: >> MJ a écrit : >>> >>> >>> On 28/02/2026 8:16 pm, BERTRAND Joël wrote: >>>> MJ a écrit : >>>>> >>>>> >>>>> On 28/02/2026 8:21 am, BERTRAND Joël wrote: >>>>>> Michael van Elst a écrit : >>>>>>> joel.bertrand%systella.fr@localhost (=?UTF-8?Q?BERTRAND_Jo=c3=abl?=) writes: >>>>>>> >>>>>>>> CPU is an i7-4770, main memory 16 GB. This server exports >>>>>>>> /srv and >>>>>>>> /home through NFS (V3/TCP, 128 threads, async) and disk I/O from >>>>>>>> NFS >>>>>>>> clients are very slow. Server load can raise until 110 or 120 >>>>>>>> during >>>>>>>> huge NFS access. >>>>>>> >>>>>>> Can you quantify what "slow" means? Any kind of benchmarks? >>>>>> >>>>>> Less than 2 MB/s. >>>>>> >>>>>>> The server load just shows that the NFS requests are distributed >>>>>>> over >>>>>>> your server threads. But that is not related to any CPU utilization. >>>>>> >>>>>> I know. NFS process never reaches 35% of one core. >>>>>> >>>>>> >>>>>> >>>>> As per Michael's email, re: isolating disk and client, can you isolate >>>>> which system it is slow serving to? I think that is important. >>>> >>>> Same result with Linux and FreeBSD. If I reduce number of nfs >>>> threads >>>> (currently 128), it seems to run better but I obtain on client side >>>> "nfs >>>> server not responding". >>>> >>>> Now, I only have two client (a Linux and a FreeBSD). FreeBSD is >>>> idle >>>> and I do some tests on Linux workstation. >>>> >>>> 1/ Linux rootfs is on a Raid1 disk on NetBSD server. >>>> 2/ iftop shows that nfs server is idle too (a few Kbps). >>>> 3/ apt update && apt dist-upgrade is very slow (-dev packages with a >>>> lot >>>> of little files). >>>> 4/ make -j1 kicad (9.0), sources on a Raid5 volume shows a nfs mean >>>> throughput around 40 Mbps. nfs process eats 1 to 2 % od server CPU. >>>> 5/ now, I start another compilation and load is rising on server side : >>>> >>>> load averages: 10.0, 3.17, 2.06; up 16+01:28:30 >>>> 10:10:27 >>>> 93 processes: 92 sleeping, 1 on CPU >>>> CPU states: 0.1% user, 0.0% nice, 0.1% system, 0.0% interrupt, 99.7% >>>> idle >>>> Memory: 7838M Act, 3937M Inact, 49M Wired, 153M Exec, 9617M File, 84M >>>> Free >>>> Swap: 16G Total, 16G Free / Pools: 3708M Used / Network: 2647K In, >>>> 8700K Out >>>> >>>> PID USERNAME PRI NICE SIZE RES STATE TIME WCPU CPU >>>> COMMAND >>>> 4326 root 85 0 600M 8248K nfsd/4 823:01 7.13% >>>> 7.13% nfsd >>>> 2553 root 85 0 20M 2640K kqueue/3 2:59 4.88% 4.88% >>>> syslogd >>>> 0 root 221 0 0K 64M rfnode/3 803:17 4.10% 4.10% >>>> [system] >>>> 930 root 85 0 12M 1900K select/0 3:26 1.71% 1.71% >>>> rpc.lockd >>>> >>>> >>>> 6/ dd if=/dev/zero of=test.dd count=10 bs=100M >>>> nfs throughput rises until 850 Mbps (iftop) >>>> but load average on server side until 56 ! >>>> >>>> hilbert:[~] > dd if=/dev/zero of=test.dd count=10 bs=100M >>>> 10+0 enregistrements lus >>>> 10+0 enregistrements écrits >>>> 1048576000 octets (1,0 GB, 1000 MiB) copiés, 69,9613 s, 15,0 MB/s >>> >>> >>> If the load is going up but not CPU usage, perhaps something is blocking >>> a process? Lock contention? >>> >>> If you do a process list (PS) are there any processes in D state on the >>> server when you perform work over NFS? >> >> Only nfsd and kernel : >> >> legendre# ps auwx | grep ' D' >> root 23258 68.9 0.1 614028 8312 ? Dsl 12:14PM 0:36.32 >> /usr/sbin/nfsd -n 128 >> root 0 0.1 1.1 0 182868 ? DKl 12Feb26 807:12.02 >> [system] > > > Yikes. Have you tried running the server with far less threads? I, > personally, can't see the benefit of such a high amount of threads. > This is probably inviting contention. > > My usual rule-of-thumb was (2 * cores ) + 2. Not scientific, just > historically, "it worked". > > So, have you tried the server running -n 16 or -n 20 or similar. I have tried with -n16. nfs server seems to run better, but client receives a lot of "nfs server not responding". Linux can handle with this message (even in diskless configuration), but FreeBSD can crashes.
Attachment:
signature.asc
Description: OpenPGP digital signature