Re: xennet performance collapses with multiple vCPU

[Jaromír Doleček <jaromir.dolecek%gmail.com@localhost>]

Le mer. 25 mars 2020 à 17:16, Jaromír Doleček <jaromir.dolecek%gmail.com@localhost> a
écrit :

> Le mar. 24 mars 2020 à 15:13, Stephen Borrill <netbsd%precedence.co.uk@localhost> a
> écrit :
>
>> All using NetBSD 9.99.51 (XEN3_DOMU) #0: Sun Mar 22 17:35:29 UTC 2020
>>
>> Note that increasing the vCPUs from 1 to 4 (2 gives same result as 4)
>> drops the throughput to about 10%. No offloads configured in domU:
>>
>>
> I found out the problem appeared between 2020-01-08 00:00 (that was still
> good) and 2020-01-14 00:00 (that is already bad).
>
> There were some CPU scheduling changes by Andrew Doran around that time,
> so it's likely the culprit.
>
> I'm working on bisecting this further. It's complicated, between 01-08 and
> 01-14 there were some bugs which prevent the Xen DomU MP kernel from
> booting.
>

I've now tracked it down to this change:

Module Name:    src
Committed By:   ad
Date:           Mon Jan 13 20:30:08 UTC 2020

Modified Files:
        src/sys/kern: subr_cpu.c

Log Message:
Fix some more bugs in the topo stuff, that prevented it from working
properly with fake topo info + MP.


To generate a diff of this commit:
cvs rdiff -u -r1.10 -r1.11 src/sys/kern/subr_cpu.c

After this change the DomU even boots visibly slower. Maybe this change
makes MP system scheduler use all CPUs, but introduces too much switching
between them? Andy, can you have a look?

I'll meanwhile check if there is anything obvious in the fake topology code