port-mips/58072: mips: mis-counting of CPU states for accounting & monitoring

[he%NetBSD.org@localhost]

>Number:         58072
>Category:       port-mips
>Synopsis:       mips: mis-counting of CPU states for accounting & monitoring
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    port-mips-maintainer
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Mar 24 11:20:00 +0000 2024
>Originator:     Havard Eidnes
>Release:        NetBSD 10.99.10
>Organization:
	I try...
>Environment:
System: NetBSD mipsel 10.99.10 NetBSD 10.99.10 (MIPSSIM) #8: Mon Dec 25 18:08:12 CET 2023  he%mt.urc.uninett.no@localhost:/u/build/HEAD/obj/evbmips-el/sys/arch/evbmips/compile/MIPSSIM evbmips
Architecture: mips
Machine: evbmips
>Description:
	The evbmips port using the MIPSSIM kernel appears to
	mis-characterize and mis-count the CPU states with respect
	to whether it's "User", "Nice", "System" or "Idle".

	While trying to test rust 1.76.0, I have two builds running
	and they are makign progress, but despite this, e.g. "top"
	reports 100% "System" CPU, which is clearly bogus:

load averages:  2.22,  2.19,  2.13;               up 90+00:06:28       17:40:21
49 processes: 2 runnable, 46 sleeping, 1 on CPU
CPU states:  0.0% user,  0.0% nice,  100% system,  0.0% interrupt,  0.0% idle
Memory: 230M Act, 112M Inact, 16K Wired, 97M Exec, 148M File, 3028K Free
Swap: 2048M Total, 361M Used, 1687M Free

  PID USERNAME PRI NICE   SIZE   RES STATE      TIME   WCPU    CPU COMMAND
27998 he        21    4   289M  122M RUN        2:53 47.51% 47.51% rustc
19669 he        21    4   212M   60M RUN      317:27 47.02% 47.02% rustc
12376 he        85    0    18M  700K select     0:26  1.03%  1.03% sshd
13513 he        85    0  6036K  784K nanoslp    0:03  0.05%  0.05% vmstat
23354 he        43    0  9168K 1276K CPU        0:11  0.00%  0.00% top

	and with vmstat:

 procs    memory      page                       disks   faults      cpu
 r b      avm    fre  flt  re  pi   po   fr   sr l0 m0   in   sy  cs us sy id
 2 0   600584   2352   12   0   1    0    0    0  1  0   52 2074  22 0 100 0
 2 0   600584   2228   10   0   1    0    0    0 12  0   56 1901  23 0 100 0
 4 0   600836   1736   28   0   1    0    0    0  0  0   52 1467  21 0 100 0
 2 0   601332   1576   11   0   0    0    0    0  0  0   52 1511  21 0 100 0
 2 0   601836   1076   33   0   1    0    0    0  1  0   52 2295  21 0 100 0
 2 0   602524   5152  216   0  13  223  223  223 25  0   70 1342  37 0 100 0
 2 0   602872   4624   39   0   2    0    0    0  3  0   53 1665  22 0 100 0
 2 0   603328   4452   13   0   1    0    0    0  5  0   58 1794  22 0 100 0

	and with systat:

    7 users    Load  2.15  2.17  2.13                  Sun Mar 24 17:41:09

Proc:r  d  s        Csw  Traps SysCal  Intr   Soft  Fault     PAGING   SWAPPING
     3    11         52    284   1858    56     24    277     in  out   in  out
                                                        ops    2
 100.0% Sy   0.0% Us   0.0% Ni   0.0% In   0.0% Id    pages
|    |    |    |    |    |    |    |    |    |    |
==================================================                      2 forks
                                                                        2 fkppw
Anon        98924  20%   zero                 56 Interrupts             2 fksvm
Exec       100132  20%   wired      16         6 int 1 (virtio)           pwait
File       151196  30%   inact  115140           int 2 (uart)           2 relck
Meta        39944   4%   bufs     3767        50 int 5 (clock)          2 rlkok
 (kB)        real   swaponly      free           int 5 (clock) m          noram
Active     234616     363800      3176                                 42 ndcpy
Namei         Sys-cache     Proc-cache                                 19 fltcp
    Calls     hits    %     hits     %                                134 zfod
       68       68  100                                                   cow
                                                                      256 fmin
  Disks:     ld0     md0                                              341 ftarg
 seeks                                                                    itarg
 xfers         1                                                          flnan
 bytes      2600                                                          pdfre
 %busy                                                                    pdscn

>How-To-Repeat:
	Run a program which consumes significant amounts of CPU time.
	Watch any of the tools above report all CPU usage as "System".
>Fix:
	Sorry, don't know.