Hi,

For the yamt-idlelwp branch I made a patch that changes the locking strategy
slightly. It assigns a per-CPU lock to LWPs as they transition to the ONPROC
state. This works well because external agents are not usually interested in
running LWPs, but an LWP must lock itself before going to sleep. It wasn't
possible to do this before since cpu_switch() made things quite a bit more
complicated.

Without the change it is possible for the scheduler to call spc_lock() and
then all running and runnable LWPs assigned to the target CPU are locked.
With this change, spc_lock() will only lock LWPs that are on a run queue.
Can anyone see a potential situation where this is likely to cause a problem
for the scheduler?

I've attached lockstat output below. It's from a 10 second run on a 4-way
system, with 10 user threads playing a game with a condition variable &
mutex pair.

Thanks,
Andrew

=> without

Total%  Count   Time/ms          Lock                       Caller
------ ------- --------- ---------------------- ------------------------------
 61.77 1198866   2256.37 lwp_park_tab+750       <all>
 57.31 1053311   2093.52 lwp_park_tab+750       sys__lwp_unpark+3d
  4.35  143627    158.95 lwp_park_tab+750       sys__lwp_park+ce
  0.11    1912      3.87 lwp_park_tab+750       sys__lwp_unpark_all+167
  0.00       7      0.01 lwp_park_tab+750       sched_pstats+c8
  0.00       4      0.01 lwp_park_tab+750       sys__lwp_unpark+e2
  0.00       4      0.01 lwp_park_tab+750       sched_pstats_hook+57
  0.00       1      0.01 lwp_park_tab+750       lwp_lock_retry+28
 38.23  589081   1396.72 sched_mutex            <all>
 37.27  567743   1361.59 sched_mutex            sys__lwp_park+dc
  0.84   12040     30.80 sched_mutex            mi_switch+37
  0.05    1758      1.89 sched_mutex            idle_loop+4d
  0.05    7131      1.85 sched_mutex            preempt+33
  0.01     344      0.51 sched_mutex            sleepq_remove+8e
  0.00      43      0.05 sched_mutex            sched_schedclock+4e
  0.00      10      0.01 sched_mutex            yield+33
  0.00       4      0.01 sched_mutex            sched_pstats+c8
  0.00       1      0.00 sched_mutex            selwakeup+95
  0.00       3      0.00 sched_mutex            sched_pstats_hook+57
  0.00       1      0.00 sched_mutex            cv_wait+9b
  0.00       2      0.00 sched_mutex            sys__lwp_unpark+e2
  0.00       1      0.00 sched_mutex            issignal+2ff

=> with

Total%  Count   Time/ms          Lock                       Caller
------ ------- --------- ---------------------- ------------------------------
 99.99  562260    824.15 lwp_park_tab+ba0       <all>
 80.69  417517    665.12 lwp_park_tab+ba0       sys__lwp_unpark+3d
 17.37  136615    143.18 lwp_park_tab+ba0       sys__lwp_park+ce
  1.92    8110     15.80 lwp_park_tab+ba0       sys__lwp_unpark_all+167
  0.00       8      0.03 lwp_park_tab+ba0       sys__lwp_unpark+e2
  0.00       7      0.02 lwp_park_tab+ba0       sched_pstats+c8
  0.00       3      0.01 lwp_park_tab+ba0       lwp_lock_retry+28
  0.01     270      0.10 sched_mutex            <all>
  0.01     235      0.09 sched_mutex            mi_switch+3b
  0.00      34      0.01 sched_mutex            sleepq_remove+96
  0.00       1      0.00 sched_mutex            lwp_lock_retry+28
  0.00       3      0.00 00000000c1649030       <all>
  0.00       1      0.00 00000000c1649030       sched_pstats+c8
  0.00       2      0.00 00000000c1649030       sys__lwp_unpark+e2
  0.00       3      0.00 cpu_info_primary+30    <all>
  0.00       2      0.00 cpu_info_primary+30    sys__lwp_unpark+e2
  0.00       1      0.00 cpu_info_primary+30    sys__lwp_park+dc