Re: Making run queues independent of the pluggable scheduler

[Mindaugas Rasiukevicius <rmind%NetBSD.org@localhost>]

Hello,

Andrew Doran <ad%netbsd.org@localhost> wrote:
> The diff below extracts the per-CPU run queue code from the M2 scheduler and
> makes it non optional, removing the 4BSD scheduler's global run queue. With
> the patch, it means that the pluggable scheduler is responsible only for
> adjusting the priority of timeshared jobs.

I would prefer to split it into the separate file, eg. kern_disp.c or other.

> - It changes the CPU selection algorithm slightly. Weak affinity is not
>   considered until the job has context switched a preset number of times,
>   currently 5. This is to try and better distribute jobs among the CPUs.  It
>   uses the new call idle_pick to find an idle CPU if possible. If no idle
>   CPUs, it does a circular scan of CPUs instead of always starting at the
>   first CPU. That's to try and ensure that we don't unfairly overload one
>   CPU. I will make the CPU selection changes a seperate commit if they have
>   been demonstrated to be worthwhile.

Have you measured what is the effect of these changes for different workloads?
Atomic bitmap of idle CPUs and circular scan (although it should be CPU
topology aware in the future) looks fine. How about 'settle_thresh'?

> - Some or all of the items in runqueue_t could be safely merged into 
>   schedstate_percpu, but I think it's better to integrate things piecemeal
>   if possible.

It looks better for me to abstract run-queues into the separate structures.
But in essence it does not matter too much..

While you are here - please change the value of 'min_catch' to 1, which seems
to be most reasonable, as we have measured in various tests.

Thanks.

-- 
Best regards,
Mindaugas
www.NetBSD.org