Re: Atomic migration chains, balancing fixes.

[Andrew Doran <ad%netbsd.org@localhost>]

On Sat, May 17, 2008 at 03:00:39AM +0100, Mindaugas Rasiukevicius wrote:
> Mindaugas Rasiukevicius <rmind%NetBSD.org@localhost> wrote:
> > Here is the patch to simplify the LWP migration by inventing atomic
> > migration chains. It should fix PR/38169, however, it is different
> > approach. Patch also makes periodical balancing mandatory, and thus fixes
> > it and the priority raising in M2, which was broken after making run-queues
> > mandatory.
> 
> Here is the updated patch:
> 
> http://www.netbsd.org/~rmind/mchains_and_fixes_2.diff
> 
> Migration points are adjusted slightly. This fixes few bugs and race
> conditions (which also exists in the -current).
> 
> Unless nobody objects, I am intended to commit this.
> Comments are welcome :)

- I think the KASSERT(l->l_stat == LSRUN) in cpu_xc_offline() is broken because
  something could, for example, make an LWP stopped or suspended.

- For cpu_need_resched(l->l_cpu, RESCHED_KPREEMPT), spc_mutex also needs to be
  held, not just spc_lwplock.

- l->l_stat = LSIDL: I think it will break lwp_suspend().

- In mi_switch: if (__predict_false(l->l_stat == LSONPROC ...) Why check 
something
  that is known to be true?

I still prefer the scheme described in PR/38169. In fact, I think it could
be even simpler - I will update the PR. The cost of switching to the idle
LWP to do a forced migration is proably small when compared to the penalty
that the migrating LWP will suffer because of cache traffic and TLB refill.

Andrew