So I have one remaining sticky issue with reviving Scheduler Activations for -current. I need help with exactly how to manage sa_switch(). sa_switch() is the routine that we call in sleepq_block() for SA threads instead of calling mi_switch(). That's where the name comes from. ;-) The big thing that sa_switch needs to do is, if our lwp is the "blessed" lwp, we need to generate an upcall and cause another lwp to run (to deliver it). Remember, the SA code maintains a number of Virtual Processors (VPs), and has a "blessed" lwp running on each. When the "blessed" lwp blocks in the kernel (sleeps on something), a new lwp gets blessed and then handed to the application to run other user threads. The code ensures that there's an lwp waiting, and one of the fixes in FixSA was to pre-allcoate the data structure. I've pulled these changes forward. Thus there is no reason to sleep in sa_switch(). We will need to lock the mutex on the waiting lwp, but that's it. The thing I'm wanting help with is how exactly do I structure making the waiting lwp runable? I've been talking with rmind, and he's helped me with some ideas. I think it's not hard, I just am not getting the details to solidify. Also, I want to use this EMail to broaden the discussion to all developers. I have a lot of flexability. My thought was to have a per-vp wait channel (not 100% sure how to do it, but lwp_park() is an example, as is the condvar code). I'm thinking a sleep queue of some sort, since the transition from LSSLEEP to LSRUN can be done w/o locking p_lock. We need to not lock p_lock since it's an adaptive mutex, and you can sleep locking it! Bad. We won't do that. Once I get to this point, I'm feeling a lack of details. Any suggestions? Should I just make a modified lwp_park()? Are there any locking tricks I need to perform in order to make a thread runable while I'm about to call mi_switch()? I'm not sure what locks other than the sleeping lwp I have locked at that point. What do I do about cpu affiliation? I really don't care what CPU this woken thread runs on. It would be best if it runs soon, and it might be easiest if it ran on the current CPU (avoiding cross-CPU communication). But we have flex. If it would help, the first thing the woken lwp can do is ask the scheduler to see if it needs to be moved to another CPU. We can do this at a point where we won't generate an upcall. I also certainly don't want to break inter-CPU thread migration, and I'd rather not have to change the migration code to special case SA lwps. Can I somehow l_switchto to tell mi_switch() to switch to the to-be-woken thread? This is the only remaining design issue I see with SA. I want to use existing methodologies to solve this problem. The only reason I could possibly see for needing new routines outside of the SA code (like say a special setrunnable()) is if say our locking environment is unique enough that that's the best solution. Suggstions? Thanks! Take care, Bill
Attachment:
pgpdB0OmoZI9f.pgp
Description: PGP signature