Re: [ANN] Lunatik -- NetBSD kernel scripting with Lua (GSoC project

[Lourival Vieira Neto <lourival.neto%gmail.com@localhost>]

On Tue, Oct 19, 2010 at 8:23 AM, Antti Kantee <pooka%cs.hut.fi@localhost> wrote:
> On Tue Oct 12 2010 at 02:17:35 -0300, Lourival Vieira Neto wrote:
>> On Tue, Oct 12, 2010 at 1:50 AM, David Holland 
>> <dholland-tech%netbsd.org@localhost> wrote:
>> > On Tue, Oct 12, 2010 at 12:53:10AM -0300, Lourival Vieira Neto wrote:
>> >  > > > > A signature only tells you whose neck to wring when the script
>> >  > > > > misbehaves. :-) Since a Lua script running in the kernel won't be
>> >  > > > > able to forge a pointer (right?), or conjure references to 
>> > methods or
>> >  > > > > data that weren't in its environment at the outset, you can run it
>> >  > > > > in a highly restricted environment so that many kinds of 
>> > misbehavior
>> >  > > > > are difficult or impossible. ?Or I would *think* you can restrict 
>> > the
>> >  > > > > environment in that way; I wonder what Lourival thinks about that.
>> >  > > >
>> >  > > > I wouldn't say better =). That's exactly how I'm thinking about
>> >  > > > address this issue: restricting access to each Lua environment. For
>> >  > > > example, a script running in packet filtering should have access to 
>> > a
>> >  > > > different set of kernel functions than a script running in process
>> >  > > > scheduling.
>> >  > >
>> >  > > ...so what do you do if the script calls a bunch of kernel functions
>> >  > > and then crashes?
>> >  >
>> >  > if a script crashes, it raises an exception that can be caught by the
>> >  > kernel (as an error code)..
>> >
>> > Right... so how do you restore the kernel to a valid state?
>>
>> Why wouldn't it be a valid state after a script crash? I didn't get
>> that. Can you exemplify it?
>
> I *guess* what David means is that to perform decisions you need a
> certain level of atomicity.  For example, just drawing something out of
> a hat, if you want to decide which thread to schedule next, you need to
> make sure the selected thread object exists over fetching the candidate
> list and the actual scheduling.  For this you use a lock or a reference
> counter or whatever.  So if your lua script crashes between fetching the
> candidates and doing the actual scheduling, you need some way of releasing
> the lock or decrementing the refcounter.  While you can of course push an
> "error branch stack" into lua or write the interfaces to follow a strict
> model where you commit state changes only at the last possible moment,
> it is additional work and probably quite error-prone.
>
> Although, on the non-academic side of things, if your thread scheduler
> crashes, you're kinda screwed anyway.
>

Hi Antti,

Sorry for the delay. I agree: we need a certain level of atomicity. I
think that level should be provided by the libraries that expose
kernel internals to Lua (binding libraries), the kernel code that
calls Lua and the Lunatik state's mutex. The functions of the binding
libraries should not finish their execution with locks (or other
resources) held. If it is really necessary, the binding libraries
could provide functions to validate the state, after the Lua
execution. However, I don't think that is a good idea to allow scripts
to call functions that uses a lock without releasing it. Moreover, we
can use the Lunatik state's mutex to perform the synchronization
(between the kernel and the script code).

In your scheduling example, we can use a refcounter (as you said)
stored in the Lua state and protected by the Lunatik state's mutex.
Thus, if our Lua script crashes between fetching and scheduling, the
caller can trace and treat that appropriately (e.g., restoring the
refcounter, deleting that script function and calling a predefined
function to perform the thread scheduling). Although, I think a better
approach for that problem would be to provide a scheduling function
that checks if the selected thread exists and fails if not (returning
a error code for the script).

In short, I think the functions provided for the scripts should be
self-contained and all the locks should be managed by the kernel code.
If functions of the binding libraries need to share and synchronize
their execution state (e.g., a refcounter), they need to do so by
storing the desired state in Lua.
--
Lourival Vieira Neto