tech-userlevel archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Lua in NetBSD

About one week ago, I suggested to the NetBSD community to add the Lua language to NetBSD as a language for scripting applications and as a general scripting language.

My original post is here:

I am overwehelmed by the number of positive reactions I got to my proposal, it looks like I have been suggesting something that has been waited for by many. While I suggested to import Lua into base, much to my surprise it was even suggested to use it in the kernel, an idea I like the longer I think of it.

There was some criticism, too, especially since I did not state what I am going use Lua for, if it was in base; I wanted to discuss the general case first, so I was vague in this regard on purpose.

I am currently working on a scheme to interface all kind of external reference clocks to the system; not to get a timing base for timecounters, but to get absolute time information. This is something different. For timecounters you need a regular pulse, but a radio clock might only give the correct time once a day. Maybe some people know about my work in OpenBSD in this area. My current work builds on top of that previous work, but provides better abstractions and better interface (and thus more usuable information).

One family of external reference clocks provides timing information using bytestream protocols, the NMEA 0183 protocol used by most GPS receivers is one of the best known protocols that falls into this category. The decoding of the NMEA protocol has been done using a tty line discipline.

Using line disciplines for this kind of decoding comes at a price: Line disciplines are a vehicle not to many programmers are familar with (I could be wrong here, this is just an assumption), they are somewhat hard to get right and to maintain. It means to do a lot of string processing in the kernel, god beware if we have a serious, undetected bug in that code... But the main problem is that there are dozens of such protocols, most GPS vendors have their own proprietary binary protocol, Meinberg has it's own serial protocol, endrun comes to my mind and so on. So for every protocol we need a tty line discipline, need to define a name for the line discipline, have a program that can attach the line discipline to an open filedescriptor and so on (for the latter I wrote ldattach(8) in OpenBSD). One hast to touch many parts of the system to support a new protocol with a line discipline.

Time sources are kernel entities (drivers) that provide time information in a uniform way. They need to be drivers, since many clocks are attached to a PCI bus or use USB etc. To avoid the line discipline mess, I have a timesource that can be fed the time externally. So there is a daemon that decodes the time information and feeds it to the system. And this is were Lua comes into play. The decoders for the protocols are written in Lua, and new decoders can thus be added with ease and without requiring a kernel/userland rebuild. Besides of that I use Lua to write software with graphical user interfaces using X11/Xt and OpenMotif and to provide access to system functionality like sysctls, gpio etc. from Lua.

Lua, being a small language that interfaces exceptionally well with C programs, could be of good use in various other places, during the discussion, the following uses have been proposed or mentioned

- extending DHCP clients for options processing
- extending the bootloader
- extending sysinst
- uses in pkgsrc
- scripting nvi
- providing access to NetBSD from Lua (though for this, the Lua in pkgsrc is just fine)
- general scriptiing
- maybe stuff that I have forgotten to list...

Thor proposed to add it to the kernel as well. He would be willing to devote time and manpower to this project, so would I, and Lourival Vieira Neto from PUC Rio (where Lua is being maintained) offered to help, since he does a similar project in Linux ("Lunatik"). Having Lua in the kernel would actually allow to use NetBSD in total novel ways or for new fields of application:

- rapid prototyping of device drivers, or device drivers in Lua
- experimentation with new network protocols
- kauth(9) security models in Lua
- controlling the hardware environment (cpu freq etc) based on evironmental measurements (the Lunatik project at PUC Rio goes somewhat in this direction it seems)
- accept filters
- filter rules

Of course some implementation issues with Lua in the kernel remain to be solved: The number format. How to integrate it in the kernel. Should the compiler be included or only the bytecode interpreter (which means Lua scripts would be compiled to bytecode in userland and the bytecode then be loaded into the kernel), etc.

A quick recap of why Lua is a language well suited for this kind of applications:

- small memory footprint
- fast execution speed
- superb integration with C
- code can be prcompiled to bytecode for faster loading
- easy to learn with a "natural" syntax, not statically typed
- extensible in useful ways using the metatables mechanism
- very good data handling capabilities
- no bloated libraries

Quite some developers told me in private that they would use Lua for their work, if it was in the system and I think this all speaks for the addition of Lua to NetBSD. If there are not to many objections to this, I will ask core to give me a thumbs-up to initiate two subprojects:

- Adding Lua to NetBSD userland
- Adding Lua to the NetBSD kernel

Thanks for your time && let's do it!

- Marc Balmer

Home | Main Index | Thread Index | Old Index