Re: Lua in NetBSD

[Marc Balmer <marc%msys.ch@localhost>]

> On Sat, Oct 24, 2009 at 10:48:41AM +0200, Marc Balmer wrote:
>> There was some criticism, too, especially since I did not state what I  
>> am going use Lua for, if it was in base;  I wanted to discuss the  
>> general case first, so I was vague in this regard on purpose.
>> 
>> I am currently working on a scheme to interface all kind of external  
>> reference clocks to the system; not to get a timing base for  
>> timecounters, but to get absolute time information. [...]
>> 
>> One family of external reference clocks provides timing information  
>> using bytestream protocols, the NMEA 0183 protocol used by most GPS  
>> receivers is one of the best known protocols that falls into this  
>> category. The decoding of the NMEA protocol has been done using a tty  
>> line discipline.
>> 
>> Using line disciplines for this kind of decoding comes at a price:
>> [...]
> 
> Line disciplines are a legacy misfeature that probably ought to go
> away in the long term. I would certainly not recommend using them for
> this, or for that matter recommend adding any new ones for doing
> anything whatsoever. Taking a trip through a userland daemon is a much
> better alternative... provided that the latencies involved are
> acceptable.

Agreed.

>> And this is were Lua comes into play.  The decoders for the
>> protocols are written in Lua, and new decoders can thus be added
>> with ease and without requiring a kernel/userland rebuild.
> 
> But, are the latencies involved acceptable? You know a lot more about
> this than I do, so if you say so I'll believe it, but it seems to me
> that switching to a userland daemon written in an interpreted language
> is not exactly what one wants for time synchronization.

If precision of the time information is key, then the received time information 
will be synchronized to a PPS signal, some GPS receivers can be programmed to 
deliver a PPS signal, then NMEA package that then follows indicates the time at 
the previous PPS signal.

Detecting the PPS pulse is always done in the kernel, and it this pulse and 
it's precise detection that defines the overall precision.

If you don't use such a pulse, your time information will not be as precise, 
but still good enough for many applications.

> It also doesn't seem to me that there's any particular reason to write
> this code in Lua instead of in C/lex/yacc/snobol/whatever other tool
> of the day. (Nor is there any particular reason not to; but we already
> have C in base.)

I and every skilled developer can of course write such decoders in any 
language.  But having the decoders is only part of the solution.  The other 
part is that I want to enable people withouth the deep programming experience 
of a BSD developer to add support for new protocols.  Make it easy for users to 
interface their systems with a language simple enough to learn in short time.

Of course, even with this goal there are plenty of languages that would fit at 
first sight.  I outlined the choice for Lua in previous mails, in short:  very 
good embeddability, small memory footprint, execution speed.

> I guess what I'd like to see is some examples or something else that
> shows that we can expect some long-term advantages of some kind,
> presumably in maintainability. Or alternatively, at least an argument
> for Lua where doing s/Lua/SNOBOL/ doesn't yield an equally persuasive
> argument for importing SNOBOL into base.

I think I have outlined the benefits of Lua vs. other languages in my previous 
mails.

> I guess I'm fundamentally not persuaded that coding in Lua offers any
> substantial advantages to coding in C. Things that might persuade me
> include

Here the same argument as above should be considered:  Lua should not be seen 
as a replacement for C to be used by NetBSD developers only.  It should also 
make it easy for users to change/extend things.

>   - standard libraries for doing things easily that are a hassle in C
>     (but half the point of Lua in this context is that it's small);

Modules could be added quite easily.

>   - language features or syntax for concisely expressing things that
>     are tedious or painful in C;
>   - language features that allow the implementation to be better at
>     rejecting invalid programs than a C compiler can be;
>   - a clear use case where we seriously expect that end users will
>     write Lua scripts to drive something, and where we seriously
>     expect that this isn't workable using C (or sh);

Well, Lua is defined, we can not invent a different Lua.

> Things that probably won't persuade me include
>   - noting that Lua is memory-safe or garbage collected and C isn't,
>     because these are not serious problems in practice;
>   - noting that Lua interfaces to C well, because where the
>     alternative is coding in C this is a nonissue;

I take the liberty to contrict you in the last point.  Coding in C can be very 
hard and error prone even to halfway experienced programmers.  Doing the same 
in Lua is much easier.  Dragos said "anecdote is not a replacement for 
evidence", I give you an anecdote anyway:

At work we develop a quite complex piece of software and initially everything 
was coded in C only.  We made very little progress and the folks were not 
really productive (in terms of hw much new functionality was added over time).  
Then we embedded Lua, exported all relevant core functionality to Lua and let 
the folks use Lua to implement funcionality.  We saw an immediate and dramatic 
productivity boost.  People felt much more comfortable with Lua than with C, 
testing got easier and in the end we get more software done in less time.

> Note that I'm not really looking for all that much, just some clear
> evidence of advantage, because i haven't really seen anything very
> persuasive yet. Because Lua is small and fairly inoffensive, the bar
> is pretty low; but because it's small and fairly inoffensive it also
> doesn't have much in the way of clear benefits.
> 
> (By contrast, for example, Haskell provides a number of substantial
> advantages over coding in C, at least for certain kinds of programs
> like compilers or code analysis tools; however, the cost of importing
> the Haskell compiler into base is monumental and the tradeoff is
> pretty strongly negative regardless.)

I am under the impression that you only look at language features in the more 
technical, narrower sense while I also take into consideration something I'd 
call "side effects" of a language:  Ease of use, learning curve, potential 
productivity and such.  And here I see definitely large benefits.

> Also note that loadable kernel extensions in Lua is a completely
> different ball of wax; since I have specific experience with this
> problem I have some specific suggestions, which I'd be happy to share
> on tech-kern.

Lua in kernel will not only be a separate "ball of wax", but a separate 
mini-project, too.
> 
>> - easy to learn with a "natural" syntax, not statically typed
> 
> Lack of static typing is not an advantage.

Here again, different points of view.  There are obviously standpoints where 
not being statically typed is an advantage.  We probably should just leave out 
the feature of static vs. non-static in the Lua discussion, since some people 
see this as an advantage (me) and some don't (you).

- Marc Balmer