Re: Is "scripting" truly Lua's future?

[David Given <dg@...>]

Matt Campbell wrote:
> If I'm designing a new application, should I
> write the application core in C and then use Lua for scripting on top of
> that, or should I write the application in Lua, using C extension
> modules only where needed?

Well, I did the latter for WordGrinder, partly as an experiment to see
how well it worked.

<plug> http://wordgrinder.sourceforge.net </plug>

The short answer: it works awesomely well.

What I have is a complete if simple word processor, written from
scratch, in 5600 lines of Lua and 1400 lines of C (not including the Lua
and LFS engines, of course). The C contains low-level screen drawing
routines, UTF-8 read/write routines, various bits of contorted setup,
and a very few specially optimised routines for doing stuff with words.
Everything else is in Lua.

Lua's talent at storing complex data structures intuitively allowed me
to put together a data model for storing documents that's fast and
reasonably efficient: a document is an array of paragraphs, a paragraph
is an array of words, and each word is a table. Because arrays are
tables, that allows me to attach methods and cached state to every
object. For example, line wrapping is done by storing, with each
paragraph, an array of lines, where each line is itself an array of the
words in the line. This then gets regenerated on demand. Since drawing
occurs at the cursor position and works up and down until the screen
bounds are reached, this means that the bulk of the document won't get
wrapped until you're actually looking at it --- which means it's fast.

There's nothing there that's particularly hard. I could do it all in C++
with the STL, for example. But it would have taken me a lot longer than
a month to do, which is how long the first working version of
WordGrinder took.

It's also very easy to modify. For the most recent version, 0.3.1, I
added a Table Of Contents feature: a dialogue that produces a
hierarchical list of headers and allows you to jump to one. This took
about an hour to write and is 97 lines of code, including comments and
whitespace:

http://wordgrinder.svn.sourceforge.net/viewvc/wordgrinder/wordgrinder/src/lua/addons/goto.lua?revision=121&view=markup

It's also *efficient*. New with 0.3.1 is a Windows version. Because
Windows machines tend not to come with package managers, I had to embed
the Lua and LFS code into the executable. Since I was doing this, I also
decided to byte-compile the Lua source code and embed that, as well, to
produce a single, stand-alone executable. This is 266kB uncompressed
EXE. That's the WordGrinder Lua code, the WordGrinder specialist C
bindings, the Lua engine itself, the LFS bindings, and all the overhead
needed for a Windows app, which is not insubstantial (including the icon
--- all 45kB of it!). I think that's quite impressive.

However, doing things in Lua also has its disadvantages. Like all
languages, Lua has its quirks --- details below. However, the major
architectural issue is that Lua, being a dynamic language which does
most things at run-time, allows you to compile in typos. 0.3.1 is a
bugfix release of 0.3 which I released when someone pointed out that if
you did File->New the application crashed. Highly embarrassing. While
fixing that I found another similar crash. I hope there aren't any more
--- but it's very hard to know.

Also, writing code in Lua, once you have it straight in your head, is
*so* fast it requires iron discipline to keep the code sensible, well
organised, and well commented. It's absolutely vital to maintain coding
standards, naming conventions, file organisation, etc. It's very
tempting to let your program fill up with quick hacks, which means it
quickly turns into an unmaintainable spaghetti mess full of special
cases. At least it's very quick to redraft large passages when you
decide to change an algorithm or data structure; this lends itself well
to evolutionary design, which is how I tend to code.

...

That concludes the 'my experiences writing a moderately complex app in
Lua' essay, so now on to some of the specific rants:

- Lua doesn't have a preprocessor. My app was made up of multiple source
files, and most of the time I wanted the same common header at the top
of each source file (importing C functions, mostly). Many's the time I
wished for #include.

- the module structure is a pain when writing single applications ---
because once you've done module(), the standard library is no longer
available. You have to import everything with 'local print = print'
statements. And if you forget one you get no warnings or error messages
until it crashes at run time, either. Again, #include would have helped
with this. Eventually I gave up on modules.

- 1-based string offsets... do, actually, cause problems. I think I've
talked about this before; the problem is string positions and string
deltas. With 0-based offsets, they're the same. With 1-based offsets,
they're not, which means you need *different code* depending on whether
you're referring to something by position or by delta. This genuinely
did cause me a lot of grief until I got everything straightened out.

However, I should point out that I would *still* rather cope with Lua's
quirks than, say, Symbian's C++ dialect, which is what I'm currently
struggling with in my day job...

-- 
┌─── ｄｇ＠ｃｏｗｌａｒｋ．ｃｏｍ ───── http://www.cowlark.com ─────
│
│ "All power corrupts, but we need electricity." --- Diana Wynne Jones,
│ _Archer's Goon_

Attachment: signature.asc
Description: OpenPGP digital signature