RE: Lua presentation at XGDC3.0

["Joshua Jensen" <jjensen@...>]

> The target level of the talk is intermediate to advanced game 
> programmers. I have 50 minutes to talk (thus it's more of an 
> "advertising" presentation rather than a tutorial -- sell 
> them on Lua and they will do the research).
> 
> Please let me know if you think anything is missing, glossed over, 
> too much information, (just plain crap ;-) etc.

:)

Hi, Mike,

Since your presentation is targeted at game developers (I'd like to be
there!), I am going to relate some issues I've had with Lua and console
development.

While I have used Lua on a PC for years (both Windows and Unix), I had
never used Lua on a console until a year ago.  What I learned is that it
is critical to route all allocations for Lua through a custom heap made
just for Lua.  If Lua allocations are intermingled with all sorts of
other game data, the fragmentation levels become unacceptable, since Lua
likes to allocate/deallocate little blocks of memory often (and
sometimes unexpectedly).

Going along with your idea of "precompiling" data files, I would make a
point to recommend Lua's use for ALL file formats in a game.  Insane?
Not during development, IMO.  In Amped, a number of the file formats
have been in Lua format throughout the course of the project.  The
flexibility of this approach is unparalleled.  Anyone can sit down and
write a script to manipulate some type of game data.  Amped's string
table is in Lua, for example.  A 4 line script was written to dump the
string table into .csv form for Excel importing.  4 lines!  (A dofile(),
for, write(), and end).  That's hard to top.  In another case, two large
Lua tables were merged into one in just a few lines of code.  It is far
easier to edit them as separate entities, but why waste the space in
game?

You might mention that having game data exporters write to a Lua script
not only information you know you're going to need but also information
you think may be useful is a good idea, too.  Anyone can write a
postprocess script to pick out the information your game needs.  At this
point, you make a choice... Does the postprocess script write the
information in Lua form or does it write it in binary form?  Several
file formats in Amped, for example, have stayed in Lua form until just
recently.  It was much easier to initially prototype code (especially
in-game scripts) when it was in this form.  Now that Amped is about to
ship, some of them, for memory usage issues, have been replaced with a
more compact binary format.  The important point is I was able to deal
with the extremely flexible nature of Lua for data description
throughout the course of the project and lock it down to a binary format
when I knew EXACTLY the information the game would finally need.

Binary data can be stored very easily in a Lua script, too.  Amped's
string table uses the binary nature of Lua's strings to represent
Unicode strings.  So I would disagree with your slide that Lua only be
used for non-binary formats.

Also, talk briefly about XML and why you should not use it (will I get
flamed for that)?  The reason is simple.  There is no language to
directly manipulate it.  Lua is fully capable of representing all data
that could be represented in XML.

For those companies who care about the licenses attached to a product
(Microsoft obviously does, based on their recent GPL commentaries... ;),
Lua's license is very acceptable.  This might be mentioned in passing.

I concur with your "Lua vs. Rolling-Your-Own" slide.  If I were to write
a language, it would almost be Lua.  :)  The only things I'd change are:

1) I'd take Edgar's "v-table" patch that he has in Sol and integrate it
into the main Lua distribution.  For lots and lots of Lua objects that
are of the same type (say, some bad guy in a game), the single table
attachment is much more memory friendly (just 4 bytes extra instead of a
minimum of 24 + 160 bytes for the new table).
2) I'd change the default number type from double to float.  Yeah, yeah,
I know I can #define it, but the truth is, it doesn't work in Lua 4.1
Alpha.  :)  (Amped crashes...)
3) I'd also have a 32-bit integer type in the language.  Doubles are
quite capable of representing 32-bit integers, but the extra 4 bytes for
storage for EVERY SINGLE VALUE in Lua is not that cool in a tight memory
environment.  A 32-bit integer would handle most game scores.
4) I'd also have a 32-bit void pointer type.  Amped doesn't use Lua
userdata, because the cost of the allocation and subsequent garbage
collection for a simple pointer passthrough (C->Lua->C callback) is too
high.  Instead, it takes advantage of the precision of the double type
to pass memory pointers to Lua and back up to C.

In your slide "Basic Language Features," it is my understanding that Lua
4.1 has a register based VM now but operates with C in a stack based
setup.  Am I wrong?

I also can't stress how EASY it is to bring up content teams with the
language format.  It is so beautiful to be able to provide the language
to an artist or designer and say, "Make a data format that fits what you
need."  And they can just do it.

In the slide "Nil, Numbers & Strings," don't get bitten by the automatic
conversion from strings to numbers and vice versa.  We were just hit by
this in a major way.  Almost all occurrences of lua_isnumber() have now
been replaced with lua_rawtag() == LUA_TNUMBER.

In "Functions - First Class Objects": I don't believe anything in Lua is
reference counted.  If it was, it would be easier to replace the mark
and sweep garbage collector.

Thanks,
Josh