Re: Is "scripting" truly Lua's future?

[Norman Ramsey <nr@...>]

My own contribution to Lua's 'essay week':

I'm a dinosaur; I was raised with sh scripts that called programs like
sed, grep, and awk.  This worked well for a long time.  I tried perl a
few times, but I never liked the perl aesthetic (tmtowtdi), and I
*really* didn't like it that every perl script I ever wrote
spontaneously stopped working after a year or two.

Starting around February 2007 I wrote two fairly significant projects
as 'Lua standalone'.  One was the mail system I mentioned at the Lua
workshop; the other was a tool to manage timely review of the 120
papers submitted to ICFP 2007.  These experiences made me wish to stop
writing Lua scripts and start writing shell scripts.   With the
standard libraries, it just doesn't work---critical pieces are not
there.  But it didn't take me long to figure out not much more is
needed.

Here are some of the key pieces that make lua standalone a great
replacement for shell scripts:

  lhf's posix library

  A function os.capture that does the equivalent of shell backquoting,
  in shell-speak `...` or $(...).  It is 10 lines of Lua but requires
  io.popen.

  A function os.quote that takes a Lua string and quotes it using
  Bourne shell conventions, so that I can easily use string.format to
  make commands that I can pass to os.execute.  This one is even
  shorter:
    local quote_me = '[^%w%+%-%=%@%_%/]' -- complement what doesn't need quotes

    function os.quote(s)
      if s:find(quote_me) or s == '' then
        return "'" .. string.gsub(s, "'", [['"'"']]) .. "'"
      else
        return s
      end
    end

I have a handful of others, like an os.execv, which takes a list,
quotes every element, and then uses os.execute(table.concat(args, ' ')).
But really just the three things above carry me a long way.
Lua's string-handling facilities are far superior than anything I can
get with sed or awk, and likewise Lua's tables.


I think there's an interesting analogy to be drawn with Linux.
Linux + Gnu tools add up to something, but you have to do everything
yourself.  On the other hand, with a Linux *distribution*, you have
something that is powerful out of the box, yet not (always)
overwhelmingly complex.  The distribution is a hard thing to design,
because the designer should make it as simple as possible---more is
definitely not better.  And yet, it shouldn't be too difficulty to
merge in new things.   I have used five Linux distributions in the
last 14 years, and the one whose design impresses me the most is
Ubuntu: good defaults, still too many features but at least they're
decently organized, and the initial presentation is overwhelming.

Just as there are multiple Linux distributions I think it quite
reasonable that there be multiple Lua distributions whose goal is to
provide a complete programming environment in Lua.  My experience
tells me that designing a good combination of language and libraries
is a much *harder* problem than designing a Linux distribution, so I
expect it may take some years for really good candidates to emerge.
But the results won't be all about the Lua language any more than
Ubuntu is all about the Linux kernel.  I am looking forward to seeing
what the brave pioneers can come up with.  (For an example of a small
but interesting and well-designed library for C programmers, see Dave
Hanson's book 'C Interfaces and Implementations'.)

Steve Donovan alluded to tcl/tk, and for those of us who remember the
agony of X programming in those days, it has always been clear that
tcl/tk succeeded *because* *of* tk and *in* *spite* *of* tcl.
It is regretful that no clear GUI kit has emerged, but I think this
situation is an inevitable consequence of the larger world in which we
find ourselves: there is no clear GUI kit for C/C++ either.  QT, GTK,
and wxwidgets all have their merits, but none of them approaches the
simplicity of tk, and even tk doesn't approach the simplicity of Lua.
And while in 1990 we were all thrilled to have a simple GUI toolkit
with just a handful of widgets, in 2008 and beyond I don't think GUI
designers are willing to settle for simplicity any more---there are
too many users, and to each segment of the user community there is
some 5% of a complex design which for them is absolutely critical,
even though nobody else cares about it.  (The numbers are made up, and
I am parroting Joel Spolsky here, but I really do believe that the GUI
problem is inherently not solvable in the Lua Way.)

Steve writes:

 > Maybe one needs explicit type annotations for the public interface
 > of modules

It is interesting that three of the talks at the Lua workshop (mine,
Jim Whitehead's, and Terry Moore's) all presented explicit type
annotations for public interfaces.  In two of the three systems, there
is some actual type checking going on at run time.  I would love to
see Lua pushed in this direction, and I have had a student work on
ML-style type *inference* for Lua---but the problem proved more
difficult than we had ever dreamed.

In summary:

  * With lhf's posix library and another 20 lines of Lua, you need
    never write another sh or perl script again.  Lua standalone is
    terrific.

  * I hope some really good designers and engineers are thinking about
    different 'Lua distributions' which will include the language and
    some harmoniously designed libraries.  Let many flowers bloom!

  * I hope the future of Lua will include some (optional!) ways of
    *specifying* interfaces that the community can agree on, and that
    we will eventually have not just run-time checking but some
    compile-time checking for compliance with these specifications.


Norman