OK, my englush skills are also as a secondary language (My native language is French), so sorry for the typos on this ML, once a mail is read a couple of time, and sent, it cannot be corrected like on an editable page on a website.
You may have read a mail in this list that spoke about the introduction of Lua in Mediawiki after some Wikimedian temas visited Brasil and started to evaluate it in comparison to _javascript_ (_javascript_ could have been a choice instead of Lua), and the net effect on Lua community, that Wikimedia with its choice widely helped.
Envin if now Mua has many other usages, it's because Lua was promoted by Wikimedia, and lot of people in the world discovered Lua via Wikimedia and started to evaluate it also for their own projects. So Lua has now many more uses for more specialized applications than just a few Brasilian projects (... that abandoned Lua for most of them, turning to _javascript_) and a few online multiplayer games (which also turned to _javascript_ for some of them). Without Wikimedia, Lua would have been already dead (even some of the important initial Brazilian developers of Lua have stopped working on Lua, so Lua is now mostly developed by an international community, which continues to expand thanks to Wikimedia continuing to promote it ans support it).
But if Lua does not solve its critical problems on tables, and on its memory usage, there's stil lthe danger that many projects will abandon Lua and will go to _javascript_, which is now very developed, fast, more easily integrable, and has a very wide ecosystem).
I am convinced that Lua must fix urgently its most important issues. And its defective implemention of tables is among the most important (for now there's been several different approaches, all of them failed, including a few that used a different syntax). Alternative implementation of tables are possible (probably based on what PHP uses, or based on new paradigms like B-trees, and the possibility to tune it and select one of several implementations, like whbat is possible in C++ and Java collections).
There are other interesting approaches, like what "Memcached" implements for a very fast hashing that can support very wide collections of keys (in memory, with a possibility to extend the memory with a local storage on disk using technics similar to paging if one needs more than a few gigabytes of data in the collections of keys and values).
Lua does not live in an isolated world: it will remain attractive as long as it compares sufficiently well with other "similar" languages (notably PHP and _javascript_, but also Perl, Python and Ruby, or even with the older PostScript, Forth and Lisp that also has associative tables named "dictionnaries") so that its ecosystem can support more kinds of applications and retain its genericity. Lua has already suffered from several incompatible changes in its syntax (including for its associative tables, which are one of the most important characteristics of the language).
And I am convinced that Lua could as well be supported by another VM (including one of the successful and very fast VMs initially developed for _javascript_, Python, Perl, Java, or DotNet) without even breaking its syntax.
Studying how Lua can be integrated should not even be limited to the existing implementation in C (and its related LuaC API), which in fact only focuses well the very small devices (used by a few "geeks" for very specialized projects, and that in fact are constantly hacking LuaC and patching this basic implementation in C as well as modying the Lua syntax itself, as can be seen on this Mailing list): Lua has already been forked by these small projects, and so Lua is very slow in eveloping its ecosystem of stable reusable libraries.
If Lua can succeed fact ot _javascript_, it must demonstrate a "killer" feature: a very simple syntax that can be parsed without ambiguity (unlike C++, Java or _javascript_) with:
- a pure LALR syntax and a single symbol of lookahead (this is almost the case, but the introduction of <const> is klling it)
- an universal extension mechanism that preserves the backward compatibility for the ecoluation of the language: no more any addition of new reserved keywords and the early reservation of some prefix or suffix for its reserved extensions
- a feature capabilities test for the language (using the new reserved prefix that should be used instead of <const> and <toclose>) so that developers can write Lua programs focusing different versions of the languages
- a syntax that unambiguously includes places for these extensions so that any parser could accept them even if they don't implement these extensions, and could compile and run these programs, using the feature-test described in the previous item.
And of course it must continue to offer an example implementation (for now only in portable C, but there could be community contributed alternatives, showing its implementation in pure PHP, pure Java without JNI, and _javascript_) so that this impelmentation is immune from the existing severe deficiencies (notably its associative tables that have severe bugs) and other less severe problems which are still bugs (like the string hashing, which is still not tweakable to fix security issues by allowing a stronger hashing, without needing to hack into the implementation, something which is impossible if Lua is on servers shared by diffferent applications with different goals, but which should be possible directly in Lua, like what Java allows natively)
I'm not "ranting" here. I really want to let the Lua community consider and discuss the bugs/vulnerabilities/limitations, and take a priority schedule to fix the important things, before Lua forks into multiple incompatible languages, or before Lua is simply abandonned by its early adopters (this has already occured, and most often _javascript_ was chosen to replace Lua).
And if there are official reports from some governmental CERT organisms speaking about severe vulnerabilities in Lua, classifying them as "severe", many projects that used Lua or considered adopting it will abandon the ship in fire in emergency and they won't come back again to Lua.
And I'm ready to see reports from the Google opensource developers community that has lots of excellent technicians that could help Lua (note that Google has not always created the best implementation of algorithms, ie. with the best performance; but its chosen implementation were resonnably fast, with the first goal of security
So before other "faster" algorithms that offer very modest speed advantages are chosen, they must:
- first pass the same security goals (minor speed improvement is hard to be convincing, and after just some months, these differences are no longer significant);
- the second goal is to pass the resource usage test (not CPU time, but memory or storage): a small additonal storage may be needed but it should not be multiplied by large factors, or have a polynomial or exponential ratio for large values of N;
- then only comes the goal of pure performance, first the time in O(f(N)) for several ranges of values of N (here also without polynomial or exponential speed ratio for large values of N);
- then comparing ratios: as long as speed ratios are reasonable, in 1/3 - 3/1, there's no advantage to switch an implementation if the new one has still not been widely tested (this can be done only in beta version sent for public evaluations or in a specific branch that application developers may activate and evaluate themselves)
