Re: table library changes (was Re: table.new in 5.3?)

[Tim Hill <drtimhill@...>]

On Nov 27, 2013, at 1:11 PM, Philipp Janda <siffiejoe@gmx.net> wrote:

> 
>> 4. Everything except 'sort' can be done (almost) as efficiently (and to taste) in Lua. It would be useful to have 'sort' surfaced in the C API as well as a Lua library function.
> 
> What makes `table.sort` different? Certainly you can implement an efficient sort routine in Lua (and it has been done before[1]). I'd say `table.unpack` and maybe `table.concat` are the only functions in the table library deserving a C implementation.

I really don’t agree with this, as I think it is based upon two false assumptions: (a) the ratio between Lua code performance and C code performance is pretty much the same across different platforms/architectures, and (b) the internal implementation of tables is fixed and well-known.

When anyone makes a statement such as “table.insert() is as fast in Lua as in C", they are really saying “the current implementation of table.insert() is not much faster than a Lua version”. If you move to a minimalist set of C functions for tables and rely on Lua code for the rest, you are locking into a particular implementation, since your Lua functions will RELY upon performance assumptions about the existing implementation. If, otoh, you keep the current model of a reasonably complete table library, then you leave open the option of significantly re-working the table implementation down the road without breaking existing code (or crippling it’s performance).

Of course, you could keep “secret” which APIs are in C and which are in Lua, which would satisfy the need for forward compatibility, but my reading is that people are trying to emasculate the core table library and replace it with some sort of source-code library of “helper” functions that can be hacked as desired. Chaos ensues...

I’m also not clear on some of the other reasoning in this thread. It has been floated that calling C functions is inefficient, and so having Lua code do table operations is better. But surely most of that code will end up making C function calls anyway, and probably many such calls (for example, to rawget/rawset items in the table) .. and so you end up with MORE not LESS calling of C code. If there IS a significant overhead for transitioning from Lua to C, then to my mind that argues for a richer set of high-level functions (like sort, merge etc etc) that try to get the most work from each such Lua-to-C transition.

Besides, perhaps it’s heresy to say it, but if Lua code is indeed as fast as the C code for some of the table primitives, then perhaps they are in need of some significant optimization?

—Tim