Re: ANN: Serpent: Lua serializer and pretty printer

[Eric Wing <ewmailing@...>]

My two cents.


> They used to be sorted. The problem is that they were sorted with
> something like this:
>
>           table.sort(o, function(a,b) -- sort keys; numbers first
>             if tonumber(a) == tonumber(b) then return tostring(a) <
> tostring(b)
>             else return (tonumber(a) or 0) < (tonumber(b) or 0) end end)
>
> to get numbers first and sort the rest and it was painfully slow
> (probably 30% slower than the current version on the benchmark I
> included). It is a huge penalty to pay for sorted keys, although in
> many cases it may not matter that much if you care about pretty
> printing mostly. I decided to remove it for this reason, although it
> may still leave as an option.

Sort as an option is nice. In most of my cases, I'm more concerned
about speed. So non-sorting doesn't bother me at all.


>> (10) Dumps of bytecode are probably not useful for pretty printing
>> unless perhaps you were to decompile the bytecode.  More useful info
>> may be found in debug info (including sometimes a file name with
>> source code), in cases debug.* is even permitted.   Some such things,
>> however, are probably outside the scope of a simple dumping module.
>> Bytecode may still have limited uses though in serialization (transfer
>> code between Lua states?).
>
> I was thinking about using debug.* to provide original function
> name/file in comments and to capture upvalues, but this was going
> against my 90% rule (I was targeting 90% of use cases with the
> existing code).
>

Much of my interest is in serializing parts of my Lua state to
something that can be read back in. I'm thinking about things like
save game files or things to help suspend/resume processes on devices
like iOS.

At the basic level, I would like simple toLuaCode serialization that
can be read in by loadstring. But I also have a secondary, more
complicated objective of wanting to support iOS. Because of App Store
rules, loadstring may or may not be available depending on how one
uses it and Apple interprets the rules. If I play it conservatively, I
assume I don't have loadstring, but then I might be able to fallback
to something LPeg and Leg to read in tables and basic types.

But thinking longer term and more generally to other platforms,
assuming I can use loadstring, being able to serialize everything and
reload it is terrific. But I assume that not everything can be easily
dumped to Lua script code (thinking about how string.dump works), so I
still consider bytecode useful.

If there was a way to get this additional stuff into actual code
instead of bytecode using the debug library, I think that would be
very cool. I would love to have that ability as an auxiliary function
if it was something that you didn't feel appropriate in the primary
APIs.


>> (12) In "nil values are included when expected ({1, nil, 3} instead of
>> {1, [3]=3})", I'm not sure what "expected" means given the
>> undefinedness of # with holes.  I'd actually expect the latter format
>> deterministically for sparse arrays.
>
> "Expected" in the sense of being "reported" by #t. if #t return 3,
> Serpent will serialize it as {1, nil, 3}, rather than {1, [3]=3}. This
> should match the original representation closer (although it probably
> doesn't matter in most cases). #{1, nil, 3}==3, but #{1, [3] = 3} ==
> 1. The array content is the "same", but #t as reported is different,
> so, I was trying to match that.

If I recall, the latter format {1, [3]=3} will make things harder for
the Lua compiler to optimize for size of the array and take a
non-optimal code path to fill the array. From a performance
standpoint, I would expect Lua to do much better with {1, nil, 3}. In
either case, I thought the #t will be somewhat unpredictable depending
on how the table was manipulated through its life. And I'm worried
that even if you could figure out how to preserve it, for complex
enough tables, I'm worried the performance will be terrible. I
personally accept that this value may change and rather have better
performance.

Thanks,
Eric
-- 
Beginning iPhone Games Development
http://playcontrol.net/iphonegamebook/