Re: C API - lua_next traversal of "array" table

[Philipp Janda <siffiejoe@...>]

Am 19.08.2016 um 03:57 schröbte Duane Leslie:
> > On 19 Aug 2016, at 11:22, ThePhD <jm3689@columbia.edu> wrote:
> >
> > I don't suppose sneaking in some kind of ridiculous key name that includes null characters and other things that are vastly improbable for a real Lua user to use (something like "__detail.do.\0.not.touch\xE2\x98\xA2.size") for when I push the table and then pop the table, and using that MAYBE as a shortcut if it's present, and otherwise falling back to iterating until I hit a `nil` might be worth it...?

If you want to be able to store arbitrary strings keys in addition to 
array data in a table, you can use the number `0` as key for the length. 
In all other circumstances a well-known and convenient string key works 
just fine.

Iterating until the first nil is O(n) which is a lot more than O(log n) 
for the current `#` operator or O(1) for an explicit length field. I'd 
rather check for the existence of the length field and fix the code that 
didn't set it.


> No, just create a global weak proxy object and store your table lengths in there.  There is never a need to store "magic" data on an allocated object itself.

Storing magic data on an allocated object has the advantage that it is 
guaranteed to go away together with the allocated object. Global weak 
tables typically hog memory corresponding to their peak use until the 
global weak tables themselves are collected at `lua_close()` time:

    local N, M = 10000, 20
    local gc = collectgarbage
    local lengths = setmetatable( {}, { __mode = "k" } )
    local function count( t )
      local n = 0
      for _ in pairs( t ) do n = n + 1 end
      return n
    end

    print( "before heavy use", gc( "count" ), count( lengths ) )
    do
      local t = {}
      for i = 1, N do
        t[ i ] = {}
        lengths[ t[ i ] ] = 0
      end
      print( "peak number of arrays", count( lengths ) )
    end
    gc() gc()
    print( "after heavy use", gc( "count" ), count( lengths ) )
    for i = 1, N do
      do
        local a = {}
        lengths[ a ] = 0
      end
      if i % M == 0 then
        gc()
      end
    end
    gc() gc()
    print( "after slight use", gc( "count" ), count( lengths ) )


Typical output:

    before heavy use	27.412109375	0
    peak number of arrays	10000
    after heavy use	537.0498046875	0
    after slight use	537.0498046875	0


Philipp