Re: new "empty" value/type in Lua?

[Philipp Janda <siffiejoe@...>]

Hi!

Am 28.06.2013 06:35 schröbte Tim Hill:
> Good points from all, but I don't think I can agree.
>
> I think the point here is that there are different degrees/scopes to standardization:
> 1. At the bottom, there is the "per project" conventions. The work-arounds I mentioned, as well as Andrews's, fall into this bucket. Yes, a single empty table could act as a flag WITHIN a single Lua project or program, but...
>
> 2. Next, you have informal conventions that are often embodied in "best practices" or accepted ways to do things. Much of the Lua module system works like this.
>
> 3. Next you have the standard runtime/library.

There already is a level-3 solution to your problem: Have a look at 
`table.pack` which supports arrays containing nils.

> 4. Finally, you have the enforced conventions of the language itself.
>
> To my mind you try to stay as far "up" that stack as you can. So why
> am I arguing for something all the way down at #4? Because at every
> other level any solution is scoped wrongly. Yes, for a given project I
> can use an empty table as a special placeholder, or (as per the other
> suggestion) I can use metatables to "hack" sparse arrays (though this is
> harder than it might at first appear, at least to do efficiently).

sparse.lua:
-----------8<-------------

    local setmetatable = setmetatable

    local function ipairs_iter( state, var )
      var = var + 1
      if var <= state.n then
        return var, state[ var ]
      end
    end

    local sparse_meta = {
      __len = function( self )
        return self.n
      end,
      __ipairs = function( self )
        return ipairs_iter, self, 0
      end,
    }

    return function( n )
      return setmetatable( { n = n }, sparse_meta )
    end

-----------8<-------------

Did I miss something?

In your case this probably is unnecessary anyway, because the user 
usually knows how many values to expect from a database operation -- 
he/she has written the SQL statement after all ...

> You see, when you move down that stack above, you are going from a
> single local project to a broader and broader scope, until at #4 you
> reach everyone.

Why do you want to reach everyone? So far the "empty" type is only 
useful under very special circumstances (to map the C/SQL NULL value to 
Lua), and you don't need a new type for that.

Then there are other implications to consider:
Should `local x` contain nil or empty? Should `table.pack( nil, nil )` 
translate the nils into empties or not? What about `{ ... }`? Should 
`table.unpack( t )` translate empties into nils? Should accessing an 
empty field invoke __(new)index metamethods? `t[ 1 ] or 0` won't work 
anymore. You would need two conditions to check for valid table 
elements. There will be sparse arrays where you *don't* want to use 
empty anyway, e.g.: `{ n=1000, [499]=1, [500]=2, [501]=1 }` ...

> To take a canonical example, what is a clean way to surface a
> databaseresult set that might include NULLs? In a standard library that many
> people are likely to use? Yes, there are a dozen ways, but I don't
> regard any of them as clean, and when I analyzed the problem, it really
> came down to this lack of a storable empty value and the overloading of nil.

Use either one of `nil`, lightuserdata NULL, or some other special 
sentinel value. The question is: How would your "empty" type help?

empty type:
    local res = sql:getrow()
    local id,name = table.unpack( res )   -- or res[ 1 ], res[ 2 ]
    if name ~= empty then ... end

nil (without sparse array metatable):
    local res = sql:getrow()
    local id,name = table.unpack( res, 1, 2 )
    if name ~= nil then ... end

nil (with sparse array metatable):
    local res = sql:getrow()
    local id,name = table.unpack( res )
    if name ~= nil then ... end

some sentinel value:
    local res = sql:getrow()
    local id,name = table.unpack( res )
    if name ~= sql.NULL then ... end   -- or name ~= sql.empty

generic:
    local res = sql:getrow()
    local id,name = table.unpack( res, 1, 2 )
    if not sql.is_null( name ) then ... end

> And, to be honest, I *do* think Lua lacks the concept of empty; the
> current Lua documentation has to kind of wriggle a bit when explaining
> arrays precisely because you cannot have an empty array element.

You cannot have an empty array element in C. In Lua you *can* have empty 
array elements, but Lua cannot automatically keep track of the array 
size for you if you do.

> --Tim

Philipp