Re: Empty? No. Array? No. Has? Yes.

[Jay Carlson <nop@...>]

On Jul 5, 2013, at 2:54 PM, Andrew Starks wrote:

> On Thursday, July 4, 2013, Jay Carlson wrote:
> 
>> I'd be willing to contemplate a uniform void value, but only if it were true-ish. We already went through nil->false, and I'm bored. I want new, interesting bugs rather than boring old NullPointerExceptions.
> 
> Jay, a question, although only out of curiosity (I've returned to "Lua is fine" mode):

One good thing about Lua language discussion is that I can chase weird designs for problems and solutions I don't understand. The list is smart, and Lua *is* fine--I am very productive in multiple PUC-Rio Lua versions, and in any case the multiple macro systems allow me to fix domain syntax indignities if I'm feeling contrary.

The greatest pain I have is that it is easier to write UTF-8-unsafe code than to write UTF-8-safe code. Untyped strings are a weakness of almost all languages, but I'm psycho about transforming out of the string domain and into the struct domain as quickly as possible. Strong typing is for weak minds--the weak minds down the hall who keep passing me HTML entities in text/plain or XML content. And my mind is pretty weak when I return to code after a few years.

> Would some kind of mechanism to discover the existence of a variable suffice? That is, the mechanism returns a boolean, not a void / etc.

I apologize for nitpicking:

I don't think you're asking the question you really mean. Whether a variable exists is not particularly interesting.

For local variables, the compiler can determine the existence of a variable. In fact your editor could do this, and some do. Local variables are names for slots, and the slot always exists. The slot may contain true, false, nil, or any other value. But it exists.

Global variable references are syntactic sugar for gets and sets on a per-chunk table (called _ENV in 5.2). In many ways it's just another local variable. Asking whether _ENV itself exists is not interesting.

It is very nice that Lua only has one aggregate type. But it does mean that changes to many different kinds of language behavior reduce to the behavior of the table type.

> I thought Lua did keep nils in tables around internally, and just didn't provide a way to determine the difference between "not there" and "not there, as far as you know," As it does with local variables. 

I agree with Tim Hill: nil assignments do not distinguish programmer intent in a couple situations they expect it to.

Let's take the following code, applied to an array-like table:

  i = #t
  t[i] = nil

This could mean three things:

  1) Shorten t by 1 by deleting the last element of the sequence. It becomes shorter.[1]

  2) Make future references to t[i] return nil; the length is not important. (It's just used like a table.)

  3) Make the sequence member t[i] return nil; that is, store nil but take no action on length.

Note that metatables can help for a fixed length array; __len can always return (say) 256, always producing behavior 3. But there can't be a shrinkable array.

So, for total overkill, I introduce new syntax to make intent clear:

  i = #t
  delete t[i]

This definitively means case 1: shorten the sequence. I am not sure what the correct semantics are for any element other than t[#t]. But in the t[#t] case, after that code block #t == i-1. This means there must be a hidden length variable somewhere, akin to the old .n. Let's call it t.# for now. (We can make it an lvalue too.)

The hidden t.# must be created when "t[i]=nil" is evaluated, if i==#t, and t[i] previously contained a non-nil value. This covers the situation where #t would decrease in Lua 5.2 but should not if our intent is to store a nil element, not delete it.

I am not certain what should happen to t.# in other cases. One obvious thing for "delete t[2]" to do is shift t[3], t[4],... down. But that requires core Lua to magically detect array-like tables with positive integer indices and perform an O(n) operation from the table library. That doesn't sound good.

There are non-array uses for "delete" assuming it triggers a separate metatable function. In particular, it allows environments and struct/class implementations to blow up on undeclared elements, yet still allow a distinction between storing s.callback=nil and undeclaring s.callback.

Finally, note that this is quite conservative in several ways. It does not create any new value types, nor does it affect existing non-array code. The hashtable implementation does not change. There is still no way of writing "if undefined(t, 'foo')"; any such protocol is by private agreement with a class. The only thing significantly changed is the notion of sequence length, which only indirectly allows an undefined() predicate in the case of sequences.

Jay

[1]: Note that in the present implementation a sequence represented by t may get *much* shorter after the t[#t]=nil case:

> t={1,2,3,nil,nil,6,7,8}
> return #t
8
> t[#t]=nil
> return #t
3