Re: Abusing Lua iterators for fun and profit

["Soni L." <fakedme@...>]

On 05/02/15 06:35 PM, Sean Conner wrote:

  Some background:  At work, I'm in the process of writing some regression
tests for our product.  We have a text file describing all our test cases. 
It's a tab-delimeted file---each line describes a test, and each field on a
line is a value to use.  As an example:

#id	use-a	use-b	datum1	datum2
1.0001	false	false	polyphyletically	pursue
1.0002	false	true	penchute	dentata
1.0003	true	false	advectitious	flinkite
1.0004	true	true	hansel	galvanocontractility

  I would like to use a construct such as:

	for testcase in next_test_cases("master-list") do
	  -- process test case
	  -- here, just print out the data
	  print(
		testcase.id,
		testcase.use_a,
		testcase.use_b,
		testcase.datum1,
		testcase.datum2
	  )
	end

  So, I need an interator.  Not a problem.  But here is my question. 
According to the Lua documenation:

	A for statement like

	     for var_1, ···, var_n in explist do block end

	is equivalent to the code: 

	     do
	       local f, s, var = explist
	       while true do
	         local var_1, ···, var_n = f(s, var)
	         if var_1 == nil then break end
	         var = var_1
	         block
	       end
	     end

	Note the following:

	* explist is evaluated only once. Its results are an iterator
	  function, a state, and an initial value for the first iterator
	  variable.
	
	* f, s, and var are invisible variables. The names are here for
	  explanatory purposes only.

	* You can use break to exit a for loop.

	* The loop variables var_i are local to the loop; you cannot use
	  their values after the for ends. If you need these values, then
	  assign them to other variables before breaking or exiting the
	  loop.

  Okay, so I write my iterator:

	function next_test_case(cases)
	  local nextline,state,line = io.lines(cases)
	  return function()
	    line = nextline(state,line)
	    if not line then
	      return nil
	    end
	
	    local nextfield,fs,fv = line:gmatch("[^\t]+")
	    local res  = {}
	    res.id     = nextfield(fs,fv)
	    res.use_a  = nextfield(fs,res.id)
	    res.use_b  = nextfield(fs,res.use_a)
	    res.datum1 = nextfield(fs,res.use_b)
	    res.datum2 = nextfield(fs,res.datum1)
	    return res
	  end
	end

  But in playing around, I notice that io.lines() and string.gmatch()
actually only return a function; they do *not* return a state and an initial
variable value.  Assuming that, I can rewrite my iterator as:

	function next_test_case(cases)
	  local nextline = io.lines(cases)

	  return function()
	    local line = nextline()
	    if not line then
	      return nil
	    end       
	          
	    local nextfield = line:gmatch("[^\t]+") 
	    return {  
	      id     = nextfield(),
	      use_a  = nextfield(),
	      use_b  = nextfield(),
	      datum1 = nextfield(),
	      datum2 = nextfield()
	    }
	  end
	end

  It's (in my opinion) cleaner and easier to read.  Now my question:  can I
rely upon this behavior of io.lines() and string.gmatch() to only return a
single function?  I tested it on Lua 5.1, 5.2, 5.3 and LuaJIT, and in each
case, only a function was returned.  Am I relying upon an implementation
detail?  Should I use the first version, where I pass in the state and value
each time, to be "safe"?

  -spc 

According to the manual: http://www.lua.org/manual/5.3/manual.html#pdf-string.gmatch

"Returns an *iterator function*"

I believe it is safe to assume "iterator function" means it returns only a function that returns the next match or nil at the end. (If it said just "an iterator", THEN it wouldn't be safe to assume so)

Same goes for io.lines and a few others:
http://www.lua.org/manual/5.3/manual.html#pdf-io.lines
http://www.lua.org/manual/5.3/manual.html#pdf-file:lines

See also: http://www.lua.org/manual/5.3/manual.html#3.3.5

"The generic for statement works over functions, called iterators. On each iteration, the iterator function is called to produce a new value, stopping when this new value is nil."
[...]
"explist is evaluated only once. Its results are an *iterator function*, a state, and an initial value for the first iterator variable."

The manual for ipairs also talks about an iterator function: http://www.lua.org/manual/5.3/manual.html#pdf-ipairs
"Returns three values (an *iterator function*, the table t, and 0)"

-- 
Disclaimer: these emails are public and can be accessed from <TODO: get a non-DHCP IP and put it here>. If you do not agree with this, DO NOT REPLY.