Possible Lua enhancements 2: Piping

[Mark Hamburg <mark@...>]

The following ideas started with studying LINQ, but LINQ is pretty  
clearly just a piping model for data together with some useful syntax  
in C# and VB and ways to delay evaluation long enough to allow for  
translation into SQL or other mechanisms. I'm not going to address how  
to get to those latter enhancements but pipes are good as any Unix  
programmer will tell you and as Diego Nehab demonstrated in LTN 12 (http://lua-users.org/wiki/FiltersSourcesAndSinks 
). So, here are two proposals for how to add pipes to Lua's syntax:

-----------------------------
Option 1: Left-to-right
-----------------------------

	source >> filter >> filter >> sink

(I'm not set on >> but I'm reasonably certain that it's available.)

The meaning of this is that when we encounter expr1 >> expr2 we treat  
this as ( expr2 )( expr1 ). This means that if expr1 returns multiple  
values then we pass multiple values to expr2. The one potential  
surprise here is with order of evaluation since I believe Lua will  
evaluate expr2 before it evaluates expr1. That could probably be  
addressed as follows:

	expr1 >> expr2

becomes

	function( ... ) return ( expr2 )( ... ) end ( expr1 )

It would probably be good, however, to avoid generating the extra  
closure so support at the VM level would probably be a good thing.

An example of usage would be:

	for obj in io.lines( "objectlist.dat" ) >> chunk_lines( "---------" )  
>> chunk_to_object do
		-- process obj
	end

This would take the file objlist.dat which would be assumed to consist  
of a series of Lua chunks defining objects divided by lines of the  
form "---------" and would chunk the file up based on these dividers  
and then evaluate each of the chunks to generate an object.

-----------------------------------------------------------------
Option 1b: Left-to-right with fixed parameter lists
-----------------------------------------------------------------

This is similar to the above, but whereas above chunk_lines needed to  
be a function returning a function, here we would define the syntax  
such that we would translate expressions as follows (ignoring order of  
evaluation):

	expr1 >> fn			becomes fn( expr1 )

	expr1 >> fn( expr2 )	becomes fn( expr2, expr1 )

And so forth for longer lists of expressions. The downside is that now  
one can't pass variable length argument lists to the steps along the  
way since the only variable length lists supported are for the piped  
values. The upside is that the individual steps are simpler to write  
and are less likely to involve generating transitory functions.

----------------------------------------
Option 2 and 2b: Right-to-left
----------------------------------------

I believe the following syntax comes from Haskell:

	expr1 $ expr2

is shorthand for ( expr1 )( expr2 ). This could be used along  
analogous lines to option 1 or option 1b depending on how one wanted  
to handle variable-length argument lists.

One benefit to this approach is that the order of operations for the  
naive translation now matches the reading order. But this also means  
that the reading order doesn't match as well with the data flow order  
from a piping standpoint.

------

I tend to see the right-to-left options as merely minor savings in  
parentheses and hence probably not worth the syntactic extension. I  
raised them because I figured that someone would cite the Haskell  
precedent and could perhaps make a stronger argument for it.

The left-to-right approaches, on the other hand, seem like they could  
be quite expressive as ways to modify iterators and would make a  
number of idioms more natural.

Mark