Re: state of the Lua nation on resource cleanup

[Mark Hamburg <mark@...>]

A series of comments though without the organization that John has  
brought to bear.

* Prompt resource finalization is a common problem and one worthy at  
least of consideration. Mutex locks, database transactions, and open  
files all benefit from prompt release and some times correctness may  
be on the line. __gc based finalization should be a backstop for those  
cases where analyzing the control flow is too difficult.

* The problems with pcall-based solutions are legion. Technically they  
are correct, but they force the introduction of extra levels of  
function scoping thereby interfering with break and return. They also  
tend to promote more frequent creation of temporary functions which  
then need to be garbage collected. This hurts performance.

* That said, pcall can be made to work. The coroutine issues can be  
addressed by using a coroutine-based implementation of pcall or by  
using something like Coco. pcall comes with expenses as outlined  
above, but in general a new mechanism is not necessarily needed.

* My solution to appendud looks something like the following:

	function appendud( tab, ud )
		tab:update_do( function()
			ud:with_lock_do( function()
				for i = 1, #ud do
					tab[ #tab + 1 ] = ud[ i ]
				end
			end )
		end )
	end

This depends on objects offering routines like update_do and  
with_lock_do rather than or in addition to the bracketing routines.  
The implementation of such routines gets a bit tricky, but could  
probably be aided with some library functions.

	local function pcall2call( success, ... )
		if not success then
			error( ( ... ) )
		else
			return ...
		end
	end

	local function _finish_update( self, ... )
		self:endupdate()
		return pcall2call( ... )
	end

	function Tab:update_do( ... )
		self:beginupdate()
		return _finish_update( self, pcall( ... ) )
	end

pcall2call should really be a library function and should do the  
appropriate scope level adjustment on error reporting.

We could generate this boilerplate using something like:

	Tab.update_do = makeWrapper( "beginUpdate", "endUpdate" )

(Though see my earlier note about wanting to add support for the  
syntax obj:[ msg ]() as shorthand for obj[ msg ]( obj ) with the  
guarantee that obj gets evaluated only once.)

* But note that my implementation of appendud generates two closures  
each time it is invoked.

* The above pattern can be continued but it really proliferates if we  
need things like tentative_pushback_do so that we can write addfriend (http://lua-users.org/lists/lua-l/2008-02/msg00243.html 
) as:

	function user:addfriend( newfriend )
		self.friends:transactional_pushback_do( newfriend, function()
			database:addfriend( self:getname(), newfriend:getname() )
		end )

This is implemented using:

	local function _finish_pushback( self, success, ... )
		if not success then
			self:popback()
			error( ( ... ) )
		else
			return ...
		end
	end

	function list:transactional_pushback_do( value, ... )
		self:pushback( value )
		return _finish_pushback( self, pcall( ... ) )
	end

But who is really going to write all of those wrappers up front?

* The obvious place to hook this logic in is at roughly the same place  
where upvalues get closed. One caveat in that regard, however, is that  
it probably won't do the expected thing in the case of a tail call  
unless it moves scope closures up a level in such cases.

* Explicit syntax for creating scope-exit logic seems valuable both  
for being clear about what is going on and so that the compiler can  
avoid any overhead of checking for such logic when it isn't needed  
(though note that moving things up the stack on tail calls would make  
some sort of check required everywhere).

* Question: Should scope logic detect coroutine yields and resumes?

* If Lua were ever to support non-local exits from functions, it needs  
some sort of unwind protection mechanism as an alternative to pcall  
since it isn't entirely clear how a pcall-based system should function  
when handling a non-local exit -- e.g., does pcall catch it and if so  
what does it return? (I have some ideas, but they aren't fleshed out  
and I would hardly call them obvious.)

 I'm not prepared at this time to endorse specific syntax or  
implementation, but I would concur with those who feel that this is an  
area Lua needs to consider extension for the sake of program clarity,  
correctness, and performance.

Mark