RE: Yielding from hooks

["Bilyk, Alex" <ABilyk@...>]

>From what I can tell luaD_callhook is the gate every hook call is going through. This call unconditionally leads to luaD_call. Any attempt to yield from within luaD_call will fail as it increments nCcalls making lua_yield bail if this value is > 0. I have just tried to create a C hook function and had the same failure regardless whether it is being set from C or from Lua.

	"The problem with implementing concurrency with yielding hooks --
	aside from any technical problem with the implementation of yield in a hook
	function -- is that it would be incompatible with the actual use of
	coroutines in the concurrent processes. But perhaps no-one thinks this is a
	problem."

The main application for yielding hooks was, as you have suggested, to implement non-cooperative multitasking. I don't see why this would be incompatible with the use of coroutines. All we are talking about is which function calls lua_yield. In any case, inside hook or not, the yield would occur within the same coroutine context, so I fail to see how it's of any concern at all. Perhaps you meant OS-level threads by "concurrent processes", however, did you? I have not considered this possibility for the lack of practical interest this scenario bares in our application.

This topic has come up during beta a number of times and I remember reading that yielding from hooks will present no problem in release. It still does, apparently. We will survive without it, but I would like to hear your ideas on how to make it work. 

Alex

-----Original Message-----
From: RLake@oxfam.org.uk [mailto:RLake@oxfam.org.uk]
Sent: Monday, April 21, 2003 8:06 AM
To: Multiple recipients of list
Subject: Re: Yielding from hooks



Alex Bilyk escribió:

> Hey all,

> Correct me if I'm wrong, but one can not yeild from a hook
> (say a "line" or a "count") function in Lua 5? I wonder if
> the script below somehow violates expectations for 'yield'
> usage. I thought a while ago we were told Lua 5 release
> version will have this capability.

Here is my understanding:

You can yield from a line or count hook. However, you cannot yield from a
metamethod (except __call metamethods) or a C function. Consequently, if
you set a line or count hook in a Lua function which is called as a
callback from a C function, or is used as a metamethod, the yield will
fail. [see note below]

I think this renders the use of yield in line and count hooks impossible,
or at least incompatible with a number of standard features in Lua
(table.foreach and generators in general; string.gsub and friends;
metamethods) because sooner or later the yield will happen inside a
metamethod or callback.

In any event, I don't believe that you can do a yield from a hook written
in Lua; I believe you have to write the hook function in C for it to work.
I think that is because the Lua hook function is actually a reentry into
Lua.

As far as I can see, a call or return (C) hook function can try to yield if
it wants to; I don't believe it will generate an error but the yield won't
happen either.

I am curious as to whether anyone is seriously contemplating using yielding
hook functions (and if so, for what) because I have some (fairly technical)
ideas about how to implement these features in an alternative manner which
might be slightly more general. It seems to me that the main reason for
yielding count/line hooks is to try to implement non-cooperative
concurrency; a debugger would probably not require yielding hooks (although
it is possible to imagine a debugger which attempted to work in this
fashion). The problem with implementing concurrency with yielding hooks --
aside from any technical problem with the implementation of yield in a hook
function -- is that it would be incompatible with the actual use of
coroutines in the concurrent processes. But perhaps no-one thinks this is a
problem.

[note] I speculate that you could use call and return hooks to attempt to
suspend line or count hooks inside a callback; you would have to turn the
line/count hook off inside the call hook (and turn it back on inside the
return hook) if the function being called were a C-function. I don't think
this would capture metamethod calls, though.

I have some (fairly technical) ideas about how to implement some of these
features, if anyone cares.