RE: Yielding from hooks

["Bilyk, Alex" <ABilyk@...>]

You are of course correct -- the C hook is called directly. But somehow the whole thing gets called from under luaD_call when the script is executed with standalone lua. Here is the stack 

lua_yield(lua_State * 0x002f2f40, int 0) line 378
app_hook(lua_State * 0x002f2f40, lua_Debug * 0x0012f474) line 426 + 11 bytes
luaD_callhook(lua_State * 0x002f2f40, int 2, int 37) line 173 + 13 bytes
traceexec(lua_State * 0x002f2f40) line 90 + 15 bytes
luaV_execute(lua_State * 0x002f2f40) line 418 + 9 bytes
luaD_call(lua_State * 0x002f2f40, lua_TObject * 0x002f08f0, int 0) line 313 + 9 bytes
f_call(lua_State * 0x002f2f40, void * 0x0012fa3c) line 672 + 22 bytes
luaD_rawrunprotected(lua_State * 0x002f2f40, void (lua_State *, void *)* 0x004047e0 f_call(lua_State *, void *), void * 0x0012fa3c) line 88 + 13 bytes
luaD_pcall(lua_State * 0x002f2f40, void (lua_State *, void *)* 0x004047e0 f_call(lua_State *, void *), void * 0x0012fa3c, int 48, int 32) line 403 + 17 bytes
lua_pcall(lua_State * 0x002f2f40, int 0, int 0, int 1) line 685 + 32 bytes
lcall(int 0, int 1) line 138 + 31 bytes
docall(int 0) line 166 + 15 bytes
file_input(const char * 0x002f11e2) line 172 + 25 bytes
handle_argv(char * * 0x002f11a0, int * 0x0012fc2c) line 373 + 9 bytes
pmain(lua_State * 0x002f2f40) line 415 + 16 bytes
luaD_precall(lua_State * 0x002f2f40, lua_TObject * 0x002f08d0) line 260 + 18 bytes
luaD_call(lua_State * 0x002f2f40, lua_TObject * 0x002f08d0, int 0) line 311 + 13 bytes
f_Ccall(lua_State * 0x002f2f40, void * 0x0012ff00) line 709 + 21 bytes
luaD_rawrunprotected(lua_State * 0x002f2f40, void (lua_State *, void *)* 0x00404890 f_Ccall(lua_State *, void *), void * 0x0012ff00) line 88 + 13 bytes
luaD_pcall(lua_State * 0x002f2f40, void (lua_State *, void *)* 0x00404890 f_Ccall(lua_State *, void *), void * 0x0012ff00, int 16, int 0) line 403 + 17 bytes
lua_cpcall(lua_State * 0x002f2f40, int (lua_State *)* 0x00401230 pmain(lua_State *), void * 0x0012ff70) line 719 + 33 bytes
main(int 2, char * * 0x002f11a0) line 442 + 18 bytes

So, yes, luaD_callhook does call the app_hook directly but it's done from under luaD_call and thus the problems is still there. I don't think I will be pursuing this any longer. 
Thanks for the ideas,

Alex

-----Original Message-----
From: RLake@oxfam.org.uk [mailto:RLake@oxfam.org.uk]
Sent: Monday, April 21, 2003 3:16 PM
To: Multiple recipients of list
Subject: RE: Yielding from hooks



> From what I can tell luaD_callhook is the gate every hook call is going
through.

I believe that is true.

> This call unconditionally leads to luaD_call.

Where? luaD_callhook calls the hook directly at line 173:

(*hook)(L, &ar),

> Any attempt to yield from within luaD_call will fail as
> it increments nCcalls making lua_yield bail if this
> value is > 0. I have just tried to create a C hook
> function and had the same failure regardless whether
> it is being set from C or from Lua.

I don't think I was being very clear. You would have to
install the hook from C, using the Lua API. The C hook
could then lua_call a lua function, but the lua function
itself could not call coroutine.yield; it would have to
be the hook function which called lua_yield. If you
install the hook function, whether a Lua closure or a
C closure, using the debug.sethook interface, it will
not be able to yield.

> The main application for yielding hooks was, as you have suggested,
> to implement non-cooperative multitasking.
> I don't see why this would be incompatible with the use of coroutines.
> All we are talking about is which function calls lua_yield.
> In any case, inside hook or not, the yield would occur within
> the same coroutine context,

That is the problem; it would not be the same coroutine context.
Say you have a scheduler, which starts up various threads. One of
these threads, call it T, decides to create a coroutine as a generator (G).
So the parent of T is the scheduler; if T yields, it returns to the
scheduler.
But the parent of G is T; when G yields, it returns its result to T. If
a yielding hook is active while G is running, it will yield to T, not
to the scheduler. That would probably be disastruous, which is why I think
that yielding hooks are incompatible with the use of coroutines.

> We will survive without it, but I would like to hear your
> ideas on how to make it work.

The basic idea is to let C functions tailcall Lua functions.
If that were allowed, then hookf (in ldblib.c) could tailcall
the hook function instead of lua_call'ing it, and there would be
no problem with yields except for the problem I mentioned above.

Something similar to this would work for metamethod calls as well,
albeit through a slightly different mechanism. In almost all cases,
the result of an opcode which invokes a metamethod is the value
returned by the metamethod, so if instead of lua_call'ing the
metamethod, the opcode mutated itself into an OP_CALL followed by
an OP_MOVE, the metamethod would be able to yield. I cannot see
any good reason why a metamethod would want to yeild, actually, but
it would also allow hook functions invoked within the metamethod
to yield.