Re: [BUG] call hook again when lua yield in count hook

["G.k Ray" <gzlkylin@...>]

In addition,repeated calling Call hook not occur when calling a
variadic function.

The Call hook entry point for variadic functions is different from
non-variadic functions. The Call hook entry point for variadic
functions is within the main loop of the interpreter, where the
vmfetch function has already handled CIST_HOOKYIELD callstatus.

- lua 5.4.6 lvm.c  non-variadic function Call hook entry point
1146 void luaV_execute (lua_State *L, CallInfo *ci) {
     ...
1161   if (l_unlikely(trap)) {
1162     if (pc == cl->p->code) {  /* first instruction (not resuming)? */
1163       if (cl->p->is_vararg)
1164         trap = 0;  /* hooks will start after VARARGPREP instruction */
1165       else  /* check 'call' hook */
1166         luaD_hookcall(L, ci);
1167     }
1168     ci->u.l.trap = 1;  /* assume trap is on, for now */
1169   }

- lua 5.4.6 lvm.c  variadic function Call hook entry point
1146 void luaV_execute (lua_State *L, CallInfo *ci) {
         ...
1884       vmcase(OP_VARARGPREP) {
1885         ProtectNT(luaT_adjustvarargs(L, GETARG_A(i), ci, cl->p));
1886         if (l_unlikely(trap)) {  /* previous "Protect" updated trap */
1887           luaD_hookcall(L, ci);
1888           L->oldpc = 1;  /* next opcode will be seen as a "new" line */
1889         }
1890         updatebase(ci);  /* function has new base after adjustment */
1891         vmbreak;
1892       }

G.k Ray <gzlkylin@gmail.com> 于2023年7月22日周六 12:41写道：
>
> Not related to program count, the reason is that the CIST_HOOKYIELD status was not handled when calling the call hook.
>
> A potential solution is to ensure that the luaD_hookcall is not executed during VM yielding, thereby preventing it from being invoked repeatedly.
>
>
> Fix:
> diff --git a/lvm.c b/lvm.c
> index 2b437bdf..fd52127c 100644
> --- a/lvm.c
> +++ b/lvm.c
> @@ -1165,7 +1165,8 @@ void luaV_execute (lua_State *L, CallInfo *ci) {
>        if (cl->p->is_vararg)
>          trap = 0;  /* hooks will start after VARARGPREP instruction */
>        else  /* check 'call' hook */
> -        luaD_hookcall(L, ci);
> +        if (!(ci->callstatus & CIST_HOOKYIELD))
> +          luaD_hookcall(L, ci);
>      }
>      ci->u.l.trap = 1;  /* assume trap is on, for now */
>    }
>
> Roberto Ierusalimschy <roberto@inf.puc-rio.br>于2023年7月22日 周六02:23写道：
>>
>> > A correction
>>
>> Thanks for the feedback.
>>
>> The bug seems to be caused by the iteraction between both hooks. It
>> seems to go like that:
>>
>> - The function starts: As the function is in its first instruction,
>> it calls the Call hook.
>>
>> - Then, it checks the Count hook. As its counter goes to zero,
>> it calls the Count hook *and* does not move the program counter,
>> because it does not execute the current instruction yet.
>>
>> - When the thread resumes, it is (yet) in the first instruction of
>> the current function, so it calls the Call hook again.
>>
>> -- Roberto