Suggested Enhancements to improve debugger support in Lua

[Dibyendu Majumdar <mobile@...>]

Hi,

Debuggers need to support the ability to 'step out' of the currently
executing function, or 'step over' a line of code that may involve
function calls etc. To do this the debugger needs to be able to tell
when the stack level has changed. Currently this cannot be done
reliably in Lua.

An unrelated problem is that of being able to catch and redirect all Lua output.

It is also necessary to prevent a user from changing the Lua hook when
the debugger is running.

Finally the debugger may need to attach additional state to Lua state
so a facility to do this is useful.

Described below are the simple changes I made in Ravi to address the
first issue. I hope these changes (or something similar) can be
adopted in Lua.

The other issues can be addressed by providing suitable APIs.


Added to CallInfo:
  short stacklevel; /* stack level, bottom level is 0 */

Added to lua_Debug:
  short stacklevel; /* Current stack level within the Lua State - base
level is 0 */

  void luaD_hook (lua_State *L, int event, int line) {
    lua_Hook hook = L->hook;
    if (hook && L->allowhook) {  /* make sure there is a hook */
      CallInfo *ci = L->ci;
      ...
      lua_Debug ar;
      ar.event = event;
      ar.currentline = line;
      ar.i_ci = ci;
      ar.stacklevel = ci->stacklevel; /* To help Debugger determine
the stack level */
      ...
    }
  }

  CallInfo *luaE_extendCI (lua_State *L) {
    CallInfo *ci = luaM_new(L, CallInfo);
    ...
    ci->stacklevel = L->ci->stacklevel+1;
    lua_assert(L->ci->next == NULL);
    L->ci->next = ci;
    ci->previous = L->ci;
    ci->next = NULL;
    L->nci++;
    return ci;
  }

  static void preinit_thread (lua_State *L, global_State *g) {
    G(L) = g;
    L->stack = NULL;
    L->ci = NULL;
    L->nci = 0;
    ...
    L->base_ci.stacklevel = 0; /* RAVI base stack level */
  }