RES: Userdata

[André de Leiradella <leiradella@...>]

I'm sure some of you already faced the problem described bellow. Which's
the best way to implement a Lua interface to a C library when one have
to deal with callbacks?

Thanks!

-----Mensagem original-----

Hi All,

As you already know, I'm writing a Lua interface to the LibVNCServer-0.6
library. I had to make a decision as how to use the full userdata type
provided by Lua to implement the interface and I'd like to have a
opinion from others.

Here's how I declare my userdata:

-----------------------------------8<-----------------------------------
/* The client structure. */
typedef struct {
	/* Client data at the beginning of the structure so we can cast
	   clientUserData to rfbClient and vice-versa. */
	rfbClient	client;
	/* The Lua state so we can call Lua functions from inside the
	   callbacks. */
	lua_State	*L;
	/* The references to Lua callback functions. */
	int		handleCursorPosRef;
	int		softCursorLockAreaRef;
	int		softCursorUnlockScreenRef;
	int		gotFrameBufferUpdateRef;
	int		getPasswordRef;
	int		bellRef;
	/* Mouse buttons states. */
	int		buttonMask;
	/* The automatic delay issued in every mouse and keyboard
	   events. */
	long		delay;
	/* The password returned by the getPassword callback. */
	char		password[32];
} clientUserData;
-----------------------------------8<-----------------------------------

And here's how I implement the C function called by Lua to get the
userdata:

-----------------------------------8<-----------------------------------
/* Creates a new VNC client, with all callbacks cleared and a automatic
   delay of 0 sec. */
static int L_rfbGetClient(lua_State *L) {
	const char	*progName;
	int		bitsPerSample, samplesPerPixel,	bytesPerPixel;
	clientUserData	**userData, *ud;
	int		argc;
	
	/* Program name is "lua" if not specified. */
	progName = luaL_optstring(L, 1, "lua");
	/* Default to 24 bpp. */
	bitsPerSample = luaL_optint(L, 2, 8);
	samplesPerPixel = luaL_optint(L, 3, 3);
	bytesPerPixel = luaL_optint(L, 4, 4);
	/* Push a new full userdata on the stack. */
	userData = (clientUserData **)lua_newuserdata(L,
		sizeof(clientUserData *));
	/* Initialize the client structure. */
	argc = 1;
	*userData = (clientUserData *)rfbGetClient(&argc,
		(char **)&progName, bitsPerSample, samplesPerPixel,
		bytesPerPixel);
	/* Hack: realloc structure to a full clientUserData. */
	ud = (clientUserData *)realloc(*userData,
		sizeof(clientUserData));
	if (ud == NULL) {
		rfbClientCleanup((rfbClient *)*userData);
		lua_pushliteral(L,
			"not enough memory for rfb client data");
		lua_error(L);
	}
	*userData = ud;
	/* Fill in the rest of the structure. */
	ud->client.HandleCursorPos = handleCursorPos;
	ud->client.SoftCursorLockArea = softCursorLockArea;
	ud->client.SoftCursorUnlockScreen = softCursorUnlockScreen;
	ud->client.GotFrameBufferUpdate = gotFrameBufferUpdate;
	ud->client.GetPassword = getPassword;
	ud->client.Bell = bell;
	ud->L = L;
	ud->handleCursorPosRef = LUA_NOREF;
	ud->softCursorLockAreaRef = LUA_NOREF;
	ud->softCursorUnlockScreenRef = LUA_NOREF;
	ud->gotFrameBufferUpdateRef = LUA_NOREF;
	ud->getPasswordRef = LUA_NOREF;
	ud->bellRef = LUA_NOREF;
	ud->buttonMask = 0;
	ud->delay = 0;
	/* Set metatable. */
	luaL_getmetatable(L, rfbClientHandle);
	lua_setmetatable(L, -2);
	/* Return client data. */
	return 1;
}
-----------------------------------8<-----------------------------------

As rfbGetClient returns a malloc'ed area enough to hold a rfbClient
structure, I realloc it to the size of my userdata to get enough space
to hold the additional information like references to callback functions
and a pointer to the Lua state. I need this because I need to handle
LibVNCServer callbakcs calling the corresponding registered Lua
callback, and for that I need both the references to them and the Lua
state.

After initializing the rest of the structure, the userdata's metatable
is set and it's returned. The userdata is a boxed pointer to my
clientUserData structure.

The __gc method that frees the userdata from memory is:

-----------------------------------8<-----------------------------------
/* Clean up the rfbClient structure. */
static int L_rfbClientCleanup(lua_State *L) {
	clientUserData	*userData;
	
	/* Check parameter. */
	userData = getClientUserData(L, 1);
	/* Unreference callback functions. */
	if (userData->handleCursorPosRef != LUA_NOREF)
		luaL_unref(L, LUA_REGISTRYINDEX,
			userData->handleCursorPosRef);
	if (userData->softCursorLockAreaRef != LUA_NOREF)
		luaL_unref(L, LUA_REGISTRYINDEX,
			userData->softCursorLockAreaRef);
	if (userData->softCursorUnlockScreenRef != LUA_NOREF)
		luaL_unref(L, LUA_REGISTRYINDEX,
			userData->softCursorUnlockScreenRef);
	if (userData->gotFrameBufferUpdateRef != LUA_NOREF)
		luaL_unref(L, LUA_REGISTRYINDEX,
			userData->gotFrameBufferUpdateRef);
	if (userData->getPasswordRef != LUA_NOREF)
		luaL_unref(L, LUA_REGISTRYINDEX,
			userData->getPasswordRef);
	if (userData->bellRef != LUA_NOREF)
		luaL_unref(L, LUA_REGISTRYINDEX, userData->bellRef);
	/* Free memory. */
	rfbClientCleanup((rfbClient *)userData);
	/* No return values. */
	return 0;
}
-----------------------------------8<-----------------------------------

I call rfbClientCleanup casting my userdata to a rfbClient structure, as
rfbClientCleanup calls free to release memory, it will handle the
additional space that I previously claimed with realloc.

And here's one of the callback methods that call the corresponding Lua
function:

-----------------------------------8<-----------------------------------
/* Callback functions, they just call the registered Lua callback
   function for the object, if any. */
static rfbBool handleCursorPos(rfbClient *client, int x, int y) {
	clientUserData	*userData = (clientUserData *)client;
	rfbBool		result;
	
	/* Call lua callback function. */
	if (userData->handleCursorPosRef == LUA_NOREF)
		return FALSE;
	lua_rawgeti(userData->L, LUA_REGISTRYINDEX,
		userData->handleCursorPosRef);
	lua_pushvalue(userData->L, 1);
	lua_pushnumber(userData->L, x);
	lua_pushnumber(userData->L, y);
	lua_call(userData->L, 3, 1);
	/* Get the result. */
	result = lua_toboolean(userData->L, -1) ? TRUE : FALSE;
	/* Empty the stack. */
	lua_settop(userData->L, 0);
	return result;
}
-----------------------------------8<-----------------------------------

Now it's easy to see why I need a pointer to the Lua state inside my
userdata.

I don't really have a problem here. Besides the problem I've just
reported about multiple arguments, everything is working just fine. But
I'm not confortable in having to realloc the memory area returned by
rfbGetClient to the size of my clientUserData, but I don't see any other
way to implement the callbacks.

One problem that might happen depending on LibVNCServer internals is if
rfbClient holds a pointer to itself or to one or more of its fields. In
this case, those pointers would point to nowhere land after my realloc.

Now what I'd like to hear from the list is: is there a better way to
implement this kind of userdata when I have callback functions and I
need to call Lua functions from inside them? Of course having a global
lua_State declared which I could use inside the callback functions is
not an option.

Thanks in advance,

Andre de Leiradella