RE: Can module updates ever be made safe?

["Thijs Schreijer" <thijs@...>]

> Behalf Of Rodriguez, John
> Sent: vrijdag 19 oktober 2012 20:19
> Subject: Can module updates ever be made safe?
> 
> I was trying to determine if updating a module, while a Lua VM is running,
can
> ever be done safely.

Interesting question/idea

> 
> There is no problem if a script is entirely in 1 file. Just write to disk
and then
> the next load make everything current.
> 
> A problem can occur when a script is in 2 or more files.

I don't see the difference, just reload both files at once, then it is the
same as with 1 file

> Assume that some thread periodically writes new module updates to disk, 1
> module at a time.
> Assume module "A" requires module "B"
> 
> Whether the modules exist on disk, in memory or in a database, in a
dynamic
> language it is always possible that:
> 1) Module "A" has started executing
> 2) The thread updates module "B" to disk, sets package.loaded["B"] = nil
> 3) Lua VM executes "require 'B'" line, loads module "B"
> 4) Module "A" has some mismatch (params, etc) because "A" is not updated
> which causes an exception
> 5) Thread updates module "A" to disk, sets package.loaded["A"] = nil
> 6) On next call to module "A", Lua VM loads "A"
> 7) Now everything is up to date and everything is safe again
> 
> It seems to me that this cannot be made safe, even if mutexes were used
> between the thread writing the modules to disk and the loader loading
> modules. For example, the loader would have to build a call tree of every
> module required by A and this may not be possible to evaluate without
> interpreting.
> 
> Or am I making this problem too hard and there is a simple way to
> synchronize writes to disk and the loading of "required" modules?
> 
> John Rodriguez

I think that in general, replacing a 'live' module puts constraints on both
the module and the code using the module.
All relevant data must be kept in globals by the module (upvalues will go
out of scope with the old version of the module). That is also the case for
users of the module, they should not create locals for any member of the
module, as they will remain linked to the old module (preventing it to go
out-of-scope and get collected) so no;
   local insert = table.insert
constructs

maybe a proxy table can be used for the module, where only the proxy (call
it an API) maintains references to the actual module table and holds the
relevant data as well. Then replacing the code can be done in the proxy.
That would be more generic, lower probability of accidental local
references, though that will constrain the updates to 'compatible proxy' or
'no-API-changes'.