RE: Lua shared between processes

[jdarling@...]

I started looking into this myself, but didn't have time to learn
everything that I needed to know about C memory management.  You could
use a basic spinlock mech in the memory manager that you provide to
aliviate some of the problem (at least then the variables could be
modified by either process w/o problems) but this will add a minor bit
of overhead.  In theory it shouldn't be hard to implement, but it takes
someone with alot of knowledge of C and of C memory managers.  After a
few of my projects drop off, I plan on going back and taking a better
look at the ability to pull this off.

 - Jeremy

"Help I suffer from the oxymoron Corporate Security."


> -------- Original Message --------
> Subject: Re: Lua shared between processes
> From: Rob Shaw <rob@haptek.com>
> Date: Mon, March 20, 2006 10:14 am
> To: Lua list <lua@bazar2.conectiva.com.br>
> 
> Hi,
> 
> I found this thread in the archives, and decided to try having two  
> processes
> look at the same lua state, as we use shared memory, and this might be
> useful.
> 
> The good news is, I got it to work.  In the thread was mentioned a  
> malloc
> written by Doug Lea, which since version 2.8 has an "mspace" option,
> allowing you to specify where the mallocs are going to be, for example,
> in shared memory.  So one invocation of the lua interpreter in one  
> window
> sets up the lua world in shared memory, and a second lua in another
> window just picks up the pointers in the shared memory.  You can type  
> away
> in one window in one process, and pick up where you left off in the  
> other
> window in another process.
> 
> The bad news is, it's useless, at least for what we want to do.  As  
> long as you
> do one thing at a time, everything is fine, but if two processes do  
> something
> at the same time, crashes result.  I should have thought of this, but  
> if there is
> just one stack, and two computations trying to use it simultaneously,  
> of course
> there are going to be problems.
> 
> I guess what we want is to have the lua variables, tables, etc. in  
> shared memory,
> but somehow have the lua virtual machines running separately.  My  
> knowledge
> of lua internals is still pretty minimal, so I don't know if this is  
> even possible.
> 
> We have other ways to do the job, just using c helper functions to  
> access the
> stuff in shared memory, that work just fine.  But I'm curious to know  
> whether
> multiple luas working on the same data is a viable concept.
> 
> Thanks!
> 
> rob
> 
> 
> ----- Original Message -----
> From: George Sealy <george@arl.co.nz>
> Date: Saturday, March 13, 2004 5:21 pm
> Subject: Lua shared between processes
> 
>  > Hi all,
>  >
>  > I've been experimenting with Lua for work, and have found it to be  
> just
>  > the tool I was looking for.  However, I have a problem that I was  
> hoping
>  > someone may be able to help me with.
>  >
>  > I need to have multiple processes running on a single machine  
> (Win32),
>  > all with access to a single instance of a "world state", which I  
> want to
>  > describe with Lua.  My initial solution was to use shared memory in a
>  > dll, which would allow all processes to access the single set of  
> data.
>  > However, when I use lua_open to create a new state, I can't see a  
> way to
>  > make that pointer be in shared memory...  Initially I thought I could
>  > create a copy of the Lua state variable, but that'll fall over
>  > internally whenever Lua needs more memory.
>  >
>  > The only other option I can think of is some form of communication
>  > between processes, such as TCP/IP or memory mapping, but the world  
> state
>  > needs to be accessed many (hundreds or even thousands of) times a
>  > second, so performance is an issue.
>  >
>  > If anyone has any advice, I'd really appreciate it.  I'd rather avoid
>  > rewriting the entire language's memory allocation if I can help it!
>  >
>  > Thanks,
>  >
>  > George Sealy
>  >
>  > --
>  > George Sealy, PhD                       email : george@arl.co.nz
>  > Software Developer / Architect          phone : +64 3 477 2995
>  > Animation Research Ltd
>  >