Re: RE: Pluto updated

[benjamin sunshine-hill <bsunshin@...>]

The big problem with the one-stage approach is that it does not correctly handle cyclical references, since there's no way to create the first one and let it refer to the other all in one step. The impact on permanents would actually be fairly minor, assuming a slightly more complex file format which could choose to refer to a full object or to a permanent reference. Overall, though I don't think I'm willing to give up the current level of generality which allows arbitrary cyclical references.

Ben

----- Original Message -----
From: Virgil Smith <Virgil@Nomadics.com>
Date: Tuesday, June 15, 2004 11:14 am
Subject: RE: Pluto updated

> Go Ben go!  Sorry (again) that I haven't had time to work on this.
> 
> One difference in implementation structure I noticed in Pluto as opposed to
> what I've done in the past is the 2 stage behavior.  The "reference table"
> need not be passed explicitly in the stream.  It can instead be spread
> throughout the stream.  This eliminates the need for having 2 separate
> stages for "__persist" specialization.
> 
> In my implementation this merely meant "counting" objects as they were
> serialized/deserialized.  As each object was serialized I performed a check
> to see if it had already been serialized and if it was then its "count" was
> retrieved in the process and could then be used as a reference.  Upon
> deserialization the list of objects was formed in reverse and any references
> that came out of the stream necessarily had to refer to already deserialized
> objects.
> 
> Another advantage of this approach is that references need not conflict with
> light-userdata (or any other type) as they can be their own "type" in the
> stream which thus signaled that they are a reference and not a "target
> object".  These values then of course are translated to LUA_NUMBER for use
> in the tables of serialized/deserialized objects.
> 
> A distinct disadvantage to this approach is that it does not integrate as
> directly with Pluto's table of "permanents" (sorry I've forgotten the term
> you used, but I am referring to your table of items that are assumed to be
> present in the deserialization environment).
> 
> However, this still isn't too much of a departure from the Pluto high level
> stream components.  Currently (or rather last I knew) Pluto serializes the
> "references table" and then the reference that indicates the top level
> serialized object.  The switch would just be that first the "permanents"
> table is serialized, and then the top level object (rather than just a
> reference to it).
> 
> Of course this would involve a change in the code architecture that is not
> at all insignificant.  Also, I feel like I'm missing some impact on the
> operation of the "permanents" table.
> 
> 
> 
> -----Original Message-----
> From: lua-bounces@bazar2.conectiva.com.br
> [lua-bounces@bazar2.conectiva.com.br]On Behalf Of benjamin
> sunshine-hill
> Sent: Tuesday, June 15, 2004 11:00 AM
> To: Lua list
> Subject: Re: Pluto updated
> 
> 
> Okeydoke, why not. I'll make it a conditional compilation option.
> 
> Now, one caveat that needs to be worked out. __persist metamethods are
> invoked during the first stage of persistence (so that all needed objects
> referenced by the closure can be visited) but nothing is written out until
> the second stage (since not all references are assigned until then). So in
> order for this to work, Pluto will have to invoke a special stage-2 persist
> metamethod during stage 2.
> 
> Here's my idea: For userdata that need this special behavior, set the
> __persist mt member to "true". Then, set some other member
> "__persistlowlevel" to the function to execute during stage 2. The writer
> and ud will be pushed on the stack as lightuserdata.
> 
> Unpersisting is a bit more tricky, since we have to know, somehow, which
> userdata to invoke low-level unpersist methods for, and what methods to
> invoke for them. This needs to happen in stage 1 of unpersisting (since
> that's when we read everything). Normal unpersist methods are executed after
> stage 2. Ideas?
> 
> Ben
> 
> ----- Original Message -----
> From: Grisha <grisha@eagle.ru>
> Date: Tuesday, June 15, 2004 5:10 am
> Subject: Re: Pluto updated
> 
> > Hi, Ben!
> >
> > We have such a system up and running, and it's not that complex at all, I
> > can send it to you if you wish. You are right, Pluto doesn't have to know
> > anything about how C++ objects are serialized, same as C-side must know
> > nothing about how Pluto does it's job. What I'm asking for is just to
> > recieve those context parameters passed to Pluto (reader/writer & (void
> > *ud)) back in persist/unpersist methods. This will not complicate things
> > much for either you, as Pluto developer, or any other programmer working
> > with Pluto (they will just ignore extra parameters).
> >
> > -- modified text from Pluto\readme
> > Here's an example of special persistence for an object:
> >
> > vec = CreateMyObject()
> > setmetatable(vec, { __persist = function(oldtbl, writer, ud)
> > local mt = getmetatable(oldtbl)
> > local state = DoCSideSaving(oldtbl, writer, ud)
> >
> > return function(newtbl, reader, ud)
> >  setmetatable(newtbl, mt)
> >  DoCSideLoading(newtbl, reader, ud, state)
> > end
> > end })
> >
> > Note that original example will work too, since it will just ignore extra
> > parameters. We really need this, we can live without function or thread
> > persistance, but without proper uderdata persistance we can't.
> >
> > Thanks in advance,
> > Grisha
> >
> > ----- Original Message -----
> > From: "benjamin sunshine-hill" <bsunshin@usc.edu>
> > To: "Lua list" <lua@bazar2.conectiva.com.br>
> > Sent: Saturday, June 12, 2004 1:17 AM
> > Subject: Re: Pluto updated
> >
> >
> > > Hmmm..... it's a tricky issue that you raise. After all, Pluto's
> designed
> > to serialize a bunch of interreferencing Lua objects, rather
> > interreferencing C++ objects. A system capable of serializing C++ objects
> in
> > a parallel manner would be complex indeed. Still, I really think that the
> > current situation will work. What would need to be done would be for the
> > __persist functions to invoke the C++ persistence framework in such a way
> > that temporary Lua objects--userdata, most likely--were created to
> > encapsulate the object state. If objects are shared between different
> > userdata (using some custom C-side reference counting mechanism, for
> > instance), some registry-related method could be used to coordinate
> > serialization between different C object entry points. This is, of course,
> a
> > rather complex method... but it's really not much more complexity than
> > absolutely necessary to serialize a C++ framework. Pluto would still
> > automatically handle fixup before running the unpersistence method
> > > s, of course.
> > >
> > > Ultimately, I don't think that the main question is whether the
> __persist
> > metamethod writes through a string/userdata or through a writer object;
> > that's just semantics, with some performance implications. The real
> question
> > is how much control __persist can exert over the serialization stage, and
> > how much control it has over internal Pluto placeholder references. I'd
> > really prefer to keep those as hidden implementation details; they've
> > already gone through a couple of revisions, and there are quite a few
> > caveats that they deal with in order to keep Lua happy.
> > >
> > > Ben
> > >
> > > P.S. I'm currently working on the string-keys-for-permanents idea you
> > proposed. It's proving to be more difficult conceptually than I'd
> expected,
> > to maintain flexibility and provide as much user-control as possible, but
> I
> > should have it ironed out in a week or so (travel will be slowing me
> down).
> > >
> >
> >
> 
> 
> 
>