Re: RE: Pluto: heavy-duty persistence for Lua

[benjamin sunshine-hill <bsunshin@...>]

I guess the potential solutions fall into two categories: Making the programmer go out of his way to make a particular aspect of an object properly persisted, or making him go out of his way to keep it from being persisted in the default manner. Either way, it seems like an intractable problem to keep things simple for both simple and complex object types.

Ultimately, I think I'd lean towards the former solution; if someone is designing a complex object, that leads to more wordy code. You're right, though; if closures are available, then they're the right way to do things, and combining them with an argument list would be a needlessly redundant feature. I think the best solution may be to explicitly remind the programmer to copy table values into local variables for __persist closures, and explicitly check for self-referential persistence (which I haven't yet thought about the complexity of, but assume wouldn't be that bad).

If someone does prefer the solution of "cleaning up" the table, then it would be fairly simple to emulate with the closure solution, since there's nothing to prevent the __persist function from making changes to the table. It would then simply return a closure which returned the table, and would somehow signal the persister to ignore the self-reference. Hmm... there might be problems there, and I don't like the features getting this wacky... I'll give it more thought.

Ben


----- Original Message -----
From: Virgil Smith <Virgil@Nomadics.com>
Date: Friday, June 4, 2004 1:56 pm
Subject: RE: Pluto: heavy-duty persistence for Lua

> Ben, that certainly reads nicely when used with "object factory" functions,
> in that what is returned is essentially a factory/constructor function along
> with a list of arguments to pass to it, but this is going to be supremely
> ugly for tables with large array sections or lots of members for some other
> reason.  Of course, this syntax does not preclude a complicated closure, it
> merely extends it.
> 
> If the "extends it" statement doesn't make immediate sense, consider the
> following...
> 
> mt.__persist = function(tbl) do
> local x,y = tbl.x, tbl.y
> return function(z) do
>    return {x=x, y=y, z=z}
> end, tbl.z
> end
> 
> ---------------
> 
> Aha!
> 
> The solution for picking out table items that should not be persisted is
> really quite simple, the mt.__persist function should just remove them from
> the original table and return the cleaned up original table (or a closure
> that returns it).  After all if the resulting table can function
> appropriately after being unpersisted, then it can function just fine after
> being persisted.  This is simple to implement and does not waste space by
> duplicating the original table.
> 
> Argh, here comes the rebuttal, I can see it now.  "But it will take X amount
> of time to reconstruct the 'transient'/'cached' values and I find that
> acceptable after a 'file load', but not after a 'file save'."
> 
> Um, the only "complete" solution I can think of is a callback function that
> the could be called for each key/value pair in the table to
> approve/disapprove its storage, or a list of "fixup" functions to be called
> after persisting is completed (so that 'transient'/'cached' values could be
> moved out of the way and then returned after the persisting operation is
> complete).
> 
> Yuck!
> 
> 
> -----Original Message-----
> From: lua-bounces@bazar2.conectiva.com.br
> [lua-bounces@bazar2.conectiva.com.br]On Behalf Of benjamin
> sunshine-hill
> Sent: Friday, June 04, 2004 12:58 PM
> To: Lua list
> Subject: Re: Pluto: heavy-duty persistence for Lua
> 
> 
> My current thought is to use the "third" option: Have __persist be a
> function which returns a function and an arbitrary number of arguments.
> Then, during unpersistence, those arguments are passed into those functions.
> That makes the easy-to-hit closure bug less likely:
> 
> mt.__persist = function(tbl) do
> return function(x,y,z) do
>    return {x=x, y=y, z=z}
> end, tbl.x,tbl.y,tbl.z
> end
> 
> Thoughts?
> 
> Ben
> 
> ----- Original Message -----
> From: Mark Hamburg <mhamburg@adobe.com>
> Date: Friday, June 4, 2004 11:53 am
> Subject: Re: Pluto: heavy-duty persistence for Lua
> 
> > on 6/4/04 8:41 AM, Virgil Smith at Virgil@Nomadics.com wrote:
> >
> > > In fact the only real "complexity"/"gotcha" here is that the following
> > > version of "Returning a closure" is VERY WRONG
> > >
> > > -- Returning a closure (INCORRECTLY)
> > > -- This will force tbl itself to be persisted
> > > -- in the stream
> > > mt.__persist = function(tbl) do
> > >  return function() do
> > >     return {x=tbl.x, y=tbl.y, z=tbl.z}
> > >  end
> > > end
> >
> > That's also potentially an easy mistake to make. Also, relying on
> generating
> > lots of closures or auxiliary tables creates just to strip stuff out
> creates
> > a lot of extra memory allocation traffic which hurts efficiency. Now, if
> one
> > went with tables rather than closures or in addition to closures as the
> > results of the __persist metamethod and passed the method an empty table
> > that the persistence mechanism would clear and reuse later, you might be
> > able to work around that memory overhead issue.
> >
> > Mark
> >
> > P.S. These sort of cases make a table.clear function that resets a table
> > back to the empty state attractive.
> >
> >
> 
> 
> 
>