Re: Functions and memory usage

["terenctb" <terenctb@...>]

I have been thinking of doing something simliar to what Alberto has 
been describing(we already do something like this for our other 
memory manager) but I am not sure that it will be enough to cut down 
the amount of data being accessed because of the method that the 
allocation/gc is being done already does lend it self to speed.

The way I was going to do it was to just throw memory at it(since I 
am in eviable position of using LUA on server with alot of 
memory)..taking it from a pre-allocated area and sizing it so that 
each allocatable area is tuned to the largest size possible. This 
means no best fit, no frag and hopefully big boost in speed. BUT I 
put some tracers on the way LUA alllocates/deallocates memory and it 
seems a better way would be just to predefine a memory 'pool' for 
each script..since each script is basically running the 'same' logic, 
pre-allocation of all necessary data would faster..

Hmmm..now if only I figure out how to do it the short time I have;-)
Do you think the new version of LUA will have multiple 'memory' 
managers instead of 1, cause I think right now this is the biggest 
thing that people always end up changing in lua.


Terence
Phoenix-GameStudios

www.phoenix-gamestudios.com

--- In lua-l@y..., "Alberto Demichelis" <alberto@c...> wrote:
> > What happens when you deallocate/free an object?
> Instead of keeping track of the pages we keep a linked list of feer 
blocks for every chunk size,
> because they are "empty" we use the block himself as node of the 
linked list so no memory overhead.
> When we create a page, the address of the first chunk is aligned in 
a way that allow us to 
> retrieve the page start address from the chunk address.The page is 
ref counted(first 4 bytes).
> When we allocate a chunk we pop it from the free list and add a 
reference to the page.
> When we free we put it back in the free list and we decrement the 
reference if is 0 we free the page.
> 
> I've took a quick look at dlmalloc, it looks not so far from our 
approach exept for the implementation.
> I'll test it.
> 
> 
> ciao
> Alberto
> 
> PS: btw 200k/300k are internal structures + "waste".
> 
> ---------------------
> Alberto Demichelis
> alberto@c...
> Crytek Studios GmbH
> www.crytek.com
> ---------------------
> 
> 
> -----Original Message-----
> From: Nick Trout [mailto:ntrout@r...]
> Sent: Freitag, 18. Oktober 2002 01:06
> To: Multiple recipients of list
> Subject: RE: Functions and memory usage
> 
> 
> 
> > 200k/300k of internal structures of the allocator on 100Mb of 
> > allocated memory sound
> > pretty good to me, I do not understand the smile :-)
> 
> Then its not memory wasted, its used to manage the memory 
management. I
> assumed you meant that 200k-300k is lost to spaces at the end of 
pages,
> and rounding up, as you describe.
> 
> The smile was to show that while I might be aiming a criticism at 
you
> there is no malice intended! Its all a bit dry this technical mumbo
> jumbo sometimes and email can often be read ambiguously.
> 
> > > I assume each pool size would have an most recently used stack, 
only
> > > allocating a new pool once you've checked the existing 
> > pools for space?
> > 
> > I don't really understand the question :P
> > What do you mean as most recently used stack?
> 
> What happens when you deallocate/free an object? Lets say you 
allocated
> 10 pools (4k blocks) of 16 byte objects and then free a load of 
them. If
> you then wanted to allocate a new 16 byte object, how would that be
> done? I assume for fast allocation you would keep track of pools, 
and
> group them by size of object, so you could quickly find a pool
> containing 16 byte objects to allocate from. I was speculating that 
a
> set of stacks of pools might be a good way to organise this. When a 
new
> pool of a certain size is required it is pushed onto the stack and 
can
> be accessed quickly. However when deallocation takes place this 
might
> not be the best idea. I'd be interested to know how this is managed
> since you mention your free time was greatly reduced.
> 
> If you are allocating 100Mb then this would be beyond the scope of
> certain platforms and so a system like yours may not be as 
efficient or
> suitable for all applications. The dlmalloc system may be a better
> general solution?
> 
> Nick