Speed of Lua's immutable strings vs buffers (split from Re: Pooling of strings is good)

[Sean Conner <sean@...>]

It was thus said that the Great Coroutines once stated:
> On Mon, Aug 25, 2014 at 1:14 PM, Roberto Ierusalimschy
> <roberto@inf.puc-rio.br> wrote:
> >> Earlier in this long and unproductive email chain [...]
> >
> > On which you created more than 40% of the messages...
> 
> Good, so you understand my disappointment -- having repeatedly
> clarified what I'm looking to do for those who didn't read.  Now that
> everybody is caught up we can start being productive ^__^

  For relative values of "productive."

  Seeing how I'm probably responsible for the other 40% of the messages, and
to be "more productive" I spent the past two days writing code to measure
the overhead of using Lua's immutable strings vs. buffers.  

  The thesis:  using Lua's immutable strings for saving network buffers is
more inefficient than using a reusable buffer.

  Coroutines:  it is more effieicent to reuse a buffer, so let's rework Lua
completely so we have both mutable and immutable strings.

  Me: it's not worth the effort, and mutable strings can cause more problems
than the benefits.

  So, the code.  It's all here:

	https://github.com/spc476/lua-net-speed

  I have a program in C that sends packets as fast as possible to the
destination.  All the packets are the same size (but the size is
configurable on the command line) and the program runs for a set amount of
time.  The results are from running the following command line (run for 10
seconds, use a packet size of 1450 bytes):

	./blast -a 127.0.0.1 -p 22222 -t 10 -s 1450

on a 2.6GHz dual-core 32-bit pentium running Linux.  The code presented is
compiled using no optimization (just gcc); the Lua interpreter is the sock
compile from PUC (this is Lua 5.1 by the way---I'm not set up to use Lua
5.2). While I have a C version and a Lua version of the same program (the C
version is all C, except it passes the packet to a Lua function) I only have
a Lua version of the "buffer userdata" but that was enough to show an
interesting result.

  The function being run for each packet is (something.lua):

	counts = {}
	for i = 0 , 255 do
	  counts[i] = 0
	end

	function main(packet)
	  local max = #packet
	  for i = 1 , #packet do
	    local c = packet:byte(i)
	    counts[c] = counts[c] + 1
	  end
	end

  The intent was to do some processing over the packet.  Immutable strings
have the byte() method defined, and for the buffer userdata, I implemented
the same function (in fact, using the same code from PUC Lua, lifted 99%
straight---the only change was from using a const char * to a userdata) for
the buffer userdata so the same code could be used for both versions.  The
programs used to receive the data are written in Lua, and are suck.lua
(immutable strings) and suckb.lua (userdata buffer).  I ran each three times
and am presenting the results for each run.

	Using Lua immutable strings:
	Packets sent	Packets recived		Memory used in K
	by blast.c	by suck.lua		by suck.lua

	1442627 	12003 			67
	1442889 	12090 			85
	1440660 	12063 			54

I know, it looks horrible---1.4 million packets sent, 12,000 processed?
Buffers surely must do better, right?

	Using a buffer userdata:
	Packets sent	Packets received	Memory used in K
	by blast.c	by suckb.lua		by suckb.lua

	1440542		7612			57
	1441593		8306			91
	1447008		8278			87

  I am willing to conceed an error in my methodology, but the ball is now in
Coroutines' court to show where I am in error.

  On a related note---for drastic changes like introducing mutable strings
to Lua, it should be up to the presenter to do the work, to do the
implementation (even if it does duplicate code) to show the idea is sound. 
Working code trumps wishes and laments.  I applaud Jan Behrens' work on
ipairs because he's taking the time to implement his ideas.

  -spc