Re: Let's make Lua mobile!

[David Hollander <dhllndr@...>]

> It's not always obvious, especially if one isn't a protocol geek, what to build network-enabled applications on top of.
> The internet is an interesting beast...

Indeed! HTTP always gets my goat because it alternates between both a
character delimited protocol, where one stumbles in the dark for a
"\r\n" terminator, and a predetermined length protocol
(Content-Length, Transfer-encoding: chunked).

Anyway, some questions I would ask (the answers may not be the same
for everyone):

Local and/or remote?   remote.
Untrusted, public, negotiated?   No, trusted and private, just
distributing computation\data.
Necessary to implement in a wide variety of languages?   no, just Lua
Data description layer needed?  No, Lua is a data description
language. Lua can parse Lua faster than it parses JSON or XML.
Sensitive data, encryption needed?  unknown.
Compression needed? CRC (error checking) needed?  Maybe. Can gzip
message bodies to kill two birds with one stone.

In the LISP world, a common strategy is simply to pass S-expressions
back and forth. If it's a private protocol for computation, I like the
idea of just sending serialized lua values back and forth. I like the
idea of ad hoc message passing where the handler observes (ip_address,
message) rather than streams, because it is agnostic as to whether
there was a reconnect, and logic is built up in the application using
Lua.

A slightly more complex protocol handle multiple values (ip_addres, ...):

1 byte: number of values
2 bytes: length of 1st value
X bytes: serialized 2nd value
...

But even this might be overkill and premature optimization, as one Lua
table can also be used to describe a list. The appropriate solution is
dependent on what you want to build. If the use case is fairly
undefined, starting simple can't hurt :) One could provide optional
"filters" that add on RSA encryption etc, so users can build the
protocol up from a simple base by wrapping it in different Lua
functions.

- David Hollander

On Sat, Oct 29, 2011 at 10:32 PM, Matthew Wild <mwild1@gmail.com> wrote:
> On 29 October 2011 22:30, David Hollander <dhllndr@gmail.com> wrote:
>>>XMPP
>>
>> Let me proposes a different protocol that might suit your needs:
>>
>> 1st and 2 byte: message length
>> X bytes: message
>>
>> message = any serialized lua value.
>>
>> Pass (senderIP, message) to a handler. The end :)
>
> Indeed, something like protocol buffers even easily automates this approach.
>
> What kind of protocol to use depends a lot on the needs of the
> application. XMPP and even ZeroMQ are high-level protocols (of
> different degrees) that give a framework for application protocols.
> They all have their strengths and weaknesses.
>
> For example, XMPP provides authentication, identity, distribution,
> extensibility, and standard features such as encryption, compression,
> connection resumption, etc. It doesn't deal so well with binary
> payloads, adds routing overhead, and is more work to parse than a
> TLV-encoded protocol.
>
> ZeroMQ is a more suitable transport for a custom binary protocol, it
> has built-in framing, nifty abilities like efficient intra- and
> inter-process communication, multicasting, etc. It is not so good at
> the identity aspect, encryption, or robustness when facing public
> networks.
>
> It's not always obvious, especially if one isn't a protocol geek, what
> to build network-enabled applications on top of. The internet is full
> of protocol abuse, from HTTP-centric hacks like comet to a case I saw
> once of someone base64-encoding video frames and sending them over an
> XMPP stream. Oh, and I've seen SMTP used for machine-to-machine
> communication on a local network (Python has an SMTP client *and
> server* in the standard library, believe it or not).
>
> Some of these abuses are pure monsters, while some are "wrong" but
> necessary hacks in a sense, e.g. comet, and fun things I've seen like
> SSH-over-XMPP... working through any number of proxies and firewalls.
> The internet is an interesting beast...
>
> Regards,
> Matthew
>
>