Re: Suggestion: Lua 5.3 -- a facility for creating and manipulting Lua file handles (C API)

[Sir Pogsalot <sir.pogsalot@...>]

You're right about the Windows BSD socket functions working on a SOCKET type, not HANDLE -- my mistake :]  Though, in Winsock 2 the SOCKET seems to be a typedef of the HANDLE type:  http://msdn.microsoft.com/en-us/library/windows/desktop/ms740522(v=vs.85).aspx

So when I create a SOCKET with socket(), I then _open_osfhandle() it to get the associated file descriptor -- AND THEN I go _fdopen() it to get the FILE*. Only then can I assign into a luaL_Stream object. :-)

Segway into seek()'ing: I haven't really considered how working on a file descriptor and then working through the FILE object might desync.  I wonder if you write to the file descriptor if the offset of where the file is being written to needs to be updated in the FILE object.  I mean if you lseek() an fd, do you need to fseek() the file to the same position to keep them "together"?  Or can you have two independant offsets to write from?  Can you write to the fd at the associated offset and also at the offset tracked by the FILE object?  Hmm.  In my socket binding project I actually have an available sendfile() on Linux and I believe you do need to update the offset in the Lua file handle with :seek(), which is why I return how many bytes were written out to the stream.

Anyway, it was a crapload of work to understand the Windows side of things... but I had fun figuring it out (yay obsession! \o/).  It'll be a greater payoff when I see it compiling in VS though.

There was a really good book I read, I think it was mentioned earlier by Sean -- UNIX Network Programming: Sockets and XTI by Richard Stevens?  This book helped SO MUCH in figuring out how to make the BSD sockets API on Windows and Linux behave relatively the same.

Essentially in Windows you have SOCKET, in Linux you have int for the file descriptors.  In Windows if socket() fails you check for INVALID_SOCKET returned by socket(), in Linux you check for a return value of < 0.  If an operation fails on a socket (like recv()) you check for that by comparing the return code on Windows against SOCKET_ERROR, on Linux you check for a return of < 0 (again).  The more specific error code to feed to strerror() is retrieved with WSAGetLastError() on Windows -- on Linux you copy out errno.  It's at least great that both Windows and Linux have strerror() -- though it's still weird as hell that the function isn't thread-safe.  I don't see a point to strerror_r() existing, would be nice if POSIX made more changes..

I definitely need to get rid of all my abusive #defines, I've been forced to take C++ classes at my college and I'm sure it would look nicer if I make this a little more object-oriented and cleanly with inline functions.  When I wrote my project I was madly in love with C89 -- base denominator and all that. :]

Wouldn't mind taking a look at your project, I'm sure it's not stupid like you say :-)

On Sat, Nov 16, 2013 at 5:31 AM, Andrew Starks <andrew.starks@trms.com> wrote:

This is a great conversation. Excuse me for butting in, please...

On Fri, Nov 15, 2013 at 5:59 PM, Sir Pogsalot <sir.pogsalot@gmail.com> wrote:

<snip some good stuff...>

> I think I'll go back on what I said about :getfd().  I don't like fds being
> known to the Lua side of things, but a fair number of libraries expect that
> method to exist.  It is somewhat common, but I think it only exists because
> people are representing files as different types (bad).  :>

I don't think that is true... well, it is, because Windows sockets are
not files, their... SOCKETS? or handles or... i can't remember.
They're not FILE *, though.

Actually, it's also not for that reason. I think it's because, while
many socket libraries act on the FDs themselves, there are some (the
nanomsg library that I'm working on) that use them only as a signal
mechanism. If you read/write to them, "undefined behaviour" happens
(shits bed).

I believe this is because nanomsg and ZMQ are better looked as
something other than a framework. They're like a reorganization of the
network stack. If traditional stacks, you have *hardware layer ->
transport layer -> network protocol -> application stack*.

[Obviously this must still exist on some level, as this is reality and
nanomsg/zmq are abstractions (frameworks) thereof, but they do their
tricks at a low level so...]

With nanomsg, you have *message pattern (domain) -> transport
protocol(s) -> address bindings -> end point(s)*

Here, the traditional methods on a stream don't work the same, because
one of the points of these libraries is that the message is atomic.
"It's not here... not here... BAM! The whole freakin' thing is here!"

So, I'd like to do the same thing that you want to do, but with
nanomsg. My approach is going to be really different than yours. Many
of the things I'd do would look stupid. I'm guessing. One or two of
them might not actually **be** stupid.

>
> The main focus of my socket bindings is that I was trying to be as true to
> the C prototype as possible so it would be easy for beginners to take a C
> networking tutorial and translate it to Lua without much guessing or digging
> through documentation.  Sometimes I feel like details about luasocket are
> hard to find...
>
> I like your socket option code in net.c though, it's much friendlier than
> what I tried to do by adding every defined CONSTANT I could and having a
> get/setsockopt().  That's the only part of my code I really am not happy
> with yet.  I still need to get it to compile in Visual Studio, though...
>

This paragraph got me really excited. Well... not excited. It's more
that I feel your pain.

My solution, after trying the "every option * getter * setter" path,
was to make an option struct that mixed and matched a small handful of
functions. Some functions were "0 == true, everything else is an int",
some were "-1 is false, everything else is an it" and one was "error
and tell them that this property is read only." Then the rest of the
struct held the property name, the default value, the void * to the
property and the sizeof.

I can say that, compared to Lua, it looks like poop with cheese on it.
Compared to your average C code, it looks pretty tightly designed, but
without all of the neat #define hacks that make C code look truly
confounding.


Also, I too must get everything that I do working in Linux, and the
Mac and Windows. I find VS to be frustrating at first, but the more I
see the methods behind their magic, the more I come to empathise with
their approaches. I'm not saying it's better. It's just
understandable.

I need to do some clean up on my project before I post it. It's not
ground-breaking, but I'd love to get your comments on it. It sounds
like you're a more experienced "me", when it comes to playing with
this stuff.

Except, well... I'm *slightly* more experienced when it comes to
whether or not I should bother to post a suggestion to change Lua's
implementation. Only slightly, though. :)


-Andrew