64-bit integers (Was: Extending Lua for multiple light userdata types)

[Rena <hyperhacker@...>]

I thought it would be appropriate to split this, since the original
topic wasn't really about 64-bit integers, but an extension to the
type system, for which implementing those would be one example use.


On 2012-11-13 6:41 PM, "Roberto Ierusalimschy" <roberto@inf.puc-rio.br> wrote:
>
> > This conversation started in IRC and I fully agree with Rena which
> > will come as no surprise as I have said this before. It is 2012 and
> > yet Lua has no access to a 64 bit int(in fact from the C API point of
> > view it was even considered to remove the default lua_Integer as a
> > typedef for ptrdiff_t ), before too long the majority of people will
> > not care about 32 bit systems and even arm is getting a 64 bit
> > version, when will Rio Lua have a 64 bit int? When will we be able to
> > sanely do bit operations on 64bit types without resorting using
> > strings *?
>
> I have been playing a little with the idea of introducing an integer
> type in Lua. I have intended to present some initial thoughts in the
> workshop.
>
> The overall idea is quite simple. There will be two kinds of numbers,
> integers and floats. All operations except division result in an
> integer if both operands are integers and in float otherwise. There
> will be two divisions: float division (the usual '/') always have
> a float result, integer division ('//'??) always have an integer
> result.
>
> Except for overflows, all operations have the same results
> independently of whether operands are represented as floats or
> as integers; but there is no automatic conversion on overflows:
> operations with integers should overflow like unsigned integers
> in C.
>
> (So, the fact that a number is represented as an integer or as
> a float is explicit and visible to the programmer, but for most
> mundane tasks this distinction is irrelevant.)
>
> -- Roberto
>
>

This all sounds good, I'm just wondering how signed/unsigned would work.

I feel like proper 64-bit integers and bit operators are two things
Lua really lacks, especially as we move to 64-bit systems. Sure, a
double can hold a 53-bit integer, and the bit32 library can be used
for simple stuff, but when you need to do serious work with them, say
working with binary protocols and file formats, those really start to
feel like using a coin as a screwdriver.

Take for example, my project to make a map editor for Mario Kart 64.
To locate the resources in the game binary (which is a single giant
executable, no filesystem) and decode texture coordinates, it uses
such lovely equations as:

CmdWord[0] = (ImgType | 0xFD000000) | 0x100000; //Set Texture Location
CmdWord[1] = (Param[0] << 0xB) + 0x05000000;
CmdWord[2] = 0xE8000000; //RDP Tile Sync
CmdWord[3] = 0;
CmdWord[4] = (((ImgType << 0x15) | 0xF5000000) | 0x100000) | (Param[2]
& 0xF); //Set Tile
CmdWord[5] = (((Param[2] & 0xF0) >> 4) << 0x18);
CmdWord[6] = 0xE6000000; //RDP Load Sync
CmdWord[7] = 0;

and:

glTexCoord2f(this->VtxCache[Command[i] >> 1].TextureCoordS /
(float)(32 << this->Tile[0].SMask),
    this->VtxCache[Command[i] >> 1].TextureCoordT / (float)(32 <<
this->Tile[0].TMask));

These are ugly enough as it is... replace all the infix binary
operators with prefix, parenthesised function calls, and it's a
downright mess. And since the GPU works with 64-bit command packets,
not having a native 64-bit type creates another complication...

Not every program has to deal with messy bit-packing code decompiled
from old video games, but plenty of programs deal with similarly ugly
binary formats, and Lua doesn't really make it easy.