Re: Platform-dependent variations in numeric conversions

[Nick Gammon <nick@...>]

On 16/06/2008, at 8:06 AM, Duck wrote:

> The Windows string conversion seems to be "unsigned 32-ing" the  
> converted
> number, whereas the Linux one is not. I presume the compiler does the
> negation of hex constants itself, therefore producing consistent  
> results.

Thanks very much for this message!

I have been using Lua for about 4 years without realizing you could  
even use hex constants like that.

I checked my hard-copy of Programming in Lua, and could not see any  
reference to using 0x constants. Also the Lua 5.1 Reference Manual  
(hard copy) does not mention it. However the online version of the  
Reference Manual slips in an extra sentence: "Lua also accepts integer  
hexadecimal constants, by prefixing them with 0x.". Must be a recent  
addition.

On to your problem:

> > return  tonumber('-0x12345678')
> 39895547400


The Reference manual (for tonumber) specifically disallows signed  
numbers for hexadecimal constants, see this part:

"In base 10 (the default), the number may have a decimal part, as well  
as an optional exponent part (see §2.1). In other bases, only unsigned  
integers are accepted.".  It is a bit ambiguous whether that applies,  
as by not supplying a base you are defaulting to 10, however you then  
use the 0x notation, which switches you to base 16.

As far as I can see from the source, the conversion is done here:

  char *endptr;
  *result = lua_str2number(s, &endptr);
  if (endptr == s) return 0;  /* conversion failed */
  if (*endptr == 'x' || *endptr == 'X')  /* maybe an hexadecimal  
constant? */
    *result = cast_num(strtoul(s, &endptr, 16));

It is calling strtoul for the hex conversion which is defined to  
return an unsigned long.

An interesting possible explanation for this behaviour, is that my  
manual entry for strtod on BSD Unix (Mac OS/X in my case) says that  
strtod will handle a leading sign, AND "a hexadecimal significand,  
consisting of a ``0X'' or ``0x'' followed by a sequence of hexadecimal  
digits".

However on Cygwin, the manual makes no such reference. This is  
relevant because lua_str2number uses strtod:

#define lua_str2number(s,p)   strtod((s), (p))

Thus, on some platforms the call to lua_str2number will handle the hex  
case, and handle the negative case as well.

However if that call fails, and we drop down to Lua's test for 'x' or  
'X' then we are calling strtoul which does not handle the negative case.

There is certainly some inconsistency there, and perhaps the Lua code  
should detect the leading "-" sign, if it is going to do the hex  
conversion itself.

- Nick