NLS functions in the Lua core (was: Unexpected malloc)

[Mike Pall <mikelu-0711@...>]

Hi,

Enrico Colombini wrote:
> I know there's little the Lua developers could (or should) do about this, 
> apart perhaps for giving the opportunity to redefine locale-related 
> functions, or just to ignore locale.

Well, this is not a new issue:
  http://lua-users.org/lists/lua-l/2005-02/msg00460.html

I'm still of the opinion that the Lua core should stay clear of
NLS functions. I.e. replace ctypes with an internal table, replace
l_strcmp with memcmp and move that strcoll mess to loslib.c or
(better yet) to an 'ancient single-char NLS compatibility' module.

I have to ask the Lua authors to reconsider the decision to use
ctypes in the lexer and strcoll:

- All recent Linux/*BSD distros and of course Mac OS X install
  with UTF-8 locales by default. Most single-char NLS functions
  don't work _at all_ with UTF-8 locales (see recent thread).

- Single-char NLS never worked too well with Windows 95 or later.

- Embedded systems are usually very ignorant about NLS and
  everyone seems to be patching out these functions, anyway.

- Any number of libraries seem to change the libc locale at a
  whim (e.g. readline and GTK+) and cause unexpected side-effects
  in the Lua lexer or the string matching functions.

- Collation in a multilingual world is a royal mess, best left to
  an external library.

Alas, one cannot completely get rid of all NLS functionality in
Lua. E.g. strtod() (or rather the workaround in the lexer) relies
on a few of them.

> (any idea where hidden mallocs could lurk? I'm only using the table 
> library)

I think many of the libc functions called in loslib.c, liolib.c
and loadlib.c may potentially call malloc (or even mmap in the
case of stdio), but none should be leaky.

Bye,
     Mike