Re: Managing Unicode (UTF-8 and UTF-16) data in Lua

[Paul Moore <p.f.moore@...>]

On 5 August 2016 at 18:51, Paul K <paul@zerobrane.com> wrote:
> You may want to look at winapi
> (http://stevedonovan.github.io/winapi/api.html) as it already handles
> most of these cases (without modifying the default tables though).

Thanks. I'll definitely take a look - but for my application,
executable size is a constraint, so I'm more interested in including
specific custom functions than a general API wrapper. I'll definitely
look at it for ideas, though.

Hmm, looks like it does more or less what I'm doing at the moment
(converting wide strings to Lua strings), but it doesn't use UTF-8,
but rather uses the ANSI codepage (so it's just as vulnerable to
mojibake as Lua's os.getenv, for example). But thanks for the pointer
anyway.

Paul

PS If it's not clear what I mean when I talk about encoding issues, on
a UK English Windows PC, create an environment variable called "AB",
with the value "123€456", and then write a Lua script to read its
value and print it. I get "123Ç456" as the output - because the Ansi
codepage is (IIRC) 1252 and the console (OEM) codepage is 850. And the
euro sign differs between those 2 code pages...