Re: Support for Windows unicode paths

[Jerome Vuarand <jerome.vuarand@...>]

2009/7/22 Thomas Harning Jr. <harningt@gmail.com>:
> On Wed, Jul 22, 2009 at 11:44 AM, David Given <dg@cowlark.com> wrote:
>>
>> Bulat Ziganshin wrote:
>> [...]
>>>
>>> just adding encoding/decoding layer is probably impossible since
>>> open/popen/... on windows uses OEM/ANSI encoding by default, and
>>> encoding from utf-8 to oem/ansi is lossy
>>
>> It's worse than you think. On Windows, the ANSI file system APIs have a
>> much shorter maximum path length than the UTF-16 ones! Read this:
>>
>> http://subversion.tigris.org/faq.html#long-paths
>>
>> So a naive mapping of the Lua functions to the corresponding Windows ones
>> is not necessarily going to work --- it's entirely possible to be in a
>> situation where just calling getcwd() fails.
>>
>> Ditching the ANSI APIs completely and using the UTF-16 ones entirely is
>> probably the best way to go, unfortunately...
>
> Yeah, that's what I'd figured I had to do.
>
> Basically replace all ANSI APIs with the UTF-16 ones, and have an automatic
> conversion from UTF-8 => UTF-16  [and back when pulling data out]

I have a patch (attached) replacing win32 API calls with the wide
versions, with the help of a couple helper functions (lua_towstring,
lua_pushwstring) that convert between utf-8 (Lua side) and utf-16
(win32 side). You can use that as a basis to replace all appropriate
clib calls. I have more complete wstring management functions in
another lib if you're interested (lua_tolwstring, lua_pushlwstring,
luaL_*, etc.).

I also have a partial win32 binding using the same mechanism (ie.
utf-8 in Lua and wide version of win32 API calls). That could be a
basis to add utf-8 support to Lua standard libraries without patching
the core. It's available as sources there:
http://piratery.net/hg/lwin32/archive/tip.tar.gz.

Attachment: win32-unicode.patch
Description: Binary data