Re: Matching multibyte alphabetical characters with LPeG

[Miles Bader <miles@...>]

Hinrik Örn Sigurðsson <hinrik.sig@gmail.com> writes:
> I've been making a parser with LPeG and I've run into the issue that I can't
> match non-ASCII words even though I'm using a utf8 locale. It seems that
> "alpha" (and "alnum", etc) from lpeg.locale() don't match anything beyond ASCII.
> See the following code:
>
>     local lpeg = require 'lpeg'
>     local locale = lpeg.locale()
>     print(lpeg.match(lpeg.C(lpeg.P("æ")), "æ"))    --> æ
>     print(lpeg.match(lpeg.C(locale.alpha), "æ"))   --> nil
>
> Is there an easy way to match non-ASCII alphabetical characters with LPeG?

No -- it's not so hard to parse utf-8 characters, but testing a
property like "alphabetic" requires unicode tables, which are a huge
and bloated dependency.

[See the LPEG home page for a simple example of how to parse utf-8
characters though.  http://www.inf.puc-rio.br/~roberto/lpeg/ ]

> If not, can LPeG be patched to support it?

Very unlikely.

-miles

-- 
Egotist, n. A person of low taste, more interested in himself than in me.