Re: [rfc] unicode escapes for extended strings

On Wed, 2009-09-09 at 18:26 +0300, Marko Kreen wrote:
> Unicode escapes for extended strings.
>
> On 4/16/09, Marko Kreen <markokr(at)gmail(dot)com> wrote:
> > Reasons:
> >
> > - More people are familiar with \u escaping, as it's standard
> > in Java/C#/Python, probably more..
> > - U& strings will not work when stdstr=off.
> >
> > Syntax:
> >
> > \uXXXX - 16-bit value
> > \UXXXXXXXX - 32-bit value
> >
> > Additionally, both \u and \U can be used to specify UTF-16 surrogate
> > pairs to encode characters with value > 0xFFFF. This is exact behaviour
> > used by Java/C#/Python. (except that Java does not have \U)
>
> v3 of the patch:
>
> - convert to new reentrant lexer API
> - add lexer targets to avoid fallback to default
> - completely disallow \U\u without proper number of hex values
> - fix logic bug in surrogate pair handling

This looks good to me. I'm implementing the surrogate pair handling for
the U& syntax for consistency. Then I'll apply this.