Re: BUG #1931: ILIKE and LIKE fails on Turkish locale

Victor Snezhko <snezhko(at)indorsoft(dot)ru> writes:
> My FreeBSD lists a whole heck of characters:

> character 0x85 is a space
> character 0xa0 is a space
> character 0xaa is alphabetical
> character 0xb5 is alphabetical
> character 0xba is alphabetical
> character 0xc0 is alphabetical
> ... 0xc1-0xfe is alphabetical
> character 0xff is alphabetical

Hm. I'm still thinking that this behavior is wrong for UTF8 encoding,
but it would be reasonable in LATINn and related encodings, so we
probably ought to do something about it.

After further thought, it's not so much that we can't tolerate
locale-dependent behavior of isspace() in general, as that in this
particular case we are expecting it to match the scanner's idea
of a space: scan.l has

space [ \t\n\r\f]

which obviously is not locale-aware. I think we need convert_ident to
use a plpgsql_isspace() that accepts these and only these as spaces.
Any high-bit-set byte is part of an identifier according to scan.l's
rules, and convert_ident must have the same behavior regardless of locale.

There may be related risks in and around the other flex scanners
... will look.

regards, tom lane