Re: Unicode full case mapping: PG_UNICODE_FAST, and standard-compliant UCS_BASIC

On Fri, Jan 17, 2025 at 04:06:20PM -0800, Jeff Davis wrote:
> Committed 0001 and 0002.

> Upon reviewing the discussion threads, I removed the Unicode "adjust to
> Cased" behavior when titlecasing. As Peter pointed out[1], it doesn't
> match the documentation or expectations for INITCAP().

While commit d3d0983 changed most of the non-test pg_u_*() "bool posix"
arguments, it left a pg_u_isalnum(u, true) in strtitle_builtin() subroutine
initcap_wbnext(). The above paragraph may or may not be saying that's
intentional. Example of the consequence at non-ASCII decimal digits:

SELECT
str,
re,
regexp_count(str COLLATE pg_c_utf8, re) AS count_c_utf8,
regexp_count(str COLLATE pg_unicode_fast, re) AS count_unicode_fast,
regexp_count(str COLLATE unicode, re) AS count_unicode,
initcap(str COLLATE pg_c_utf8) AS initcap_c_utf8,
initcap(str COLLATE pg_unicode_fast) AS initcap_unicode_fast,
initcap(str COLLATE unicode) AS initcap_unicode
FROM
(VALUES (U&'foo\0661bar baz')) AS str_t(str),
(VALUES ('[[:digit:]]')) AS re_t(re)
ORDER BY 1, 2;

str │ foo١bar baz
re │ [[:digit:]]
count_c_utf8 │ 0
count_unicode_fast │ 1
count_unicode │ 1
initcap_c_utf8 │ Foo١Bar Baz
initcap_unicode_fast │ Foo١Bar Baz
initcap_unicode │ Foo١bar Baz

Should initcap_wbnext() pass in a locale-dependent "bool posix" argument like
the others calls the commit changed? Related message from the development of
pg_c_utf8, which you shared downthread:
https://www.postgresql.org/message-id/610d7f1b-c68c-4eb8-a03d-1515da304c58%40manitou-mail.org

Long-term, pg_u_isword() should have a "bool posix" argument. Currently, only
tests call that function. If it got a non-test caller,
https://www.unicode.org/reports/tr18/#word would have pg_u_isword() follow the
choice of posix compatibility like pg_u_isalnum() does.