pgsql: ICU support

From: Peter Eisentraut <peter_e(at)gmx(dot)net>
To: pgsql-committers(at)postgresql(dot)org
Subject: pgsql: ICU support
Date: 2017-03-23 19:33:53
Message-ID: E1cr8Ub-0003aO-Kl@gemulon.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

ICU support

Add a column collprovider to pg_collation that determines which library
provides the collation data. The existing choices are default and libc,
and this adds an icu choice, which uses the ICU4C library.

The pg_locale_t type is changed to a union that contains the
provider-specific locale handles. Users of locale information are
changed to look into that struct for the appropriate handle to use.

Also add a collversion column that records the version of the collation
when it is created, and check at run time whether it is still the same.
This detects potentially incompatible library upgrades that can corrupt
indexes and other structures. This is currently only supported by
ICU-provided collations.

initdb initializes the default collation set as before from the `locale
-a` output but also adds all available ICU locales with a "-x-icu"
appended.

Currently, ICU-provided collations can only be explicitly named
collations. The global database locales are still always libc-provided.

ICU support is enabled by configure --with-icu.

Reviewed-by: Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com>
Reviewed-by: Andreas Karlsson <andreas(at)proxel(dot)se>

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/eccfef81e1f73ee41f1d8bfe4fa4e80576945048

Modified Files
--------------
aclocal.m4 | 1 +
config/pkg.m4 | 275 ++++++
configure | 313 ++++++
configure.in | 35 +
doc/src/sgml/catalogs.sgml | 19 +
doc/src/sgml/charset.sgml | 177 +++-
doc/src/sgml/func.sgml | 17 +
doc/src/sgml/installation.sgml | 15 +
doc/src/sgml/mvcc.sgml | 3 +-
doc/src/sgml/ref/alter_collation.sgml | 55 ++
doc/src/sgml/ref/create_collation.sgml | 37 +-
src/Makefile.global.in | 4 +
src/backend/Makefile | 2 +-
src/backend/catalog/pg_collation.c | 52 +-
src/backend/commands/collationcmds.c | 288 +++++-
src/backend/common.mk | 2 +
src/backend/nodes/copyfuncs.c | 13 +
src/backend/nodes/equalfuncs.c | 11 +
src/backend/parser/gram.y | 18 +-
src/backend/regex/regc_pg_locale.c | 110 ++-
src/backend/tcop/utility.c | 8 +
src/backend/utils/adt/formatting.c | 453 +++++----
src/backend/utils/adt/like.c | 53 +-
src/backend/utils/adt/pg_locale.c | 266 ++++-
src/backend/utils/adt/selfuncs.c | 8 +-
src/backend/utils/adt/varlena.c | 179 +++-
src/backend/utils/mb/encnames.c | 76 ++
src/bin/initdb/initdb.c | 3 +-
src/bin/pg_dump/pg_dump.c | 75 +-
src/bin/pg_dump/t/002_pg_dump.pl | 2 +-
src/bin/psql/describe.c | 7 +-
src/include/catalog/pg_collation.h | 25 +-
src/include/catalog/pg_collation_fn.h | 2 +
src/include/catalog/pg_proc.h | 3 +
src/include/commands/collationcmds.h | 1 +
src/include/mb/pg_wchar.h | 6 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 11 +
src/include/pg_config.h.in | 6 +
src/include/utils/pg_locale.h | 32 +-
src/test/regress/GNUmakefile | 3 +
src/test/regress/expected/collate.icu.out | 1126 ++++++++++++++++++++++
src/test/regress/expected/collate.linux.utf8.out | 80 +-
src/test/regress/sql/collate.icu.sql | 433 +++++++++
src/test/regress/sql/collate.linux.utf8.sql | 32 +
45 files changed, 3929 insertions(+), 409 deletions(-)

Responses

Browse pgsql-committers by date

  From Date Subject
Next Message Peter Eisentraut 2017-03-23 20:07:23 Re: pgsql: ICU support
Previous Message Robert Haas 2017-03-23 18:29:33 pgsql: Track the oldest XID that can be safely looked up in CLOG.