Re: Logical Replication and Character encoding

Hello,

At Tue, 31 Jan 2017 12:46:18 +0000, "Shinoda, Noriyoshi" <noriyoshi(dot)shinoda(at)hpe(dot)com> wrote in <AT5PR84MB0084FAE5976D89CDE9733093EE4A0(at)AT5PR84MB0084(dot)NAMPRD84(dot)PROD(dot)OUTLOOK(dot)COM>
> I tried a committed Logical Replication environment. I found
> that replication between databases of different encodings did
> not convert encodings in character type columns. Is this
> behavior correct?

The output plugin for subscription is pgoutput and it currently
doesn't consider encoding but would easiliy be added if desired
encoding is informed.

The easiest (but somewhat seems fragile) way I can guess is,

- Subscriber connects with client_encoding specification and the
output plugin pgoutput decide whether it accepts the encoding
or not. If the subscriber doesn't, pgoutput send data without
conversion.

The attached small patch does this and works with the following
CREATE SUBSCRIPTION.

CREATE SUBSCRIPTION sub1 CONNECTION 'host=/tmp port=5432 dbname=postgres client_encoding=EUC_JP' PUBLICATION pub1;

Also we may have explicit negotiation on, for example,
CREATE_REPLICATION_SLOT.

'CREATE_REPLICATION_SLOT sub1 LOGICAL pgoutput ENCODING EUC_JP'

Or output plugin may take options.

'CREATE_REPLICATION_SLOT sub1 LOGICAL pgoutput OPTIONS(encoding EUC_JP)'

Any opinions?

--
Kyotaro Horiguchi
NTT Open Source Software Center