Re: Logical Replication and Character encoding

>From: Peter Eisentraut [mailto:peter(dot)eisentraut(at)2ndquadrant(dot)com]
>Sent: Friday, February 24, 2017 1:32 AM
>To: Petr Jelinek <petr(dot)jelinek(at)2ndquadrant(dot)com>; Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
>Cc: craig(at)2ndquadrant(dot)com; Shinoda, Noriyoshi <noriyoshi(dot)shinoda(at)hpe(dot)com>; pgsql-hackers(at)postgresql(dot)org
>Subject: Re: [HACKERS] Logical Replication and Character encoding
>
>On 2/17/17 10:14, Peter Eisentraut wrote:
>> Well, it is sort of a libpq connection, and a proper libpq client
>> should set the client encoding, and a proper libpq server should do
>> encoding conversion accordingly. If we just play along with this, it
>> all works correctly.
>>
>> Other output plugins are free to ignore the encoding settings (just
>> like libpq can send binary data in some cases).
>>
>> The attached patch puts it all together.
>
>committed

Hi,

Thank you very much for making a new patch. I tried a new committed version.
In the case of PUBLICATION(EUC_JP) and SUBSCRIPTION(UTF-8) environment, it worked as expected. Great!.
However, in the case of PUBLICATION(UTF-8) and SUBSCRIOTION(EUC_JP) environment, the following error was output and the process went down.

- PUBLICATION (UTF-8)
postgres=> INSERT INTO encode1 VALUES (1, 'ascii') ;
INSERT 0 1
postgres=> INSERT INTO encode1 VALUES (2, '漢') ; -- Expect UTF-8 Character 0xE6BCA2 will be convert EUC_JP 0xB4C1
INSERT 0 1

- SUBSCRIPTION (EUC_JP)
postgres=> SELECT * FROM encode1;
c1 | c2
----+-------
1 | ascii
(1 row)

$ tail data.euc/pg_log/postgresql.log
LOG: starting logical replication worker for subscription "sub1"
LOG: logical replication apply for subscription "sub1" has started
ERROR: insufficient data left in message
LOG: worker process: logical replication worker for subscription 16439 (PID 22583) exited with exit code 1

Snapshot:
https://ftp.postgresql.org/pub/snapshot/dev/postgresql-snapshot.tar.gz 2017-02-24 00:28:58
Operating System:
Red Hat Enterprise Linux 7 Update 2 (x86-64)

Regards.