Re: Built-in plugin for logical decoding output

2017-09-22 19:28 GMT-03:00 Gregory Brail <gregbrail(at)google(dot)com>:
> We have been working on a project that makes extensive use of logical
> replication for use inside Apigee (which is a very small part of Google):
>
> https://github.com/apigee-labs/transicator
>
> In order to do this, we had to write our own logical replication plugin
> because the supplied "test_decoding" plugin from the "contrib" directory was
> hard for us to work with. Primarily:
>
test_decoding is a proof of concept to illustrate the logical decoding
potential. It is not intended for production. I developed wal2json [1]
for general use. It outputs changes in JSON. It was one of the first
logical decoding plugins.

> 1) It doesn't include all the fields that we need for Transicator (most
> importantly we need the LSN and the 64-bit transaction ID).
>
wal2json includes both.

> 2) It outputs a text format that is hard to parse.
>
There are a lot of JSON parsers.

> I imagine that other users of logical decoding are facing similar problems.
>
> Would the community support the development of another plugin that is
> distributed as part of "contrib" that addresses these issues? I'd be happy
> to submit a patch, or GitHub repo, or whatever works best as an example.
> (Also, although Transicator uses protobuf, I'm happy to have it output a
> simple binary format as well.)
>
There was a prior discussion and it was suggestted that we have a
ready-for-production plugin in core (besides pgoutput). It was
suggested [1] that I submit wal2json for 11. I'm in process to clean
up the code and hope to submit it to CF2.

[1] https://github.com/eulerto/wal2json
[2] https://www.postgresql.org/message-id/CAHE3wggh6ucSCB%2BhnSK04xEQx75f3DTz0wPSjRMJFjum6pRrPQ%40mail.gmail.com

--
Euler Taveira Timbira -
http://www.timbira.com.br/
PostgreSQL: Consultoria, Desenvolvimento, Suporte 24x7 e Treinamento