Re: Built-in plugin for logical decoding output

On 25/09/17 18:48, Alvaro Hernandez wrote:
>
>
> On 25/09/17 19:39, Petr Jelinek wrote:
>>
>> Well, test_decoding is not meant for production use anyway, no need for
>> middleware to support it. The pgoutput is primarily used for internal
>> replication purposes, which is why we need something with more
>> interoperability in mind in the first place. The new plugin should still
>> support publications etc though IMHO.
>>
>>>      However, having said that, and while json is a great output format
>>> for interoperability, if there's a discussion on which plugin to include
>>> next, I'd also favor one that has some more compact representation
>>> format (or that supports several formats, not only json).
>>>
>> JSON is indeed great for interoperability, if you want more compact
>> format, use either pgoutput or write something of your own or do
>> conversion to something else in your consumer. I don't think postgres
>> needs to provide 100 different formats out of the box when there is an
>> API. The JSON output does not have to be extremely chatty either btw.
>>
>
>     In my opinion, logical decoding plugins that don't come with core
> are close to worthless (don't get me wrong):
>

I respectfully disagree.

> - They very unlikely will be installed in managed environments (an area
> growing significantly).
> - As anything that is not in core, raises concerns by users.
> - Distribution and testing are non-trivial: many OS/archs combinations.
>
>     Given the above, I believe having a general-purpose output plugin
> in-core is critical to the use of logical decoding. As for 9.4-9.6 there
> is test_decoding, and given that AWS uses it for production, that's kind
> of fine. For 10 there is at least pgoutput, which could be used (even
> though it was meant for replication). But if a new plugin is to be
> developed for 11+, one really general purpose one, I'd say json is not a
> good choice if it is the only output it would support. json is too
> verbose, and replication, if anything, needs performance (it is both
> network heavy and serialization/deserialization is quite expensive). Why
> not, if one and only one plugin would be developed for 11+, general
> purpose, do something that is, indeed, more general, i.e., that supports
> high-performance scenarios too?
>

IMO for general purpose use the adoption/ease of parsing is more
important criteria which is why JSON is good option. Again if you need
something more tuned to performance and you are fine with some hardship
in terms of parsing, what's stopping you from using pgoutput?

--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services