Re: Options OUTPUT_PLUGIN_* controlling format are confusing (Was: Misleading error message in logical decoding)

Hi,

On 2014-09-18 02:14:48 -0500, Michael Paquier wrote:
> As there has been a dump of CATALOG_VERSION_NO on REL9_4_STABLE
> recently, I am coming back to the options OUTPUT_PLUGIN_* that are
> rather confusing for developers of decoder plugins. In short, when
> extracting changes from a replication slot, a decoder plugin is free
> to set one option that influences the output that is fetched by a
> logical receiver: OUTPUT_PLUGIN_TEXTUAL_OUTPUT or
> OUTPUT_PLUGIN_BINARY_OUTPUT. The interesting point about this format
> option is that it does not directly influence the changes generated by
> an output plugin: its value only has effect on the set of functions
> pg_logical_[get|peek]_[binary_|]changes that can be used by a
> non-replication connection to get individual changes from a repslot.
>
> So a receiver fetching changes using PQgetCopyData is not really
> impacted by this format option. Even better it is even possible to use
> a custom option that is part of output_plugin_options to switch the
> OUTPUT_PLUGIN_* value (cf option force-binary in
> contrib/test_decoding).
>
> My point is: logical decoding is presenting in its infrastructure API
> a format option that could be added as a custom option in a decoder
> plugin. Isn't that orthogonal? To illustrate this argument here is an
> example: we could remove force-binary in test_decoding and replace it
> by a option that allows the user to directly choose the output format,
> like format=[binary|textual] and do the conversion in the plugin. In
> the same manner, the functions pg_logical_[get|peek]_binary_changes
> are equivalent to their cousin pg_logical_[get|peek]_changes casted to
> bytea.

The point is that operating with byteas on SQL level is freaking
painful.

> I am raising this point before the 9.4 ship sails, thinking long-term
> and to faciliate the maintenance of existing code. Attached is a patch
> that simplifies the current logical decoding API regarding that.

Sorry, -1.

Improving the docs here is on my roadmap, but I don't see the benefit of
this.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services