Re: Options OUTPUT_PLUGIN_* controlling format are confusing (Was: Misleading error message in logical decoding)

On 2014-09-18 08:57:26 -0500, Michael Paquier wrote:
> On Thu, Sep 18, 2014 at 2:18 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> > The point is that operating with byteas on SQL level is freaking
> > painful.
> An example perhaps? I fail to see why it is related to the fact that a
> user could simply use that to fetch changes in bytea from a slot:
> select data::bytea from pg_logical_slot_get_changes('foo', NULL, NULL);

Uh. Because that'd mean that pg_logical_slot_get_changes() would return
text, that's not actually text? I.e. it'll contain 0 bytes and things
outside the server's encoding? Not good.

> That's perhaps not more simple than using the binary functions, which
> actually explain their existence because of the textual/binary format
> context with OUTPUT_PLUGIN_*, but that's possible. It is as well
> possible to pay quite a lot with custom data types and casts to
> transform textual changes (not to mention that a plugin could send out
> bytea changes by itself through the COPY string). Looking at the code,
> the only difference between textual and binary is an assertion check
> using the database encoding, something unlikely to be triggered on
> production systems btw.

No. There's a *SIGNIFICANT* additional difference. If you use the *non
binary* get/peek_changes on a output plugin which has declared (and we
normally believe C code) that it produces binary output you will get an
error.

Again:
The output plugin *declares* whether it returns binary data (i.e. data
that's not in the server's encoding and/or possibly contains 0
bytes.). The textual get/peek_changes cannot accept changes from plugins
declaring that. The binary variant can accept changes regardless of
that.

I have the feeling that you have some fundamentally different
understanding of this in your head and aren't really following my
explanations.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services