Re: Allow logical replication to copy tables in binary format

From: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
To: "shiy(dot)fnst(at)fujitsu(dot)com" <shiy(dot)fnst(at)fujitsu(dot)com>
Cc: Melih Mutlu <m(dot)melihmutlu(at)gmail(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, "Takamichi Osumi (Fujitsu)" <osumi(dot)takamichi(at)fujitsu(dot)com>, Euler Taveira <euler(at)eulerto(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Allow logical replication to copy tables in binary format
Date: 2023-02-27 09:01:37
Message-ID: CAA4eK1JSifU_Ja+O-Mt9d77ELup3BUZ+Yhio=_1J5S_YrKfJaQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Feb 20, 2023 at 3:37 PM shiy(dot)fnst(at)fujitsu(dot)com
<shiy(dot)fnst(at)fujitsu(dot)com> wrote:
>
> On Thu, Feb 16, 2023 8:48 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> > So, doesn't this mean that there is no separate failure mode during
> > the initial copy? I am clarifying this to see if the patch really
> > needs a separate copy_format option for initial sync?
> >
>
> In the case that the data type doesn't have binary output function, for apply
> phase, the column will be sent in text format (see logicalrep_write_tuple()) and
> it works fine. But with copy_format = binary, the walsender exits with an
> error.
>
...
...
>
> Then I got the following error in the publisher log.
>
> walsender ERROR: no binary output function available for type public.myvarchar
> walsender STATEMENT: COPY public.tbl1 (a) TO STDOUT WITH (FORMAT binary)
>

Thanks for sharing the example. I think to address this user can
create a SUBSCRIPTION with 'binary = false' and then after the initial
copy enables it with ALTER SUBSCRIPTION. Personally, I feel it is not
required to have a separate option to allow copy in binary mode. Note,
where there is some use for it but having more options for similar
work is also confusing as users need to pay attention to different
options and their values. It won't be difficult to add such an option
in the future if we see such cases and or users specifically require
something like this.

--
With Regards,
Amit Kapila.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Heikki Linnakangas 2023-02-27 09:10:38 Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)
Previous Message Kyotaro Horiguchi 2023-02-27 08:54:06 Re: Provide PID data for "cannot wait on a latch owned by another process" in latch.c