Re: Allow logical replication to copy tables in binary format

From: Peter Smith <smithpb2250(at)gmail(dot)com>
To: Melih Mutlu <m(dot)melihmutlu(at)gmail(dot)com>
Cc: "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>, "shiy(dot)fnst(at)fujitsu(dot)com" <shiy(dot)fnst(at)fujitsu(dot)com>, "Takamichi Osumi (Fujitsu)" <osumi(dot)takamichi(at)fujitsu(dot)com>, Euler Taveira <euler(at)eulerto(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>
Subject: Re: Allow logical replication to copy tables in binary format
Date: 2023-03-02 01:57:09
Message-ID: CAHut+PtcbBRNwsFQGjs1VngiNkRwwsbDNA3GX1V=W01=_iWGAQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, Mar 2, 2023 at 5:10 AM Melih Mutlu <m(dot)melihmutlu(at)gmail(dot)com> wrote:
>
> Hi,
>
> Hayato Kuroda (Fujitsu) <kuroda(dot)hayato(at)fujitsu(dot)com>, 1 Mar 2023 Çar, 18:40 tarihinde şunu yazdı:
>>
>> Dear Melih,
>>
>> If we do not have to treat the case Shi pointed out[1] as code-level, I agreed to
>> same option binary because it is simpler.
>
>
> How is this an issue if we let the binary option do binary copy and not an issue if we have a separate copy_binary option?
> You can easily have the similar errors when you set copy_binary=true if a type is missing binary send/receive functions.
> And also, as Amit mentioned, the same issue can easily be avoided if binary=false until the initial sync is done. It can be set to true later.
>
>>

IIUC most people seem to be coming down in favour of there being a
single unified option (the existing 'binary==true/false) which would
apply to both the COPY and the data replication parts.

I also agree
- Yes, it is simpler.
- Yes, there are various workarounds in case the COPY part failed

But, AFAICT the main question remains unanswered -- Are we happy to
break existing applications already using binary=true. E.g. I think
there might be cases where applications are working *only* because
their binary=true is internally (and probably unbeknownst to the user)
reverting to text. So if we unified everything under one 'binary'
option then binary=true will force COPY binary so now some previously
working applications will get COPY errors requiring workarounds. Is
that acceptable?

TBH I am not sure anymore if the complications justify the patch.

It seems we have to choose from 2 bad choices:
- separate options = this works but would be more confusing for the user
- unified option = this would be simpler and faster, but risks
breaking existing applications currently using 'binary=true'

------
Kind Regards,
Peter Smith.
Fujitsu Australia

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Masahiko Sawada 2023-03-02 02:08:00 Re: Time delayed LR (WAS Re: logical replication restrictions)
Previous Message Melanie Plageman 2023-03-02 01:41:14 Re: Should vacuum process config file reload more often