RE: Allow logical replication to copy tables in binary format

From: "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>
To: 'Melih Mutlu' <m(dot)melihmutlu(at)gmail(dot)com>
Cc: "shiy(dot)fnst(at)fujitsu(dot)com" <shiy(dot)fnst(at)fujitsu(dot)com>, "Takamichi Osumi (Fujitsu)" <osumi(dot)takamichi(at)fujitsu(dot)com>, Euler Taveira <euler(at)eulerto(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>
Subject: RE: Allow logical replication to copy tables in binary format
Date: 2023-03-08 05:24:55
Message-ID: TYAPR01MB5866E1D5B7E008AD62793108F5B49@TYAPR01MB5866.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Dear Melih,

>> I think we should add description to doc that it is more likely happen to fail
>> the initial copy user should enable binary format after synchronization if
>> tables have original datatype.
>
> I tried to explain when binary copy can cause failures in the doc. What exactly
> do you think is missing?

I assumed here that "copy_format" and "binary" were combined into one option.
Currently the binary option has descriptions :

```
Even when this option is enabled, only data types having binary send and receive functions will be transferred in binary
```

But this is not suitable for initial data sync, as we knew. I meant to say that
it must be updated if options are combined.

Note that following is not necessary for PG16, just an improvement for newer version.

Is it possible to automatically switch the binary option from 'true' to 'false'
when data transfer fails? As we found that while synchronizing the initial data
with binary format may lead another error, and it can be solved if the options
is changed. When DBAs check a log after synchronization and find the output like
"binary option was changed and worker will restart..." or something, they can turn
"binary" on again.

Best Regards,
Hayato Kuroda
FUJITSU LIMITED

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Keisuke Kuroda 2023-03-08 05:42:15 Re: Date-time extraneous fields with reserved keywords
Previous Message Peter Geoghegan 2023-03-08 05:21:00 Re: Testing autovacuum wraparound (including failsafe)