RE: Support logical replication of DDLs

From: "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Japin Li <japinli(at)hotmail(dot)com>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, rajesh singarapu <rajesh(dot)rs0541(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Zheng Li <zhengli10(at)gmail(dot)com>
Subject: RE: Support logical replication of DDLs
Date: 2022-06-29 09:54:40
Message-ID: OS0PR01MB571676298B814F7F5690D92994BB9@OS0PR01MB5716.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

On Wednesday, June 29, 2022 11:07 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Tue, Jun 28, 2022 at 5:43 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
> wrote:
> >
> > 5.
> > +static ObjTree *
> > +deparse_CreateStmt(Oid objectId, Node *parsetree)
> > {
> > ...
> > + tmp = new_objtree_VA("TABLESPACE %{tablespace}I", 0); if
> > + (node->tablespacename) append_string_object(tmp, "tablespace",
> > + node->tablespacename); else { append_null_object(tmp, "tablespace");
> > + append_bool_object(tmp, "present", false); }
> > + append_object_object(createStmt, "tablespace", tmp);
> > ...
> > }
> >
> > Why do we need to append the objects (tablespace, with clause, etc.)
> > when they are not present in the actual CREATE TABLE statement? The
> > reason to ask this is that this makes the string that we want to send
> > downstream much longer than the actual statement given by the user on
> > the publisher.
> >
>
> After thinking some more on this, it seems the user may want to optionally
> change some of these attributes, for example, on the subscriber, it may want to
> associate the table with a different tablespace. I think to address that we can
> append these additional attributes optionally, say via an additional parameter
> (append_all_options/append_all_attributes or something like that) in exposed
> APIs like deparse_utility_command().

I agree and will research this part.

And here is the new version patch set.
Most of changes are in the deparser which include:

support CREATE PARTITIONED TABLE
support ALTER ATTACH/DETACH PARTITION
support CREATE/ALTER TABLE with ACCESS METHOD
support CREATE TABLE OF
support CREATE/ALTER TABLE with GENERATED COLUMN
support CREATE/ALTER TABLE with DENTITY COLUMN
support CREATE/ALTER TABLE with COMPRESSION METHOD
support ALTER COLUMN numofcol SET STATISTICS (mentioned by sawada-san [1])
support ALTER SCHEMA
support CRAETE/DROP INDEX

Note that, for ATTACH/DETACH PARTITION, I haven't added extra logic on
subscriber to handle the case where the table on publisher is a PARTITIONED
TABLE while the target table on subscriber side is NORMAL table. We will
research this more and improve this later.

Besides, the new version event trigger won't WAL log the DDL whose target
table is a temporary table so that the problem reported by Vignesh[2] is
fixed.

About the recent comment from Amit[3] and Vignesh[4], I will investigate the
comments and address them in next version.

[1] https://www.postgresql.org/message-id/CAD21AoBVCoPPRKvU_5-%3DwEXsa92GsNJFJOcYyXzvoSEJCx5dKw%40mail.gmail.com
[2] https://www.postgresql.org/message-id/CALDaNm33W35pcBE3zOpJhwnYBdBoZDpKxssemAN21NwVhJuong%40mail.gmail.com
[3] https://www.postgresql.org/message-id/CAA4eK1K88SMoBq%3DDRA4XU-F3FG6qyzCjGMMKsPpcRBPRcrELrw%40mail.gmail.com
[4] https://www.postgresql.org/message-id/CALDaNm3rEA_zmnDMOCT7NqK4aAffhAgooLf8rXjUN%3DYwA8ASFw%40mail.gmail.com

Best regards,
Hou zj

Attachment Content-Type Size
v10-0003-Support-CREATE-TABLE-AS-SELECT-INTO.patch application/octet-stream 14.6 KB
v10-0001-Functions-to-deparse-DDL-commands.patch application/octet-stream 132.6 KB
v10-0002-Support-DDL-replication.patch application/octet-stream 125.4 KB
v10-0004-Test-cases-for-DDL-replication.patch application/octet-stream 22.5 KB

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Amit Kapila 2022-06-29 10:48:12 Re: Support logical replication of DDLs
Previous Message houzj.fnst@fujitsu.com 2022-06-29 09:47:09 RE: Support logical replication of DDLs

Browse pgsql-hackers by date

  From Date Subject
Next Message huyajun 2022-06-29 09:56:39 Re: Implementing Incremental View Maintenance
Previous Message houzj.fnst@fujitsu.com 2022-06-29 09:47:09 RE: Support logical replication of DDLs