Re: Support logical replication of DDLs

From: Zheng Li <zhengli10(at)gmail(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Japin Li <japinli(at)hotmail(dot)com>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, rajesh singarapu <rajesh(dot)rs0541(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Peter Smith <smithpb2250(at)gmail(dot)com>
Subject: Re: Support logical replication of DDLs
Date: 2022-08-08 20:22:33
Message-ID: CAAD30UL6CVphMN3Fn3iwfU=oh0GOasygxM19ZCzB6o+RZkD49Q@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

> In general, I agree with your comment below that we can work on this
> after we have some more concrete plans/discussions. I think we can
> probably consider this when we have more discussion around the
> publication commands for the DDL objects. However, it would be good if
> you can add some more details about the above two options.

Thanks for the support. I have started a new thread on supporting replication of
global object command and have added more detailed discussion:
https://www.postgresql.org/message-id/CAAD30UKD7YPEbYcs_L9PYLcLZjnxyqO%3DJF5_mnAwx7g_PtOi3A%40mail.gmail.com

> I noticed that LogLogicalDDLMessage() uses MyDatabaseId and then
> decoding can take some action (filtering) based on that. Is it Okay to
> use that function for global objects, if so, you might want to add a
> comment for the same?

Could you elaborate on the concern? The dbid filtering happens in
logicalddlmsg_decode but I don't see why I can't use LogLogicalDDLMessage
to log global commands. Are there any global commands that are
non-transactional?
logicalddlmsg_decode:
if (message->dbId != ctx->slot->data.database ||
FilterByOrigin(ctx, origin_id))
return;

> As per my understanding, the overall work on this project includes the
> following sub-tasks:
> a. DDL Deparsing: This is required to support DDL replication of
> non-global objects. The work for this is in progress, this is based on
> prior work by Alvaro.
> b. DDL Replication: This includes replication of DDL commands based on
> event triggers and DDL deparsing. The work on this is also in
> progress.
> c. DDL Replication of global objects: It requires a different approach
> due to the reasons quoted above in your email. Zheng has started
> working on it.
> d. Initial Sync: I think there is a brief discussion about this in the
> thread but no concrete proposal yet. I think it is important to solve
> this part of the puzzle as well to have an overall design ready for
> this project. Do let me know if you, Sawada-San, or anybody else
> intends to work on it? I think that will avoid overlap of work.

Euler mentioned that he has plan to work on the initial schema sync in
[1]. We can help with this effort as well.

Regards,
Zheng

[1] https://www.postgresql.org/message-id/45d0d97c-3322-4054-b94f-3c08774bbd90%40www.fastmail.com

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Saul Perdomo 2022-08-08 23:55:23 Re: 20220722-pg_dump: error: invalid number of parents 0 for table
Previous Message Marcos Pegoraro 2022-08-08 19:26:36 Re: pg_restore remap schema

Browse pgsql-hackers by date

  From Date Subject
Next Message Thomas Munro 2022-08-08 20:30:11 Re: Checking pgwin32_is_junction() errors
Previous Message Zheng Li 2022-08-08 20:01:33 Support logical replication of global object commands