Re: logical replication restrictions

On Mon, Aug 1, 2022 at 6:46 PM Euler Taveira <euler(at)eulerto(dot)com> wrote:
>
> On Tue, Jul 5, 2022, at 9:29 AM, Amit Kapila wrote:
>
> I wonder why we don't apply the delay on commit/commit_prepared
> records only similar to physical replication. See recoveryApplyDelay.
> One more advantage would be then we don't need to worry about
> transactions that we are going to skip due SKIP feature for
> subscribers.
>
> I added an explanation at the top of apply_delay(). I didn't read the "parallel
> apply" patch yet. I'll do soon to understand how the current design for
> streamed transactions conflicts with the parallel apply patch.
>
> + * It applies the delay for the next transaction but before starting the
> + * transaction. The main reason for this design is to avoid a long-running
> + * transaction (which can cause some operational challenges) if the user sets a
> + * high value for the delay. This design is different from the physical
> + * replication (that applies the delay at commit time) mainly because write
> + * operations may allow some issues (such as bloat and locks) that can be
> + * minimized if it does not keep the transaction open for such a long time.
> + */

Your explanation makes sense to me. The other point to consider is
that there can be cases where we may not apply operation for the
transaction because of empty transactions (we don't yet skip empty
xacts for prepared transactions). So, won't it be better to apply the
delay just before we apply the first change for a transaction? Do we
want to apply the delay during table sync as we sometimes do need to
enter apply phase while doing table sync?

>
> One more thing that might be worth discussing is whether introducing a
> new subscription parameter for this feature is a better idea or can we
> use guc (either an existing or a new one). Users may want to set this
> only for a particular subscription or set of subscriptions in which
> case it is better to have this as a subscription level parameter.
> OTOH, I was slightly worried that if this will be used for all
> subscriptions on a subscriber then it will be burdensome for users.
>
> That's a good point. Logical replication is per database and it is slightly
> different from physical replication that is per cluster. In physical
> replication, you have no choice but to have a GUC. It is very unlikely that
> someone wants to delay all logical replicas. Therefore, the benefit of having a
> GUC is quite small.
>

Fair enough.

--
With Regards,
Amit Kapila.