RE: Time delayed LR (WAS Re: logical replication restrictions)

Dear Amit,

> > > > Perhaps what I understand
> > > > correctly is that we could delay right before only sending commit
> > > > records in this case. If we delay at publisher end, all changes will
> > > > be sent at once if !streaming, and otherwise, all changes in a
> > > > transaction will be spooled at subscriber end. In any case, apply
> > > > worker won't be holding an active transaction unnecessarily.
> > >
> > > What about parallel case? Latest patch does not reject the combination of
> parallel
> > > streaming mode and delay. If delay is done at commit and subscriber uses an
> parallel
> > > apply worker, it may acquire lock for a long time.
> >
> > I didn't looked too closely, but my guess is that transactions are
> > conveyed in spool files in parallel mode, with each file storing a
> > complete transaction.
> >
>
> No, we don't try to collect all the data in files for parallel mode.
> Having said that, it doesn't matter because we won't know the time of
> the commit (which is used to compute delay) before we encounter the
> commit record in WAL. So, I feel for this approach, we can follow what
> you said.

Right. And new patch follows the opinion.

> > > > Of
> > > > course we need add the mechanism to process keep-alive and status
> > > > report messages.
> > >
> > > Could you share the good way to handle keep-alive and status messages if
> you have?
> > > If we changed to the decoding layer, it is strange to call walsender function
> > > directly.
> >
> > I'm sorry, but I don't have a concrete idea at the moment. When I read
> > through the last patch, I missed that WalSndDelay is actually a subset
> > of WalSndLoop. Although it can handle keep-alives correctly, I'm not
> > sure we can accept that structure..
> >
>
> I think we can use update_progress_txn() for this purpose but note
> that we are discussing to change the same in thread [1].
>
> [1] -
> https://www.postgresql.org/message-id/20230210210423.r26ndnfmuifie4f6%40
> awork3.anarazel.de

I did not reuse update_progress_txn() because we cannot use it straightforward,
But I can change if we have better idea than present.

New patch was posted in [1].

[1]: https://www.postgresql.org/message-id/TYAPR01MB5866F00191375D0193320A4DF5A19%40TYAPR01MB5866.jpnprd01.prod.outlook.com

Best Regards,
Hayato Kuroda
FUJITSU LIMITED