Re: Force streaming every change in logical decoding

On Tue, Dec 6, 2022 at 11:53 AM shiy(dot)fnst(at)fujitsu(dot)com
<shiy(dot)fnst(at)fujitsu(dot)com> wrote:
>
> Hi hackers,
>
> In logical decoding, when logical_decoding_work_mem is exceeded, the changes are
> sent to output plugin in streaming mode. But there is a restriction that the
> minimum value of logical_decoding_work_mem is 64kB. I tried to add a GUC to
> allow sending every change to output plugin without waiting until
> logical_decoding_work_mem is exceeded.
>
> This helps to test streaming mode. For example, to test "Avoid streaming the
> transaction which are skipped" [1], it needs many XLOG_XACT_INVALIDATIONS
> messages. With the new option, it can be tested with fewer changes and in less
> time. Also, this new option helps to test more scenarios for "Perform streaming
> logical transactions by background workers" [2].

Some comments on the patch

1. Can you add one test case using this option

2. + <varlistentry id="guc-force-stream-mode" xreflabel="force_stream_mode">
+ <term><varname>force_stream_mode</varname> (<type>boolean</type>)
+ <indexterm>
+ <primary><varname>force_stream_mode</varname> configuration
parameter</primary>
+ </indexterm>
+ </term>

This GUC name "force_stream_mode" somehow appears like we are forcing
this streaming mode irrespective of whether the
subscriber has requested for this mode or not. But actually it is not
that, it is just streaming each change if
it is enabled. So we might need to think on the name (at least we
should avoid using *mode* in the name IMHO).

3.
- while (rb->size >= logical_decoding_work_mem * 1024L)
+ while ((!force_stream && rb->size >= logical_decoding_work_mem * 1024L) ||
+ (force_stream && rb->size > 0))
{

It seems like if force_stream is on then indirectly it is enabling
force serialization as well. Because once we enter into the loop
based on "force_stream" then it will either stream or serialize but I
guess we do not want to force serialize based on this parameter.

--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com