Re: PG 14 release notes, first draft

On Mon, May 10, 2021 at 7:30 PM Justin Pryzby <pryzby(at)telsasoft(dot)com> wrote:
>
> Can these be merged:
> Allow logical replication to stream long transactions to standbys (Dilip Kumar, Tomas Vondra, Amit Kapila, Nikhil Sontakke)
> Improve the logical replication API to allow streaming large in-progress transactions (Tomas Vondra, Dilip Kumar, Amit Kapila)
>

I think we should merge the below two:
Add support for streaming to built-in logical replication (Vondra,
Dilip Kumar, Amit Kapila, Ajin Cherian, Nikhil Sontakke, Stas Kelvich)

Allow logical replication to stream long transactions to standbys
(Dilip Kumar, Tomas Vondra, Amit Kapila, Nikhil Sontakke)

and say something like:
Allow logical replication to stream long in-progress transactions to standbys
Previously transactions that exceeded logical_decoding_work_mem were
written to disk until the transaction completed.

Also, the commits that lead to the above feature is:
2020-09-03 [464824323] Add support for streaming to built-in logical replicatio
2020-08-08 [7259736a6] Implement streaming mode in ReorderBuffer.

* Improve the logical replication API to allow streaming large
in-progress transactions (Tomas Vondra, Dilip Kumar, Amit Kapila)
The output functions begin with "stream". test_decoding also supports these.

In the above, I think it is better to say Enhance instead of Improve.

* Add system view pg_stat_replication_slots to report replication slot
activity (Sawada Masahiko, Amit Kapila)

I think for this item, we can give Vignesh C as author credit because
he helped a lot enhance/stabilize this feature after the initial
commit. See
https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=f5fc2f5b23d1b1dff60f8ca5dc211161df47eda4
https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=cca57c1d9bf7eeba5b81115e0b82651cf3d8e4ea
https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=592f00f8dec68038301467a904ac514eddabf6cd

* Allow multiple xacts during table sync in logical replication (Peter
Smith, Amit Kapila, and Takamichi Osumi)

IMPORTANT?

Yes, this is a big enough improvement in the initial table sync
mechanism of logical replication that it is worth mentioning in
Release Notes. One of the main user-visible effects this commit can
have is that it will reduce the time for which we need to hold WAL on
the server during the initial table sync phase. Then it can avoid
large copies in case there is any error during the time when
apply-worker and tablesync-worker synchronizes the apply location.

* Enable logical replication to handle two phase commits (Ajin Cherian)
This is controlled via pg_create_logical_replication_slot().

We still haven't supported two-phase commits for built-in logical
replication but we have exposed APIs for decoding plugins to achieve
that. Using these APIs, we are planning to support it for built-in
logical replication but that will be probably done in PG-15. So, I
suggest we can modify it to something like:

* Enhance logical decoding APIs to handle two-phase commits
This is controlled via pg_create_logical_replication_slot().

Most of the work for this feature is done via commits:
https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=a271a1b50e9bec07e2ef3a05e38e7285113e4ce6
https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=0aa8a01d04c8fe200b7a106878eebc3d0af9105c

So, I think the author credit should be given to Ajin Cherian, Amit
Kapila, Nikhil Sontakke, and Stas Kelvich

* I think we should mention about commit:
https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=d7eb52d7181d83cf2363570f7a205b8eb1008dbc

This allows Logical decoding to work efficiently in presence of a
large amount of DDLs. This was a complaint from many users for long
time. Apart from the discussion thread, see some of the earlier
complaints:
https://www.postgresql.org/message-id/flat/20f3de7675f83176253f607b5e199b228406c21c(dot)camel(at)cybertec(dot)at
https://www.postgresql.org/message-id/flat/CAHoiPjzea6N0zuCi=+f9v_j94nfsy6y8SU7-=bp4=7qw6_i=Rg(at)mail(dot)gmail(dot)com

and most recently at:
https://www.postgresql.org/message-id/DB8PR10MB36126BC0DC3570637FDFED12EB4F9%40DB8PR10MB3612.EURPRD10.PROD.OUTLOOK.COM

* Speed truncation of small tables on large shared buffer servers (Kirk Jamison)

This optimization applies during recovery. Shouldn't we mention that
in some way?

--
With Regards,
Amit Kapila.