Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths

From: Matthias van de Meent <boekewurm+postgres(at)gmail(dot)com>
To: David Zhang <david(dot)zhang(at)highgo(dot)ca>
Cc: Michael Paquier <michael(at)paquier(dot)xyz>, Andres Freund <andres(at)anarazel(dot)de>, Robert Haas <robertmhaas(at)gmail(dot)com>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths
Date: 2022-07-11 12:26:46
Message-ID: CAEze2Wjbthhf28uKrteQrAkJEzOpE=m-N5W6XPqTXyxS3xCCRw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, 8 Jul 2022 at 21:35, David Zhang <david(dot)zhang(at)highgo(dot)ca> wrote:
>
> Hi,
>
> I tried to apply this patch v5 to current master branch but it complains,
> "git apply --check
> v5-0001-Add-protections-in-xlog-record-APIs-against-large.patch
> error: patch failed: src/include/access/xloginsert.h:43
> error: src/include/access/xloginsert.h: patch does not apply"
>
> then I checked it out before the commit
> `b0a55e43299c4ea2a9a8c757f9c26352407d0ccc` and applied this v5 patch.

The attached rebased patchset should work with master @ 2cd2569c and
REL_15_STABLE @ 53df1e28. I've also added a patch that works for PG14
and earlier, which should be correct for all versions that include
commit 2c03216d (that is, all versions back to 9.5).

> 1) both make check and make installcheck passed.
>
> 2) and I can also see this patch v5 prevents the error happens previously,
>
> "postgres=# SELECT pg_logical_emit_message(false, long, long) FROM
> repeat(repeat(' ', 1024), 1024*1023) as l(long);
> ERROR: too much WAL data"
>
> 3) without this v5 patch, the same test will cause the standby crash
> like below, and the standby not be able to boot up after this crash.

Thanks for reviewing.

Kind regards,

Matthias van de Meent

Attachment Content-Type Size
v6-0001-Add-protections-in-xlog-record-APIs-against-large.patch application/octet-stream 9.3 KB
v6-0001-Add-protections-in-xlog-record-APIs-against-large.v15-patch application/octet-stream 8.1 KB
v6-0001-Add-protections-in-xlog-record-APIs-against-large.v14-patch application/octet-stream 8.0 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Matthias van de Meent 2022-07-11 13:07:47 Re: Commitfest Update
Previous Message Ranier Vilela 2022-07-11 12:25:15 Re: Reducing Memory Consumption (aset and generation)