pgsql: logical decoding: Fix handling of large old tuples with replica

From: Andres Freund <andres(at)anarazel(dot)de>
To: pgsql-committers(at)postgresql(dot)org
Subject: pgsql: logical decoding: Fix handling of large old tuples with replica
Date: 2016-03-06 02:16:52
Message-ID: E1acOFY-0001Zr-Au@gemulon.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

logical decoding: Fix handling of large old tuples with replica identity full.

When decoding the old version of an UPDATE or DELETE change, and if that
tuple was bigger than MaxHeapTupleSize, we either Assert'ed out, or
failed in more subtle ways in non-assert builds. Normally individual
tuples aren't bigger than MaxHeapTupleSize, with big datums toasted.
But that's not the case for the old version of a tuple for logical
decoding; the replica identity is logged as one piece. With the default
replica identity btree limits that to small tuples, but that's not the
case for FULL.

Change the tuple buffer infrastructure to separate allocate over-large
tuples, instead of always going through the slab cache.

This unfortunately requires changing the ReorderBufferTupleBuf
definition, we need to store the allocated size someplace. To avoid
requiring output plugins to recompile, don't store HeapTupleHeaderData
directly after HeapTupleData, but point to it via t_data; that leaves
rooms for the allocated size. As there's no reason for an output plugin
to look at ReorderBufferTupleBuf->t_data.header, remove the field. It
was just a minor convenience having it directly accessible.

Reported-By: Adam Dratwiński
Discussion: CAKg6ypLd7773AOX4DiOGRwQk1TVOQKhNwjYiVjJnpq8Wo+i62Q(at)mail(dot)gmail(dot)com

Branch
------
REL9_4_STABLE

Details
-------
http://git.postgresql.org/pg/commitdiff/3b94b3a496fd978ce481ba147627569bf7adc58f

Modified Files
--------------
contrib/test_decoding/expected/toast.out | 58 ++++++++++++
contrib/test_decoding/sql/toast.sql | 37 ++++++++
src/backend/replication/logical/decode.c | 82 ++++++++++-------
src/backend/replication/logical/reorderbuffer.c | 116 +++++++++++++++++-------
src/include/replication/reorderbuffer.h | 15 ++-
5 files changed, 236 insertions(+), 72 deletions(-)

Browse pgsql-committers by date

  From Date Subject
Next Message Andres Freund 2016-03-06 02:25:49 pgsql: Add valgrind suppression for padding bytes in twophase records.
Previous Message Andres Freund 2016-03-06 02:16:50 pgsql: logical decoding: Tell reorderbuffer about all xids.