pgsql: Properly detoast data in brin_form_tuple

From: Tomas Vondra <tomas(dot)vondra(at)postgresql(dot)org>
To: pgsql-committers(at)lists(dot)postgresql(dot)org
Subject: pgsql: Properly detoast data in brin_form_tuple
Date: 2020-11-06 23:40:55
Message-ID: E1kbBLj-0006Pl-61@gemulon.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Properly detoast data in brin_form_tuple

brin_form_tuple failed to consider the values may be toasted, inserting
the toast pointer into the index. This may easily result in index
corruption, as the toast data may be deleted and cleaned up by vacuum.
The cleanup however does not care about indexes, leaving invalid toast
pointers behind, which triggers errors like this:

ERROR: missing chunk number 0 for toast value 16433 in pg_toast_16426

A less severe consequence are inconsistent failures due to the index row
being too large, depending on whether brin_form_tuple operated on plain
or toasted version of the row. For example

CREATE TABLE t (val TEXT);
INSERT INTO t VALUES ('... long value ...')
CREATE INDEX idx ON t USING brin (val);

would likely succeed, as the row would likely include toast pointer.
Switching the order of INSERT and CREATE INDEX would likely fail:

ERROR: index row size 8712 exceeds maximum 8152 for index "idx"

because this happens before the row values are toasted.

The bug exists since PostgreSQL 9.5 where BRIN indexes were introduced.
So backpatch all the way back.

Author: Tomas Vondra
Reviewed-by: Alvaro Herrera
Backpatch-through: 9.5
Discussion: https://postgr.es/m/20201001184133.oq5uq75sb45pu3aw@development
Discussion: https://postgr.es/m/20201104010544.zexj52mlldagzowv%40development

Branch
------
REL_12_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/8149e9f9a0d684dce883d9e5a9f7380afbdbd7d6

Modified Files
--------------
src/backend/access/brin/brin_tuple.c | 91 +++++++++++++++++++++++++++++++++++-
src/test/regress/expected/brin.out | 41 ++++++++++++++++
src/test/regress/sql/brin.sql | 39 ++++++++++++++++
3 files changed, 170 insertions(+), 1 deletion(-)

Browse pgsql-committers by date

  From Date Subject
Next Message Tomas Vondra 2020-11-06 23:41:12 pgsql: Properly detoast data in brin_form_tuple
Previous Message Tomas Vondra 2020-11-06 23:40:15 pgsql: Properly detoast data in brin_form_tuple