GiST index corruption with large tuples

Hi,

We have been seeing a GiST index becoming corrupt when large tuples are
inserted into an indexed table. On our production database we've seen the
corruption manifest itself as various obscure errors, including: "failed to
add item to index page", "fixing incomplete split in index", "no unpinned
buffers available", and "stack depth limit exceeded" during INSERT-s and
REINDEX-es.

We've also observed that if we ignore these errors and retry our transaction
repeatedly, the index will sometimes start to grow without bounds -- until we
run out of disk space.

We've reproduced the bug on both CentOS 7 and Fedora 30 using the official RPM repositories.

* PostgreSQL 11.3 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36), 64-bit
* PostgreSQL 11.3 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 9.0.1 20190312 (Red Hat 9.0.1-0.10), 64-bit

Here's a small reproducer:

CREATE EXTENSION IF NOT EXISTS btree_gist;
CREATE TABLE aoeu(id INT GENERATED BY DEFAULT AS IDENTITY, value TEXT NOT NULL);
CREATE INDEX aoeu_value_idx ON aoeu USING GIST (value);

INSERT INTO aoeu (value) VALUES (repeat('a', 65536*9));
INSERT INTO aoeu (value) VALUES (repeat('a', 65536));

-- ERROR: XX000: failed to add item to index page in "aoeu_value_idx"
-- LOCATION: gistplacetopage, gist.c:417
INSERT INTO aoeu (value) VALUES (repeat('a', 65536));

-- ERROR: 54000: index row requires 13536 bytes, maximum size is 8191
-- LOCATION: index_form_tuple, indextuple.c:177
REINDEX INDEX aoeu_value_idx;

Please let us know if there's any additional information requested.

Thanks in advance,
Gabríel