From: | "R, Siva" <sivasubr(at)amazon(dot)com> |
---|---|
To: | pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Bug in gin insert redo code path during re-compression of empty gin data leaf pages |
Date: | 2018-07-17 22:40:12 |
Message-ID: | 1531867212836.63354@amazon.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hi,
We came across an issue during replay of a gin insert record on a pre-9.4 uncompressed data leaf page that does not have any items in it. The engine version where the replay is done is 9.6.3. The redo logic attempts to compress the existing page items before trying to insert new items from the WAL record[1]. In this particular case, we noticed that if the data leaf page does not have any items in it (see below on how such a page can be present), the compression should not be attempted as the algorithm expects the page to have at least one item [2]. This will lead to incorrect data size set on the page and also makes the assertion that expects npacked and nuncompressed to be equal, false [3].
In Postgres 9.3, when the gin posting tree is vacuumed, the pages that are in the leftmost and rightmost branches are not deleted [4]. These empty pages can be part of the database after upgrading to 9.6.3. We verified this by doing the following test:
Step 1: On Postgresql 9.3, create a table with gin index, insert and delete some data followed by vacuum.
* CREATE EXTENSION IF NOT EXISTS pg_trgm;
* CREATE TABLE gin_test (first_name text, last_name text);
* INSERT INTO gin_test SELECT md5(random()::text), md5(random()::text) FROM (SELECT * FROM generate_series(1,100000) AS id) AS x;
* CREATE INDEX gin_t_idx ON gin_test USING gin (first_name gin_trgm_ops, last_name gin_trgm_ops) WITH (fastupdate = OFF);
* DELETE FROM gin_test;
* VACUUM gin_test;
Step 2: Upgrade the database to 9.6.3 using pg_upgrade
* pg_upgrade -b $93_POSTGRES_DIR/bin -B $96_POSTGRES_DIR/bin -d $93_DATADIR -D $96_DATADIR
Step 3: Check the gin page opaque information of a uncompressed data leaf page to see if it has no items
* CREATE EXTENSION IF NOT EXISTS pageinspect;
* SELECT * FROM gin_metapage_info(get_raw_page('gin_t_idx', 0)); >>> Get number of data pages
* SELECT * FROM gin_page_opaque_info(get_raw_page('gin_t_idx', 8000)); >>> some block number close to total number of blocks
* This will return a page that has rightlink 2^32-1, maxoff 0 and flags {data,leaf}.
Attached is a patch that will skip compression of the posting list in the case that the data leaf page is empty.
Please let us know any comments or feedback. Thanks!
Best
Siva
*References:*
[1] - https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/access/gin/ginxlog.c;h=a40f1683dd80e4620ba6ccd93ae0e178e9616cf7;hb=refs/heads/REL9_6_STABLE#l147
[2] - https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/access/gin/ginpostinglist.c;h=54d5f6f630c4638474d3208489c5fb6eb1ca1eeb;hb=refs/heads/REL9_6_STABLE#l201
[3] - https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/access/gin/ginxlog.c;h=a40f1683dd80e4620ba6ccd93ae0e178e9616cf7;hb=refs/heads/REL9_6_STABLE#l163
[4] - https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/access/gin/ginvacuum.c;h=31b765287f6ac9da798d2637046e706f67bdac7a;hb=refs/heads/REL9_3_STABLE#l430
Attachment | Content-Type | Size |
---|---|---|
skip-compression-of-pre-9.4-empty-gin-data-leaf-page-replay_v1.patch | application/octet-stream | 1.7 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Konstantin Knizhnik | 2018-07-17 22:41:47 | Re: [HACKERS] PATCH: multivariate histograms and MCV lists |
Previous Message | Tom Lane | 2018-07-17 22:10:06 | Re: "Write amplification" is made worse by "getting tired" while inserting into nbtree secondary indexes (Was: Why B-Tree suffix truncation matters) |