Re: [HACKERS] [WIP] Effective storage of duplicates in B-tree index.

On Mon, Nov 18, 2019 at 5:26 PM Peter Geoghegan <pg(at)bowt(dot)ie> wrote:
> Attached is v24. This revision doesn't fix the problem with
> xl_btree_insert record bloat

Attached is v25. This version:

* Adds more documentation.

* Adds a new GUC -- bree_deduplication.

A new GUC seems necessary. Users will want to be able to configure the
feature system-wide. A storage parameter won't let them do that --
only a GUC will. This also makes it easy to enable the feature with
unique indexes.

* Fixes the xl_btree_insert record bloat issue.

* Fixes a smaller issue with VACUUM/xl_btree_vacuum record bloat.

We shouldn't be using noticeably more WAL than before, at least in
cases that don't use deduplication. These two items fix cases where
that was possible.

There is a new refactoring patch including with v25 that helps with
the xl_btree_vacuum issue. This new patch removes unnecessary "pin
scan" code used by B-Tree VACUUMs, which was effectively disabled by
commit 3e4b7d87 without being removed. This is independently useful
work that I planned on doing already, that also cleans things up for
VACUUM with posting list tuples. It reclaims some space within the
xl_btree_vacuum record type that was wasted (we don't even use the
lastBlockVacuumed field anymore), allowing us to use that space for
new deduplication-related fields without increasing total WAL space.

Anastasia: I hope to be able to commit the first patch before too
long. It would be great if you could review that.

--
Peter Geoghegan