brin autosummarization -- autovacuum "work items"

From: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>
To: Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: brin autosummarization -- autovacuum "work items"
Date: 2017-03-01 04:58:23
Message-ID: 20170301045823.vneqdqkmsd4as4ds@alvherre.pgsql
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I think one of the most serious issues with BRIN indexes is how they
don't get updated automatically as the table is filled. This patch
attempts to improve on that. During brininsert() time, we check whether
we're inserting the first item on the first page in a range. If we are,
request autovacuum to do a summarization run on that table. This is
dependent on a new reloption for BRIN called "autosummarize", default
off.

The way the request works is that autovacuum maintains a DSA which can
be filled by backends with "work items". Currently, work items can
specify a BRIN summarization of some specific index; in the future we
could use this framework to request other kinds of things that do not
fit in the "dead tuples / recently inserted tuples" logic that autovac
currently uses to decide to vacuum/analyze tables.

However, it seems I have not quite gotten the hang of DSA just yet,
because after a couple of iterations, crashes occur. I think the reason
has to do with either a resource owner clearing the DSA at an unwelcome
time, or perhaps there's a mistake in my handling of DSA "relative
pointers" stuff.

This patch was initially written by Simon Riggs, who envisioned that
brininsert itself would invoke the summarization. However, this doesn't
work because summarization requires having ShareUpdateExclusive lock,
which brininsert doesn't have. So I modified things to instead use the
DSA stuff. (He also set things up so that brininsert would only
summarize the just-filled range, but I didn't preserve that idea in the
autovacuum-based implementation; some changed lines there can probably
be removed.)

--
Álvaro Herrera PostgreSQL Expert, https://www.2ndQuadrant.com/

Attachment Content-Type Size
brin-autosummarize.patch text/plain 24.1 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Venkata B Nagothi 2017-03-01 04:59:16 Re: patch proposal
Previous Message Peter Eisentraut 2017-03-01 04:51:17 Re: rename pg_log directory?