pgsql: snapshot scalability: cache snapshots using a xact completion co

From: Andres Freund <andres(at)anarazel(dot)de>
To: pgsql-committers(at)lists(dot)postgresql(dot)org
Subject: pgsql: snapshot scalability: cache snapshots using a xact completion co
Date: 2020-08-18 04:30:21
Message-ID: E1k7tGP-0005V0-5k@gemulon.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers pgsql-hackers

snapshot scalability: cache snapshots using a xact completion counter.

Previous commits made it faster/more scalable to compute snapshots. But not
building a snapshot is still faster. Now that GetSnapshotData() does not
maintain RecentGlobal* anymore, that is actually not too hard:

This commit introduces xactCompletionCount, which tracks the number of
top-level transactions with xids (i.e. which may have modified the database)
that completed in some form since the start of the server.

We can avoid rebuilding the snapshot's contents whenever the current
xactCompletionCount is the same as it was when the snapshot was
originally built. Currently this check happens while holding
ProcArrayLock. While it's likely possible to perform the check without
acquiring ProcArrayLock, it seems better to do that separately /
later, some careful analysis is required. Even with the lock this is a
significant win on its own.

On a smaller two socket machine this gains another ~1.03x, on a larger
machine the effect is roughly double (earlier patch version tested
though). If we were able to safely avoid the lock there'd be another
significant gain on top of that.

Author: Andres Freund <andres(at)anarazel(dot)de>
Reviewed-By: Robert Haas <robertmhaas(at)gmail(dot)com>
Reviewed-By: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
Reviewed-By: David Rowley <dgrowleyml(at)gmail(dot)com>
Discussion: https://postgr.es/m/20200301083601.ews6hz5dduc3w2se@alap3.anarazel.de

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/623a9ba79bbdd11c5eccb30b8bd5c446130e521c

Modified Files
--------------
src/backend/replication/logical/snapbuild.c | 1 +
src/backend/storage/ipc/procarray.c | 125 +++++++++++++++++++++++-----
src/backend/utils/time/snapmgr.c | 4 +
src/include/access/transam.h | 9 ++
src/include/utils/snapshot.h | 7 ++
5 files changed, 126 insertions(+), 20 deletions(-)

Responses

Browse pgsql-committers by date

  From Date Subject
Next Message Michael Paquier 2020-08-18 04:52:46 Re: pgsql: snapshot scalability: cache snapshots using a xact completion co
Previous Message Michael Paquier 2020-08-18 03:34:16 pgsql: Fix use-after-release issue in PL/Sample

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2020-08-18 04:52:46 Re: pgsql: snapshot scalability: cache snapshots using a xact completion co
Previous Message Amit Kapila 2020-08-18 04:06:21 Re: display offset along with block number in vacuum errors