Re: Speedup of relation deletes during recovery

From: Andres Freund <andres(at)anarazel(dot)de>
To: Fujii Masao <masao(dot)fujii(at)gmail(dot)com>
Cc: Michael Paquier <michael(at)paquier(dot)xyz>, Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Speedup of relation deletes during recovery
Date: 2018-06-15 17:45:04
Message-ID: 20180615174504.cf2ba4p6xuu36hn6@alap3.anarazel.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

We just had a customer hit this issue. I kind of wonder whether this
shouldn't be backpatched: Currently the execution on the primary is
O(NBuffers * log(ndrels)) whereas it's O(NBuffers * ndrels) on the
standby - with a lot higher constants to boot. That means it's very
easy to get into situations where the standy starts to lag behind very significantly.

> --- a/src/backend/access/transam/twophase.c
> +++ b/src/backend/access/transam/twophase.c
> @@ -1445,6 +1445,7 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
> int ndelrels;
> SharedInvalidationMessage *invalmsgs;
> int i;
> + SMgrRelation *srels = NULL;
>
> /*
> * Validate the GID, and lock the GXACT to ensure that two backends do not
> @@ -1534,13 +1535,16 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
> delrels = abortrels;
> ndelrels = hdr->nabortrels;
> }
> +
> + srels = palloc(sizeof(SMgrRelation) * ndelrels);
> for (i = 0; i < ndelrels; i++)
> - {
> - SMgrRelation srel = smgropen(delrels[i], InvalidBackendId);
> + srels[i] = smgropen(delrels[i], InvalidBackendId);
>
> - smgrdounlink(srel, false);
> - smgrclose(srel);
> - }
> + smgrdounlinkall(srels, ndelrels, false);
> +
> + for (i = 0; i < ndelrels; i++)
> + smgrclose(srels[i]);
> + pfree(srels);

This code is now duplicated three times - shouldn't we just add a
function that encapsulates dropping relations in a commit/abort record?

Greetings,

Andres Freund

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Teodor Sigaev 2018-06-15 17:54:41 Re: Speedup of relation deletes during recovery
Previous Message Neil Conway 2018-06-15 17:22:30 Re: row_to_json(), NULL values, and AS