Re: SUBTRANS: Minimizing calls to SubTransSetParent()

On Tue, 9 Aug 2022 at 12:39, Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:

> > This short patchset compiles and passes make check-world, with lengthy comments.
>
> Does this patch set work independently or it has dependency on the
> patches on the other thread "Smoothing the subtrans performance
> catastrophe"?

Patches 001 and 002 are common elements of a different patch,
"Smoothing the subtrans performance catastrophe", but other than that,
the two patches are otherwise independent of each other.

i.e. there are common elements in both patches
001 puts all subxid data in a snapshot (up to a limit of 64 xids per
topxid), even if one or more xids overflows.

> Because in this patch I see no code where we are
> changing anything to control the access of SubTransGetParent() from
> SubTransGetTopmostTransactionId()?

Those calls are unaffected, i.e. they both still work.

Right now, we register all subxids in subtrans. But not all xids are
subxids, so in fact, subtrans has many "holes" in it, where if you
look up the parent for an xid it will just return
InvalidTransactionId. There is a protection against that causing a
problem because if you call TransactionIdDidCommit/Abort you can get a
WARNING, or if you call SubTransGetTopmostTransaction() you can get an
ERROR, but it is possible if you do a lookup for an inappropriate xid.
i.e. if you call TransactionIdDidCommit() without first calling
TransactionIdIsInProgress() as you are supposed to do.

What this patch does is increase the number of "holes" in subtrans,
reducing the overhead and making the subtrans data structure more
amenable to using a dense structure rather than a sparse structure as
we do now, which then leads to I/O overheads. But in this patch, we
only have holes when we can prove that the subxid's parent will never
be requested.

Specifically, with this patch, running PL/pgSQL with a few
subtransactions in will cause those subxids to be logged in subtrans
about 1% as often as they are now, so greatly reducing the number of
subtrans calls.

Happy to provide more detailed review thoughts, so please keep asking questions.

--
Simon Riggs http://www.EnterpriseDB.com/