Re: Proposal: New LWLockmode LW_OWNER

Tom Lane wrote:
> "Jignesh K. Shah" <J(dot)K(dot)Shah(at)Sun(dot)COM> writes:
>
>> Tom Lane wrote:
>>
>>> This seems rather crazy, and you haven't actually given a single
>>> convincing use-case.
>>>
>
>
>> One area that I find it useful is where it will be useful is in
>> ProcArrayEndTransaction where it uses exclusive to update proc array
>> structure where right now it uses EXCLUSIVE and most commit transactions
>> are updating their own proc array structure this lock semantic can be
>> useful..
>>
>
> That is exactly a place where you CAN'T use this, because it will break
> transactional semantics, specifically serialization of commits relative
> to snapshots. See all the discussions around the last refactoring of
> ProcArray locking, and particularly the summary in
> src/backend/access/transam/README.
>
>
>

Quoting from the README that you mentioned:

Formally, the correctness requirement is "if a snapshot A considers
transaction X as committed, and any of transaction X's snapshots
considered transaction Y as committed, then snapshot A must consider
transaction Y as committed".

What we actually enforce is strict serialization of commits and
rollbacks with snapshot-taking: we do not allow any transaction to exit
the set of running transactions while a snapshot is being taken. (This
rule is stronger than necessary for consistency, but is relatively
simple to enforce, and it assists with some other issues as explained
below.) The implementation of this is that GetSnapshotData takes the
ProcArrayLock in shared mode (so that multiple backends can take
snapshots in parallel), but ProcArrayEndTransaction must take the
ProcArrayLock in exclusive mode while clearing MyProc->xid at
transaction end (either commit or abort).

ProcArrayEndTransaction also holds the lock while advancing the shared
latestCompletedXid variable. This allows GetSnapshotData to use
latestCompletedXid + 1 as xmax for its snapshot: there can be no
transaction >= this xid value that the snapshot needs to consider as
completed.

Quote End

What I understand is the rule of serializations comes into play when
Snapshot is being taken... Snapshot is being taken only when SHARED lock
has already been acquired. If EX_OWNER is being used in this scenario,
it still works as designed. If EX_OWNER has been acquired, NOBODY can
get SHARED Lock which means the rule is still satisfied. Of course I am
still worried about latestCompletedXid being written at the same time
(maybe use some atomic compare and swap function for it to solve the
problem) but I dont understand why you think that this is the place
where it cannot be used?

Simon, Scalability increases 50% from what I see (throughput increases
from 200k tpm to 300k tpm and system utilization went from 50% to 80% or
so) and is a step forward in overall performance and system utilization.
Also response time on high loads also falls drastically to something
similar to low load response times.

Regards,
Jignesh