Re: alternative model for handling locking in parallel groups

On Tue, Nov 18, 2014 at 8:53 AM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> Won't this be addressed because both updates issued from myfunc()
> are considered as separate commands, so w.r.t lock it should behave
> as 2 different updates in same transaction. I think there may be more
> things to make updates possible via parallel workers apart from tuple lock.

Hmm. It won't work to have different backends with different command
counters anyway. If backend 1 is using command counter 5 and backend
2 is using command counter 6, then backend 2 will see its snapshot
changing under it as backend 1 makes changes. That's not OK.

But that's not really the point. Heavyweight locks are sometimes used
to protect pretty low-level things, like the right to update a tuple,
or physically extend a relation by a block on disk, or split a bucket
in a hash index. It won't be OK to just ignore the requirement of
mutual exclusion in those cases, I think. We could of course decide
that the parallel processes must have their own mechanism to prevent
such conflicts apart from the lock manager, but they've got to be
prevented somehow.

>> After thinking about these cases for a bit, I came up with a new
>> possible approach to this problem. Suppose that, at the beginning of
>> parallelism, when we decide to start up workers, we grant all of the
>> locks already held by the master to each worker (ignoring the normal
>> rules for lock conflicts). Thereafter, we do everything the same as
>> now, with no changes to the deadlock detector. That allows the lock
>> conflicts to happen normally in the first two cases above, while
>> preventing the unwanted lock conflicts in the second two cases.
>
> Here I think we have to consider how to pass the information about
> all the locks held by master to worker backends. Also I think assuming
> we have such an information available, still it will be considerable work
> to grant locks considering the number of locks we acquire [1] (based
> on Simon's analysis) and the additional memory they require. Finally
> I think deadlock detector work might also be increased as there will be
> now more procs to visit.
>
> In general, I think this scheme will work, but I am not sure it is worth
> at this stage (considering initial goal to make parallel workers will be
> used for read operations).

I think this scheme might be quite easy to implement - we just need
the user backend to iterate over the locks it holds and serialize them
(similar to what pg_background does for GUCs) and store that in the
DSM; the parallel worker calls some function on that serialized
representation to put all those locks back in place. The actual
deadlock detector should need few or no changes, which seems like a
major advantage in comparison with the approach dicussed on the other
thread.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company