Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

On Mon, Feb 17, 2020 at 2:42 AM Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>
> Andres Freund <andres(at)anarazel(dot)de> writes:
> > On 2020-02-14 13:34:03 -0500, Robert Haas wrote:
> >> I think the group locking + deadlock detection things are more
> >> fundamental than you might be crediting, but I agree that having
> >> parallel mechanisms has its own set of pitfalls.
>
> > It's possible. But I'm also hesitant to believe that we'll not need
> > other lock types that conflict between leader/worker, but that still
> > need deadlock detection. The more work we want to parallelize, the more
> > likely that imo will become.
>
> Yeah. The concept that leader and workers can't conflict seems to me
> to be dependent, in a very fundamental way, on the assumption that
> we only need to parallelize read-only workloads. I don't think that's
> going to have a long half-life.
>

Surely, someday, we need to solve that problem. But it is not clear
when because if we see the operations for which we want to solve the
relation extension lock problem doesn't require that. For example,
for a parallel copy or further improving parallel vacuum to allow
multiple workers to scan and process the heap and individual index, we
don't need to change anything in group locking as far as I understand.

Now, for parallel deletes/updates, I think it will depend on how we
choose to parallelize those operations. I mean if we decide that each
worker will work on an independent set of pages like we do for a
sequential scan, we again might not need to change the group locking
unless I am missing something which is possible.

I think till we know the real need for changing group locking, going
in the direction of what Tom suggested to use an array of LWLocks [1]
to address the problems in hand is a good idea. It is not very clear
to me that are we thinking to give up on Tom's idea [1] and change
group locking even though it is not clear or at least nobody has
proposed an idea/patch which requires that? Or are we thinking that
we can do what Tom suggested for relation extension lock and also plan
to change group locking for future parallel operations that might
require it?

[1] - https://www.postgresql.org/message-id/19443.1581435793%40sss.pgh.pa.us

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com