Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock(PG10.7)

On Sun, Sep 29, 2019 at 10:53 PM Peter Geoghegan <pg(at)bowt(dot)ie> wrote:
> On Sun, Sep 29, 2019 at 7:38 AM Alexander Korotkov
> <a(dot)korotkov(at)postgrespro(dot)ru> wrote:
> > Starting from root seems OK for me, because vacuum blocks all
> > concurrent inserts before doing this. But this needs to be properly
> > documented in readme.
>
> I never got an adequate answer to this closely related question almost
> two years ago:
>
> https://www.postgresql.org/message-id/CAH2-Wz=GTnAPzEEZqYELOv3h1Fxpo5xhMrBP6aMGEKLKv95csQ@mail.gmail.com
>
> In general, ginInsertCleanup() seems badly designed. Why is it okay
> that there is no nbtree-style distinction between page deletion and
> page recycling?

Thank you for pointing. I remember this thread, but don't remember
details. I'll reread it.

> > Locking from right to left is clearly wrong. It could deadlock with
> > concurrent ginStepRight(), which locks from left to right. I expect
> > this happened in your case. I'm going to reproduce this and fix.
>
> I am sick and tired of seeing extremely basic errors like this within
> GIN's locking protocols. Bugs happen, but these are not ordinary bugs.
> They're more or less all a result of giving no thought to the high
> level design. I'm not blaming you for this, or any one person. But
> this is not okay.
>
> Anything around index concurrency needs to be explained in
> excruciating detail, while taking a top-down approach that applies
> general rules (e.g. you can only do lock coupling left to right, or
> bottom to top in nbtree). Anything less than that should be assumed to
> be wrong on general principle.

Frankly speaking I'm not very happy with special version of B-tree,
which is builtin to GIN. This version of B-tree is lacking of high
keys. AFAIR because of lack of high keys, we can't implement the same
concurrency model as nbtree. Instead we have to do super-locking for
page deletion and so on. That looks ridiculous for me. I think in
future we should somehow reimplement GIN on top of nbtree.

But we have to provide some way less radical fixes for our stable
releases. In particular, I believe patch I've posted in this thread
makes situation better not worse. That is it fixes one bug without
introducing mode bugs. But I'm going to analyze more on this and
document GIN concurrency better in the README. Probably, I'll spot
more details.

------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company