Re: Signaling of waiting for a cleanup lock?

On Mon, Apr 14, 2014 at 12:02 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Andres Freund <andres(at)2ndquadrant(dot)com> writes:
>> On 2014-04-14 11:30:02 -0400, Tom Lane wrote:
>>> I wonder whether we should not try to fix this by making the process wait
>>> on a heavyweight lock, if it has to wait. That would also get us out of
>>> the rather grotty business of using a special-purpose signal to wake it
>>> up. However, there's still a visibility problem, in that there'd be no
>>> way to tell which other processes are blocking it (which is the thing
>>> you *really* want to know).
>
>> That'd be neat, but I am not really sure how? Which lock would we be
>> waiting on?
>
> Well, we already have locktags for relation pages, so that aspect isn't
> that hard. The bigger problem is what are we waiting *for*, that is,
> what is it that blocks the lock request from being granted?
>
> In an ideal world, when we needed to wait for a cleanup lock, we'd cause
> the lock manager to set up pre-granted sharable page locks for all the
> processes currently holding buffer pins, and then wait for an exclusive
> page lock. The current hack of signaling when you're the last one off the
> page would be replaced by releasing your lock (if it exists) when you drop
> your own pin. I'm not sure it's really worth the trouble to try to do
> this, but it would solve the visibility problem; and it might allow us to
> be a bit smarter about the priority of a cleanup lock request versus
> incoming regular pin requests.

AFAICS, the big advantage of something like this is that we'd get
proper deadlock detection, and that's not a trivial point. But other
than that I don't like it much. The fast-path locking stuff was
basically this kind of thing in reverse: instead of trying to
implement something with a separate locking implementation into the
regular lock manager, we were trying to take something that was
handled by the regular lock manager in a completely generic way and
give it an optimized path that performs much better on the cases that
actually arise in practice. And, it was worth it, because we got a
huge performance boost, but it was also messy and, from your recent
reports, apparently still not fully debugged.

Now, I don't really have a specific proposal in mind that I think
would be better than shoehorning more stuff into the lock manager, so
it would be hard for me to oppose that if someone worked out all of
the problems and proposed a patch. But I have uneasy feelings about
it. It's hard to see how we can have a reasonable deadlock detector
without some kind of very generic locking mechanism, and that's what
the lock manager is, but we also know from experience that it's really
slow in some circumstances and frustratingly easy to run out of shared
memory, so I can't help but wonder if we really ought to be giving the
whole way we do locking and deadlock detection a broader rethink
somehow.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company