Re: condition variables

On Thu, Aug 11, 2016 at 6:12 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> But if we replace the io_in_progress locks with
>> condition variables, then that doesn't happen any more. Nobody is
>> "holding" the condition variable, so it doesn't get "released" when
>> the process doing I/O aborts. Instead, they just keep sleeping until
>> the aborting process reaches AbortBufferIO, and then it broadcasts on
>> the condition variable and wakes everybody up, which seems a good deal
>> nicer.
>
> Hmm. I fear the only reason you see an advantage there is that you don't
> (yet) have any general-purpose mechanism for an aborting transaction to
> satisfy its responsibilities vis-a-vis waiters on condition variables.

In the specific case of parallel query, this problem doesn't really
arise: if an error happens, it will be propagated back to the leader
(unless it occurs in the leader as an initial matter) and the leader
will then kill of all of the other workers. Since waiting for a
condition variable doesn't block interrupts, the workers will all stop
waiting and die like the obedient lemmings they are.

> Instead, this wins specifically because you stuck some bespoke logic into
> AbortBufferIO. OK ... but that sounds like we're going to end up with
> every single condition variable that ever exists in the system needing to
> be catered for separately and explicitly during transaction abort cleanup.
> Which does not sound promising from a reliability standpoint. On the
> other hand, I don't know what the equivalent rule to "release all LWLocks
> during abort" might look like for condition variables, so I don't know
> if it's even possible to avoid that.

I don't think it is possible to avoid that. Certainly the fact that
LWLocks are automatically released is in most scenarios a big
advantage, but for content locks it isn't. I note that I didn't so
much insert bespoke logic there as replace the existing bespoke logic
with less-ugly bespoke logic.

One way to conceptualize a condition variable is that it is the wait
queue for an LWLock without the lock itself. The "lock modes" are
defined by the interaction between the condition checked by waiters
before sleeping and the shared memory updates performed before
signalling or broadcasting. This separation of concerns allows for
enormous flexibility - you could use CVs to implement an LWLock-like
construct with an arbitrary set of lock modes and an arbitrary
conflict matrix, for example. But I think that in most cases it will
be best to leave mutual exclusion to LWLocks, and use CVs when what we
need is not mutual exclusion but rather waiting for an operation
performed by some other backend to complete (successfully or
otherwise). The IO-in-progress example is just such a case: the
current spinning behavior arises from the fact that the buffer's
I/O-in-progress flag does not get cleared until AFTER processes has
been dropped off the wait queue. The normal pattern with a CV will be
(1) perform a shared memory update that will, if seen by other
processes, confirm that the condition has been met and then (2)
broadcast on the CV. There's no way for transaction abort to know
what (1) looks like, even if somehow were made aware of the CV so that
it could do (2).

> I encourage you to pursue this, because indeed LWLocks aren't always
> an ideal solution, but I think it requires some careful thought about
> what transaction aborts will do with condition variables.

Thanks. As stated above, I believe the right answer is in fact
"nothing", because this facility is too low-level to permit a
categorical answer.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company