Re: Reducing overhead of frequent table locks

On Fri, May 13, 2011 at 4:16 PM, Noah Misch <noah(at)leadboat(dot)com> wrote:
> The key is putting a rapid hard stop to all fast-path lock acquisitions and
> then reconstructing a valid global picture of the affected lock table regions.
> Your 1024-way table of strong lock counts sounds promising.  (Offhand, I do
> think they would need to be counts, not just flags.)
>
> If I'm understanding correctly, your pseudocode would look roughly like this:
>
>        if (level >= ShareUpdateExclusiveLock)
>                ++strong_lock_counts[my_strong_lock_count_partition]
>                sfence
>                if (strong_lock_counts[my_strong_lock_count_partition] == 1)
>                        /* marker 1 */
>                        import_all_local_locks
>                normal_LockAcquireEx
>        else if (level <= RowExclusiveLock)
>                lfence
>                if (strong_lock_counts[my_strong_lock_count_partition] == 0)
>                        /* marker 2 */
>                        local_only
>                        /* marker 3 */
>                else
>                        normal_LockAcquireEx
>        else
>                normal_LockAcquireEx

I think ShareUpdateExclusiveLock should be treated as neither weak nor
strong. It certainly can't be treated as weak - i.e. use the fast
path - because it's self-conflicting. It could be treated as strong,
but since it doesn't conflict with any of the weak lock types, that
would only serve to prevent fast-path lock acquisitions that otherwise
could have succeeded. In particular, it would unnecessarily disable
fast-path lock acquisition for any relation being vacuumed, which
could be really ugly considering that one of the main workloads that
would benefit from something like this is the case where lots of
backends are fighting over a lock manager partition lock on a table
they all want to run read and/or modify. I think it's best for
ShareUpdateExclusiveLock to always use the regular lock-acquisition
path, but it need not worry about incrementing strong_lock_counts[] or
importing local locks in so doing.

Also, I think in the step just after marker one, we'd only import only
local locks whose lock tags were equal to the lock tag on which we
were attempting to acquire a strong lock. The downside of this whole
approach is that acquiring a strong lock becomes, at least
potentially, a lot slower, because you have to scan through the whole
backend array looking for fast-path locks to import (let's not use the
term "local lock", which is already in use within the lock manager
code). But maybe that can be optimized enough not to matter. After
all, if the lock manager scaled perfectly at high concurrency, we
wouldn't be thinking about this in the first place.

> At marker 1, we need to block until no code is running between markers two and
> three.  You could do that with a per-backend lock (LW_SHARED by the strong
> locker, LW_EXCLUSIVE by the backend).  That would probably still be a win over
> the current situation, but it would be nice to have something even cheaper.

I don't have a better idea than to use an LWLock. I have a patch
floating around to speed up our LWLock implementation, but I haven't
got a workload where the bottleneck is the actual speed of operation
of the LWLock rather than the fact that it's contended in the first
place. And the whole point of this would be to arrange things so that
the LWLocks are uncontended nearly all the time.

> Then you have the actual procedure for transfer of local locks to the global
> lock manager.  Using record locks in a file could work, but that's a system call
> per lock acquisition regardless of the presence of strong locks.  Is that cost
> sufficiently trivial?

No, I don't think we want to go into kernel space. When I spoke of
using a file, I was imagining it as an mmap'd region that one backend
could write to which, at need, another backend could mmap and grovel
through. Another (perhaps simpler) option would be to just put it in
shared memory. That doesn't give you as much flexibility in terms of
expanding the segment, but it would be reasonable to allow space for
only, dunno, say 32 locks per backend in shared memory; if you need
more than that, you flush them all to the main lock table and start
over. You could possibly even just make this a hack for the
particular special case where we're taking a relation lock on a
non-shared relation; then you'd need only 128 bytes for a 32-entry
array, plus the LWLock (I think the database ID is already visible in
shared memory).

> I wonder if, instead, we could signal all backends at
> marker 1 to dump the applicable parts of their local (memory) lock tables to
> files.  Or to another shared memory region, if that didn't mean statically
> allocating the largest possible required amount.  If we were willing to wait
> until all backends reach a CHECK_FOR_INTERRUPTS, they could instead make the
> global insertions directly.  That might yield a decent amount of bug swatting to
> fill in missing CHECK_FOR_INTERRUPTS, though.

I've thought about this; I believe it's unworkable. If one backend
goes into the tank (think: SIGSTOP, or blocking on I/O to an
unreadable disk sector) this could lead to cascading failure.

> Improvements in this area would also have good synergy with efforts to reduce
> the global impact of temporary table usage.  CREATE TEMP TABLE can be the
> major source of AccessExclusiveLock acquisitions.  However, with the strong
> lock indicator partitioned 1024 ways or more, that shouldn't be a deal killer.

If that particular case is a problem for you, it seems like optimizing
away the exclusive lock altogether might be possible. No other
backend should be able to touch the table until the transaction
commits anyhow, and at that point we're going to release the lock.
There are possibly some sticky wickets here but it seems at least
worth thinking about.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company