Re: Further reduction of bufmgr lock contention

Simon Riggs <simon(at)2ndquadrant(dot)com> writes:
> lmgr partitioning uses either 4 or 16, restricted by the hash function,
> for various reasons. I see no similar restriction on using a hash
> function here - we could equally well use range partitioning.

I don't really see any difference at all between the two cases as far as
what hashing we use.

The thing that's nagging at me at the moment is the realization that a
partitioned hashtable will eat more shared memory than a single
hashtable. It wasn't that long ago that we had to do some hacking to
ensure that the buffer hashtable couldn't run out of memory after
startup, and I'm afraid of re-introducing that failure mode. The lock
manager can run out of memory without crashing the system, but the
bufmgr can't (or at least could not in the recent past...)

Now that we're considering using partitioning methods for both the
buffer and lock hashtables, I wonder if there is any value in teaching
dynahash.c itself about concurrent access --- that is, find a way to use
a single shared hashtable instead of separate hashtables for the
different partitions, by having the hashtable itself deal with
concurrency to some extent. This is just handwaving at the moment...

regards, tom lane