Re: Logical decoding CPU-bound w/ large number of tables

Hi,

On 2017-05-05 20:59:09 -0400, Tom Lane wrote:
> Hmm ... as for RelfilenodeMapInvalidateCallback, the lack of calls to
> hash_search() from it in your trace says that it usually isn't doing
> anything useful. All the time is being spent in hash_seq_search,
> uselessly iterating over the hashtable. I'm inclined to think that
> we need a smarter data structure there, maybe an independent hashtable
> tracking the reverse map from relation OID to filenode map entry.

Yea, that might be worthwhile. Let me try to address the issue that we
do way too much invalidation, then we can check whether this is still
exercised hotly. On the other hand, it's still a dumb invalidation
approach, so if somebody feels like working on this...

> As for CatalogCacheIdInvalidate, I wonder how many of those cycles
> are doing something useful, and how many are being wasted in the outer
> loop that just iterates over the cache list. We could trivially get
> rid of that outer search by using syscache.c's array, as in the
> attached patch. It'd be interesting to see if this patch helps your
> scenario #1. (Patch is against HEAD but seems to apply cleanly to 9.5)

I've seen this be a significant fraction of CPU time completely
independent of logical decoding, so I'd guess this is worthwhile
independently. Not sure what a good benchmark for this would be.

Greetings,

Andres Freund