Re: Global shared meta cache

Hi,

On 2018-07-05 10:00:13 -0400, Robert Haas wrote:
> I think we need to take a little bit broader view of this problem.
> For instance, maybe we could have backend-local caches that are kept
> relatively small, and then a larger shared cache that can hold more
> entries.

I think it's pretty much *required* that we have that. Not just for
speed, but for correctness. It'll otherwise be very hard to deal with
transactional DDL. I'm pretty sure that we'll have to have everything
modified by the local transaction in that cache. There's a lot of
interesting additional problems with snapshots stil, but that seems like
a baseline requirement.

> Obviously, loading an entry into our backend-private cache must be a
> LOT slower than consulting one.

I'm not actually sure it makes *that* much of a difference, if the local
cache is small.

> I would guess that we'd want to try to use something like the
> st_changecount protocol to make reads very cheap and writes
> comparatively more expensive, since by and large cache invalidations
> aren't that frequent.

That strikes me as hard to get right and fatal to get wrong. I suspect
normal RW locking ought to do fine.

> One experiment I think would be interesting is to study how much
> catcache traffic we're actually generating and see if we can find any
> way to reduce it. For instance, somebody could write code to record
> the file and line number for every catcache lookup and then run a test
> workload (or several test workloads). That might give us some ideas
> about stuff we could just decide not to cache, especially if we also
> knew how large each cache ended up being.

We definitely do a lot of redundant lookups for the same entries.

Greetings,

Andres Freund