RE: Global shared meta cache

From: "ideriha(dot)takeshi(at)fujitsu(dot)com" <ideriha(dot)takeshi(at)fujitsu(dot)com>
To: 'Tomas Vondra' <tomas(dot)vondra(at)2ndquadrant(dot)com>, Michael Paquier <michael(at)paquier(dot)xyz>
Cc: 'Konstantin Knizhnik' <k(dot)knizhnik(at)postgrespro(dot)ru>, 'Amit Langote' <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, Robert Haas <robertmhaas(at)gmail(dot)com>, "andres(at)anarazel(dot)de" <andres(at)anarazel(dot)de>, 'Thomas Munro' <thomas(dot)munro(at)gmail(dot)com>
Subject: RE: Global shared meta cache
Date: 2020-02-05 04:50:32
Message-ID: OSAPR01MB1985DECF32668664BDF0ED62EA020@OSAPR01MB1985.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,
Thank you for handling it as CFM and sorry for waiting long time.

>-----Original Message-----
>From: Tomas Vondra [mailto:tomas(dot)vondra(at)2ndquadrant(dot)com]
>
>This patch was broken and waiting for author since early December, so I've marked it
>as returned with feedback. Feel free to resubmit an updated version to a future
>commitfest.

I couldn't make it until last January but now I resubmit this patch.
Previous version doesn't pass RT but now it passed.

Transaction consistency is controlled by invalidation messages
and location of cache.

As I mentioned before, for committed catalog tuple searching order is local memory
-> shared memory -> catalog file (either shared buffer or disk).
Local cache header has a pointer to global cache and we can use it.

But after system catalog is updated in transaction,
we need to make uncommitted-catalog-tuple cache from catalog file based on MVCC.
Searching for global cache should be skipped because global cache entry may be old
for that transaction.

So at the end of command local cache is invalidated as usual and in addition,
that process creates a dummy cache. This cache indicates that if found cache is only
this dummy one, we need to skip global search and search actual file.
Created uncommitted cache is located in local memory so that other process cannot find it.

- shared_memory_cotnext-v01.patch
Memory context for global system catalog cache and relation cache backed by DSA
- global_catalog_cache-v01.patch
For global system catalog

Things not included in this patch:
- sweep shared area logic
- global relation cache

Hope you have some comments!

Regards,
Takeshi Ideriha

Attachment Content-Type Size
global_catalog_cache-v01.patch application/octet-stream 106.5 KB
shared_memory_cotnext-v01.patch application/octet-stream 49.7 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Masahiko Sawada 2020-02-05 04:52:05 Re: base backup client as auxiliary backend process
Previous Message Dilip Kumar 2020-02-05 04:18:48 Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions