RE: Cache relation sizes?

From: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
> On Mon, Nov 16, 2020 at 11:01 PM Konstantin Knizhnik
> <k(dot)knizhnik(at)postgrespro(dot)ru> wrote:
> > I will look at your implementation more precisely latter.
>
> Thanks! Warning: I thought about making a thing like this for a
> while, but the patch itself is only a one-day prototype, so I am sure
> you can find many bugs... Hmm, I guess the smgrnblocks_fast() code
> really needs a nice big comment to explain the theory about why this
> value is fresh enough, based on earlier ideas about implicit memory
> barriers. (Something like: if you're extending, you must have
> acquired the extension lock; if you're scanning you must have acquired
> a snapshot that only needs to see blocks that existed at that time and
> concurrent truncations are impossible; I'm wondering about special
> snapshots like VACUUM, and whether this is too relaxed for that, or if
> it's covered by existing horizon-based interlocking...).

We'd like to join the discussion and review, too, so that Kirk's optimization for VACUUM truncation and TRUNCATRE can extend beyond recovery to operations during normal operation. In that regard, would you mind becoming a committer for his patch? We think it's committable now. I'm fine with using the unmodified smgrnblocks() as your devil's suggestion.

> It's a good question. I don't know exactly how to make a shared
> relation cache (I have only some vague ideas and read some of
> Idehira-san's work) but I agree that it would be very nice. However,
> I think that's an independent and higher level thing, because Relation
> != SMgrRelation.

> Sure, we can talk about how to make a shared relation cache (and
> syscache etc). I think it'll be much harder than this shared SMGR
> concept. Even if we figure out how to share palloc'd trees of nodes
> with correct invalidation and interlocking (ie like Ideriha-san's work
> in [1]), including good performance and space management/replacement,
> I guess we'll still need per-backend Relation objects to point to the
> per-backend SMgrRelation objects to hold the per-backend file
> descriptors. (Until we have threads...). But... do you think sharing
> SMGR relations to the maximum extent possible conflicts with that?
> Are you thinking there should be some general component here, perhaps
> a generic system for pools of shmem objects with mapping tables and a
> replacement policy?

Yes, Ideriha-san and we have completed the feature called global metacache for placing relation and catalog caches in shared memory and limiting their sizes, and incorporated it in our product to meet our customer's request. I hope we'll submit the patch in the near future when our resource permits.

Regards
Takayuki Tsunakawa