Re: Separate memory contexts for relcache and catcache

Hi Melih, Jeff,

I tested the v4 patch along with the memory statistics reporting patch from
[1] thread.

After running a plpgsql procedure periodically and querying its memory
consumption,
I see various child contexts of CacheMemoryContext created as shown in the
image below or link [2].

[image: Memorycontext.drawio.png]
Observations:
1. While there are a number of child contexts like index info of
RelCacheContext,
CatCacheContext does not have any children.
2. While there is a bunch of used memory in RelCacheContext and
CatCacheContext,
SPICacheContext and PlanCacheContext do not have any allocations of their
own
and serve only as parents for SPI and CachedPlan related contexts
respectively.

Having reviewed the discussion regarding potential fragmentation issues
caused by
creating a large number of new contexts in each backend, I would like to
take a step
back and better understand the motivation behind separating these contexts.

IIUC, segregating cache memory allocations into RelCacheContext and
CatCacheContext
allows for grouping a large number of memory allocations under a
common context, which, in turn, aids in monitoring memory consumption.
However,
I believe this reasoning does not apply to SPICacheContext and
PlanCacheContext,
as these contexts do not have any allocations of their own.

How, then, does separating these contexts from CacheMemoryContext improve
monitoring?
Additionally, IIUC, these contexts are created as long-lived contexts, so
they are not designed
to optimize deletion of all their children via MemoryContextDelete on the
parent.

> Attached a separate patch to change initial sizes for relcache and
> catcache contexts as they grow
>

> large from the start. This was suggested in the thread previously [1].
> Also changed CacheMemoryContext to use ALLOCSET_START_SMALL_SIZES, so it
> starts from 1KB.
>

Applying the same change to use ALLOCSET_START_SMALL_SIZES would be
beneficial for
SPICacheContext and PlanCacheContext contexts as well.

On documentation front, the newly added contexts would require a mention in
src/backend/utils/mmgr/README.

[1] PostgreSQL: Enhancing Memory Context Statistics Reporting
<https://www.postgresql.org/message-id/flat/CAH2L28v8mc9HDt8QoSJ8TRmKau_8FM_HKS41NeO9-6ZAkuZKXw(at)mail(dot)gmail(dot)com>
[2] Memorycontext.png - Page-1
<https://viewer.diagrams.net/index.html?tags=%7B%7D&lightbox=1&highlight=0000ff&edit=_blank&layers=1&nav=1&title=Memorycontext.png#R%3Cmxfile%3E%3Cdiagram%20id%3D%22prtHgNgQTEPvFCAcTncT%22%20name%3D%22Page-1%22%3E7Ztbc9o4FMc%2FDY9kLF%2FhMSHptjNNhy17aZ92hBFYHWN5ZRFgP%2F1KtoSvgCE2mAwvrXWs6%2Fn%2Fjmwdk54xWm5%2BozD0XskM%2BT1dm216xnNP152Bzf8Vhm1iMG0nMSwoniUmkBom%2BD8kjZq0rvAMRbmKjBCf4TBvdEkQIJflbJBSss5XmxM%2FP2oIF6hkmLjQL1v%2FxjPmJdaB7qT2zwgvPDUysIfJnSVUleVKIg%2FOyDpjMl56xogSwpKr5WaEfOE75Zek3ac9d3cToyhgdRqQyVibPgZgYn3%2FFv6Fwx9%2FzmEfyG7eoL%2BSK5azZVvlAjTjHpFFQplHFiSA%2FktqfaJkFcyQGEfjpbTOV0JCbgTc%2BAsxtpXywhUj3OSxpS%2FvJmOKgfYuTpoisqIuOrAiOX8G6QKxA%2FWcnQQcXUSWiNEtb0eRDxl%2By88DSogWu3qpn%2FmFdPUJbtcrvG77fLpPUQiDnPvtf1eCkKc5CVg%2Fih34yCsAELLYcZqPA9T3JIPxLY2vzYpvqsb8aiH%2BH0HXQ69oSeh2xPtDG6bG5ctIhk4qliDIS7z2MEOTEMZKrHngV8n5higf4bCgZQFkA0OFjdw3bFlcp0EIlM3LBKCyNS6ZsVcyoUwtycxwU6nKH9swFqYsSdJ1RyQpKAKcCkn0S0pinh5FpwRLQd85dPN9fUb%2BG2LYhdmm78BgBFktDLoZsMDqGh5WexE79mFwEyE76Joo9l5RpkXCp%2FQcC59VqadQvCD6MIpyYr9G5BuhS%2BgfDf3zQSnP8zvyT4vysNRXM4463ZnvgRn6eBHwa5eTi2gzdFud23Kciz6RzsdyMv5yw48aE5gd033YrO5arOMcLrG%2FTW6OePxMKe7pI9EZDKJ%2BhCiet0tJhCB1vX9CcZLNbBGfMgPBpZA3mEZhnpcqjN49n5ASF0URDhY9oW67g7kiPm4sMOxh1zZElcY5mF8IZo8iUyOeDuIhjd28Y%2FJeRBvMfojrB0uWfsp64vp5k6n2vFWFgK8laWQaQ2X4GRs0y1aGtHFcyrUe83DjLhGPrtjYu3zGwqrWPqOtVSGtstVObMgRxgTHEaTO4wW0Ssgky5StsmmoQkemWegIFDpK%2FFDqKMZvt%2Bx3EFmVe2mISDAY1GeSF4pQZTi1cpQeIfQKMJp3GJuAsSqr1BCMORQfNPChcbTvODaBY1VGrRkcMzBqD451mMWUOzvLXV9TGN%2FA09q4E9kEkVVJvEaIBDkiawNp2KCIpGneCJLDayJpa9UfdE4l0nD0XD9ANy9LZFUG86onGuHyPJGWcWUirZpEgsGH2CUNxaDqyLrwLjmoYLKhPEicJ5yJDx5d%2FcgBCh85DPWMylA0PEBR80mP%2FenABuWYJEHYhihtJOsNs2MqqTxYKxu5Uzh9JXv0uacv2ypt8eBmtvirvnV8mBdhtfQ29pTJ%2BAu%2F3eUtvvjFZ4fHscS22druoZf0wNwlG9EumJOre8yyteMeAxf1WIu5rrNenK%2BV03Jq7pz7FL7vnKdx115S67a4s%2Btyp9%2B5a4K71lJX4itm4fXSOfn1skGwPsgrnuEMcx2Z2oWB2f%2FbuNq%2FCj9%2BbPx9hYQ3W%2FgBwyVOjTtNmj818mL6xxiJpulftBgv%2FwM%3D%3C%2Fdiagram%3E%3C%2Fmxfile%3E#%7B%22pageId%22%3A%22prtHgNgQTEPvFCAcTncT%22%7D>

Thank you,
Rahila Syed