Re: introduce dynamic shared memory registry

On Mon, Dec 04, 2023 at 09:46:47PM -0600, Nathan Bossart wrote:
> The attached 0001 introduces a "DSM registry" to solve this problem. The
> API provides an easy way to allocate/initialize a segment or to attach to
> an existing one. The registry itself is just a dshash table that stores
> the handles keyed by a module-specified string. 0002 adds a test for the
> registry that demonstrates basic usage.
>
> I don't presently have any concrete plans to use this for anything, but I
> thought it might be useful for extensions for caching, etc. and wanted to
> see whether there was any interest in the feature.

Yes, tracking that in a more central way can have many usages, so your
patch sounds like a good idea. Note that we have one case in core
that be improved and make use of what you have here: autoprewarm.c.

The module supports the launch of dynamic workers but the library may
not be loaded with shared_preload_libraries, meaning that it can
allocate a chunk of shared memory worth a size of
AutoPrewarmSharedState without having requested it in a
shmem_request_hook. AutoPrewarmSharedState could be moved to a DSM
and tracked with the shared hash table introduced by the patch instead
of acquiring AddinShmemInitLock while eating the plate of other
facilities that asked for a chunk of shmem, leaving any conflict
handling to dsm_registry_table.

+dsm_registry_init_or_attach(const char *key, void **ptr, size_t size,
+ void (*init_callback) (void *ptr))

This is shaped around dshash_find_or_insert(), but it looks like you'd
want an equivalent for dshash_find(), as well.
--
Michael