Re: BGWorkers, shared memory pointers, and postmaster restart

On 04/16/2014 02:37 PM, Craig Ringer wrote:
> Hi all
>
> I've been using the dynamic BGWorker support for some recent work, and I
> think I've found an issue with how postmaster restarts are handled.
>
> TL;DR: I don't think there's a safe way to use a BGWorker (static or
> dynamic) with bgw_restart_time != BGW_NEVER_RESTART and a bgw_main_arg
> Datum that points into shared memory, and think we might need a API
> change to fix that.

Andres sensibly points out that part of this is easily solved by passing
the bgworker an index into an array in a named shmem block. A simple
pass-by-value Datum that can be turned into a pointer to a shmem struct.

This still doesn't solve the other half of the issue, which is how to
handle dynamic bgworkers after a postmaster restart. They're effectively
lost/leaked: there's no way to retain a bgworker handle across restart,
and no way to list bgworkers, nor is there any idempotent way to say
"Start a worker to do <x> only if it doesn't already exist" (unique
names, magic cookie hashes, whatever).

With the current API the only solution to the second half that I see is
to have bgworkers run in non-restart mode and manage them yourself. Not
ideal.

Instead we need one of:

- A flag like BGW_UNREGISTER_ON_RESTART;

- To always unregister dynamic bgws on postmaster shm clear + restart;

- A way to list bgws, inspect their BackgroundWorker structs and obtain
their handles; or

- A way to idempotently register a bgw only if it doesn't already exist

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services