Re: Resource cleanup callbacks for foreign data wrappers

On Tue, Oct 30, 2018 at 11:01 AM Mikhail Bautin <mbautinpgsql(at)gmail(dot)com>
wrote:

> Hello All,
>
> We are using the foreign data wrapper API to integrate YugaByte DB (
> https://github.com/yugabyte/yugabyte-db) with PostgreSQL, so scanning
> tables stored in YugaByte DB is being done through Foreign Scan callbacks.
> These callbacks in turn read data from the underlying YugaByte DB cluster
> and maintain reference counts on RPC-related resources needed for that.
> Currently we are using PostgreSQL 10.4 codebase for this integration. I am
> trying to figure out what is the right way for foreign data wrappers to
> ensure resources used in a query are freed in all cases. It looks like the
> EndForeignScan callback is not being called in case an error happens during
> expression evaluation, such as during this query from int2.sql:
>
> SELECT '' AS five, i.f1, i.f1 * int2 '2' AS x FROM INT2_TBL i;
> ERROR: smallint out of range
>
> For the error-free case EndForeignScan appears on this codepath:
>
> ExecEndForeignScan (postgres/src/backend/executor/nodeForeignscan.c:293)
> ExecEndNode (postgres/src/backend/executor/execProcnode.c:649)
> ExecEndPlan (postgres/src/backend/executor/execMain.c:1607)
> standard_ExecutorEnd (postgres/src/backend/executor/execMain.c:495)
> ExecutorEnd (postgres/src/backend/executor/execMain.c:466)
> PortalCleanup (postgres/src/backend/commands/portalcmds.c:302)
> PortalDrop (postgres/src/backend/utils/mmgr/portalmem.c:489)
> exec_simple_query (postgres/src/backend/tcop/postgres.c:1118)
> PostgresMain (postgres/src/backend/tcop/postgres.c:4126)
> BackendRun (postgres/src/backend/postmaster/postmaster.c:4411)
> BackendStartup (postgres/src/backend/postmaster/postmaster.c:4083)
> ServerLoop (postgres/src/backend/postmaster/postmaster.c:1758)
> PostmasterMain (postgres/src/backend/postmaster/postmaster.c:1366)
> PostgresServerProcessMain (postgres/src/backend/main/main.c:228)
>
> However, I see that ExecutorEnd is not being executed at all in
> PortalCleanup in portalcmds.c in case portal->status is PORTAL_FAILED. If
> this is by design, what would be the right callback mechanism for foreign
> data wrappers to ensure resources are
> freed?Would RegisterResourceReleaseCallback from resowner.h be more
> appropriate for this use case? I only see that being used for
> OpenSSL-related resources at the moment.
>

Calling EndForeignScan in error scenario is not possible because of the
PostgreSQL design, as it aborts all the processing further do the cleanup.

You can register the transaction callback function and try to close the
remote
connection as it is done in postgres_fdw module GetConnection() function.

Instead of inventing new resource owner callback, transaction callback may
solve your problem.

Regards,
Haribabu Kommi
Fujitsu Australia