Resource cleanup callbacks for foreign data wrappers

From: Mikhail Bautin <mbautinpgsql(at)gmail(dot)com>
To: pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Resource cleanup callbacks for foreign data wrappers
Date: 2018-10-30 00:00:39
Message-ID: CA+7awWNPBd0vFqip0xeBtG+jXFjOGgdD2vcmb0udP-SP--34GQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello All,

We are using the foreign data wrapper API to integrate YugaByte DB (
https://github.com/yugabyte/yugabyte-db) with PostgreSQL, so scanning
tables stored in YugaByte DB is being done through Foreign Scan callbacks.
These callbacks in turn read data from the underlying YugaByte DB cluster
and maintain reference counts on RPC-related resources needed for that.
Currently we are using PostgreSQL 10.4 codebase for this integration. I am
trying to figure out what is the right way for foreign data wrappers to
ensure resources used in a query are freed in all cases. It looks like the
EndForeignScan callback is not being called in case an error happens during
expression evaluation, such as during this query from int2.sql:

SELECT '' AS five, i.f1, i.f1 * int2 '2' AS x FROM INT2_TBL i;
ERROR: smallint out of range

For the error-free case EndForeignScan appears on this codepath:

ExecEndForeignScan (postgres/src/backend/executor/nodeForeignscan.c:293)
ExecEndNode (postgres/src/backend/executor/execProcnode.c:649)
ExecEndPlan (postgres/src/backend/executor/execMain.c:1607)
standard_ExecutorEnd (postgres/src/backend/executor/execMain.c:495)
ExecutorEnd (postgres/src/backend/executor/execMain.c:466)
PortalCleanup (postgres/src/backend/commands/portalcmds.c:302)
PortalDrop (postgres/src/backend/utils/mmgr/portalmem.c:489)
exec_simple_query (postgres/src/backend/tcop/postgres.c:1118)
PostgresMain (postgres/src/backend/tcop/postgres.c:4126)
BackendRun (postgres/src/backend/postmaster/postmaster.c:4411)
BackendStartup (postgres/src/backend/postmaster/postmaster.c:4083)
ServerLoop (postgres/src/backend/postmaster/postmaster.c:1758)
PostmasterMain (postgres/src/backend/postmaster/postmaster.c:1366)
PostgresServerProcessMain (postgres/src/backend/main/main.c:228)

However, I see that ExecutorEnd is not being executed at all in
PortalCleanup in portalcmds.c in case portal->status is PORTAL_FAILED. If
this is by design, what would be the right callback mechanism for foreign
data wrappers to ensure resources are
freed?Would RegisterResourceReleaseCallback from resowner.h be more
appropriate for this use case? I only see that being used for
OpenSSL-related resources at the moment.

Thanks,
Mikhail

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tomas Vondra 2018-10-30 00:49:59 Re: shared-memory based stats collector
Previous Message Krzysztof Nienartowicz 2018-10-29 23:41:27 Re: Speeding up INSERTs and UPDATEs to partitioned tables