CustomScan support on readfuncs.c

From: Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com>
To: Robert Haas <robertmhaas(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>
Cc: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: CustomScan support on readfuncs.c
Date: 2015-09-25 01:19:11
Message-ID: 9A28C8860F777E439AA12E8AEA7694F80114A8EE@BPXM15GP.gisp.nec.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

I tried to define two additional callbacks to support CustomScan
on readfuncs.c.

First of all, we need to pay attention how to treat output of
TextOutCustomScan when additional text output is generated.
Only custom-scan provider knows what shall be displayed, so
all we can do is invoke a new callback: TextReadCustomScan().
Because private fields shall be displayed next to the common
field of CustomScan, this callback is invoked once pg_strtok
pointer reached to the last field of CustomScan. Then, custom
scan provider allocates CustomScan or a structure embedding
CustomScan; only CSP knows exact size to be allocated.
It can fetch some tokens using pg_strtok and reconstruct its
private fields, but no need to restore the common field because
it shall be done by _readCustomScan.
If no callbacks are defined, _readCustomScan allocates
a CustomScan object and read only known fields.

Then, let's look back a bit. Next issue is how to reproduce
the "methods" pointer from the text representation.
I try to lookup the methods table using a pair of library
and symbol name; probably, it is a straightforward way.
The "methods" field is put earlier than all private fields
generated by TextOutCustomScan, so it should be reconstructable
prior to TextReadCustomScan.
To support this feature, I had to add an interface contract
that requires extensions to put library and symbol name on
CustomScanMethods table.
Although INIT_CUSTOM_SCAN_METHODS() macro can set up these
fields, author of extension needs to pay attention.

In addition to these enhancement, a new NodeCopyCustomScan
callback eliminates a restriction; custom-scan provider
cannot define its own structure that embeds CustomScan.
Only CSP knows exact size of the structure, this callback
is intended to allocate a new one and copy the private fields,
but no need to copy the common fields.

These three callbacks (one existing, two new) will make
CustomScan node copyObject, nodeToString and stringToNode
aware.

I also noticed that some static functions are valuable for
extensions, to avoid reinvention of same routines.
How about to expose these functions?
- _outToken
- _outBitmapset
- nullable_string
- _readBitmapset

The attached patch is conceptual, just compilable but not
tested yet. I'll try to make a test case soon, however,
it makes sense to get feedbacks earlier even if it is
based on the concept design.

Thanks,
--
NEC Business Creation Division / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>

Attachment Content-Type Size
custom-scan-on-readfuncs.v1.patch application/octet-stream 11.3 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message David Rowley 2015-09-25 01:39:05 Re: PATCH: use foreign keys to improve join estimates v1
Previous Message Robert Haas 2015-09-25 00:54:39 Re: DBT-3 with SF=20 got failed