From: | Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp> |
---|---|
To: | pgsql-hackers(at)postgresql(dot)org |
Subject: | Async execution of postgres_fdw. |
Date: | 2014-12-15 09:09:44 |
Message-ID: | 20141215.180944.242184148.horiguchi.kyotaro@lab.ntt.co.jp |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hello, this is the 2nd session of 'intoroducing parallelism using
postgres_fdw'.
The two patch attached are as following,
- 0001-Async-exec-of-postgres_fdw.patch
Main patch, which includes all functions.
- 0002-rename-PGConn-variable.patch
Renaming the variable conn for readability. No functional
effect.
* Outline of this patch
From some consideration after the previous discussion and
comments from others, I judged the original (WIP) patch was
overdone as the first step. So I reduced the patch to minimal
function. The new patch does the following,
- Wrapping PGconn by PgFdwConn in order to handle multiple scans
on one connection.
- The core async logic was added in fetch_more_data().
- Invoking remote commands asynchronously in ExecInitForeignScan.
- Canceling async invocation if any other foreign scans,
modifies, deletes use the same connection.
Cancellation is done by immediately fetching the return of
already-invoked acync command.
* Where this patch will be effective.
With upcoming inheritance-partition feature, this patch enables
stating and running foreign scans asynchronously. It will be more
effective for longer TAT or remote startup times, and larger
number of foreign servers. No negative performance effect on
other situations.
* Concerns about this patch.
- This breaks the assumption that scan starts at ExecForeignScan,
not ExecInitForeignScan, which might cause some problem.
- error reporting code in do_sql_command is quite ugly..
regards,
--
Kyotaro Horiguchi
NTT Open Source Software Center
Attachment | Content-Type | Size |
---|---|---|
0001-Async-exec-of-postgres_fdw.patch | text/x-patch | 26.3 KB |
0002-rename-PGConn-variable.patch | text/x-patch | 13.9 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Samrat Revagade | 2014-12-15 09:16:02 | Re: pg_rewind in contrib |
Previous Message | Petr Jelinek | 2014-12-15 08:41:36 | Re: tracking commit timestamps |