Re: PassDownLimitBound for ForeignScan/CustomScan

On Tue, Sep 13, 2016 at 3:48 AM, Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com> wrote:
> Sorry for my late.
>
> The attached patch fixed the wording problems on SGML part.

I agree that we should have some way for foreign data wrappers and
custom scans and perhaps also other executor nodes to find out whether
there's a known limit to the number of tuples that they might need to
produce, but I wonder if we should be doing something more general
than this. For example, suppose we add a new PlanState member "long
numTuples" where 0 means that the number of tuples that will be needed
is unknown (so that most node types need not initialize it), a
positive value is an upper bound on the number of tuples that will be
fetched, and -1 means that it is known for certain that we will need
all of the tuples. This might be relevant to the executor batching
stuff that Andres has been working on, because you could for example
set ps->numTuples == 1 on the inner side of a semi-join, warning the
executor node that it shouldn't bother trying to batch anything.

On a more practical level, I notice that you haven't adapted
postgres_fdw or file_fdw to benefit from this new callback. It seems
like postgres_fdw could benefit, because it could fetch only the
required number of tuples if that happens to be a smaller number than
the configured fetch_size.

Andres, anyone, thoughts?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company