Re: Join push-down support for foreign tables

On Thu, Sep 4, 2014 at 9:33 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:

> On Thu, Sep 4, 2014 at 09:31:20PM +0530, Atri Sharma wrote:
> > I am thinking we would eventually have to cache the statistics, then
> get
> > some kind of invalidation message from the foreign server. I am also
> > thinking that cache would have to be global across all backends, I
> guess
> > similar to our invalidation cache.
> >
> >
> >
> >
> > That could lead to some bloat in storing statistics since we may have a
> lot of
> > tables for a lot of foreign servers. Also, will we have VACUUM look at
> > ANALYZING the foreign tables?
>
> > Also, how will we decide that the statistics are invalid? Will we have
> the FDW
> > query the foreign server and do some sort of comparison between the
> statistics
> > the foreign server has and the statistics we locally have? I am trying to
> > understand how the idea of invalidation message from foreign server will
> work.
>
> Well, ANALYZING is running on the foreign server, and somehow it would
> be nice if it would send a message to us about its new statistics, or we
> can do it like http does and it gives us a last-refresh statistics date
> when we connect.
>

Not sure how that would work without changing the way ANALYZE works on the
foreign server. http idea could work,though.

>
> I am not sure how it will work --- I am just suspecting that we might
> get to a point where the statistics lookup overhead on the foreign
> server might become a bottleneck.
>

Totally agree, but doing the planning only locally opens the questions I
mentioned above, and also deprives the foreign server database to do any
optimizations that it may want to do (assuming that the foreign database
and postgres query planner do not generate identical plans). This is only
my thought though, we could also be planning better than the foreign server
database, so the optimization part I raised is debatable.

Regards,

Atri

--
Regards,

Atri
*l'apprenant*