Re: Disk buffering of resultsets

Well, that's a very different approach now as a result of our discussion.

As of partial EXECUTE, it's point (3) in my plan. But it buys us only fast
cancel and I am really not sure I understand all the consequences (e.g.
transaction handling), so I don't think I will be doing it.

As of "next query fetching remaining results" and "thread pool", I've
incorporated it into my plan as point 2. Basically, there are next modes:
1) (And it's the only mode now until after p.2 is implemented) - swapping
to temp file is done on query execute time as before. The only change is
that it's not loading to memory, but copies to a temp file, so no OOMs
anymore.
2) Client-supplied executor
3) System-global cachedExecutor
4) Deferred swapping in the same thread much like in the first patch.
So, it's up for user to decide. The question is which mode (1,3 or 4)
should be default.

Best regards, Vitalii Tymchyshyn

2014-10-21 8:48 GMT-04:00 Craig Ringer <craig(at)2ndquadrant(dot)com>:

> On 10/21/2014 07:06 PM, Dave Cramer wrote:
> > Hi Vitalii,
> >
> > Thanks for this. I am headed for pgconf, hopefully I will have time to
> > look at it on the plane.
>
> Great that you're coming.
>
> I'm still concerned by this patch, particularly the fact that it doesn't
> properly use partial result set fetching at the protocol level, or
> didn't last time I looked.
>
> I really think it should be restricted to v3 protocol support and use
> partial EXECUTE.
>
> I'm generally opposed to the idea of the next query incurring the cost
> of fetching remaining results from the prior one, and possibly throwing
> an unrelated exception. That's IMO just wrong.
>
> I'm not at all convinced we shouldn't be using server-side portals, and
> possibly a client-supplied helper thread pool. I really don't like
> hijacking a possibly unrelated thread (that's trying to run the next
> statement) to do work deferred by a prior connection.
>
> Instead we should probably:
>
> WARNING: Discarding unfetched results from prior connection
> HINT: To avoid this warning, explicitly close() the result set or
> fully read it before running a new statement.
>
> or even make it an ERROR.
>
> --
> Craig Ringer http://www.2ndQuadrant.com/
> PostgreSQL Development, 24x7 Support, Training & Services
>
>
> --
> Sent via pgsql-jdbc mailing list (pgsql-jdbc(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-jdbc
>