Re: libpq sendQuery -- getResult not returning until all queries complete

On Wed, Dec 22, 2010 at 9:18 AM, Merlin Moncure <mmoncure(at)gmail(dot)com> wrote:
> On Wed, Dec 22, 2010 at 10:07 AM, Kelly Burkhart
> <kelly(dot)burkhart(at)gmail(dot)com> wrote:
>> On Tue, Dec 21, 2010 at 6:11 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> Merlin Moncure <mmoncure(at)gmail(dot)com> writes:
>>>> On Tue, Dec 21, 2010 at 6:49 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>>>> If you just unconditionally flush there, it will result in an extra
>>>>> network message in the normal case where there's not another query
>>>>> to do.  The current code is designed not to flush until it sends
>>>>> ReadyForQuery.
>>>
>>>> yeah, I was looking at that.  I don't see an easy way to test if there
>>>> is another query waiting to execute right there.  Maybe a
>>>> documentation patch is in order :-).
>>>
>>> dest.c doesn't have the info available.  I think that to do this, we'd
>>> need to move the responsibility for calling pq_flush out to postgres.c.
>>> Not sure if it's worth it.
>>
>> So if I understand correctly, a flush will occur when all commands are
>> completed and prior to completion, whenever PqSendBuffer is full.
>> Analogous to stdio full vs. line buffering, this is full rather than
>> result buffering.
>>
>> It seems to me that is not quite optimal and 'result buffering' would
>> be better.  Did you come to the same conclusion but decide that it's
>> not 'better enough' to justify polluting postgres.c with a special
>> flush to satisfy this case?
>
> The basic issue is that multiple queries per libpq call is more or
> less a misfeature -- it's not worth refactoring the backend protocol
> handling to peek ahead to know if it needs to flush on command
> complete.  Note that you can't use parametrized statements when using
> this method, and use of parameterized statements should always be
> encouraged.
>
> Maybe there is another way to do what you are trying to do?

The real world thing I'm trying to accomplish: We have several GUI
applications that start up and must perform several queries before
displaying anything useful. When the app is (latency wise) far away
from the DB, startup time is noticeably slower. I was experimenting
with the multiple queries per libpq call (which doesn't seem like a
misfeature at all to me) to suggest to the owners of these apps a way
to realize faster startup.

In my particular case, sending multiple queries in one go will help
them regardless of weather they get the results back as they complete
or all at once. Current behavior is surprising though based on how
the documentation is written. I think I agree that 'result buffering'
is not better enough to justify significant work, on the other hand if
the fix is simple it would IMO be an improvement.

-K