Re: PATCH: Batch/pipelining support for libpq

From: Shay Rojansky <roji(at)roji(dot)org>
To: Craig Ringer <craig(at)2ndquadrant(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, Michael Paquier <michael(dot)paquier(at)gmail(dot)com>, Manuel Kniep <m(dot)kniep(at)web(dot)de>, "fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp" <fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp>
Subject: Re: PATCH: Batch/pipelining support for libpq
Date: 2016-10-14 14:15:55
Message-ID: CADT4RqBN-WkdgDLX3hAgsfGdJpqGv8BaZQknX7dyjmTT6jHgmg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

>
> > Of course, this is a
> > relatively minor performance issue (especially when compared to the
> overall
> > performance benefits provided by batching), and providing an API
> distinction
> > between adding a Sync and flushing the buffer may over-complicate the
> API. I
> > just thought I'd mention it.
>
> Unnecessary IMO. If we really want to add it later we'd probably do so
> by setting a "no flush on endbatch" mode and adding an explicit flush
> call. But I expect TCP_CORK will satisfy all realistic needs.
>

Unless I'm mistaken TCP_CORK is not necessarily going to work across all
platforms (e.g. Windows), although SO_LINGER (which is more standard) also
helps here.

> When it's about to send a batch, Npgsql knows whether it's in an
> (explicit)
> > transaction or not (by examining the transaction indicator on the last
> > ReadyForQuery message it received). If it's not in an (explicit)
> > transaction, it automatically inserts a Sync message after every
> Execute. If
> > some statement happens to be a BEGIN, it will be executed as a normal
> > statement and so on. The only issue is that if an error occurs after a
> > sneaked-in BEGIN, all subsequent statements will fail, and all have the
> Sync
> > messages Npgsql inserted. The end result will be a series of errors that
> > will be raised up to the user, but this isn't fundamentally different
> from
> > the case of a simple auto-commit batch containing multiple failures
> (because
> > of unique constraint violation or whatever) - multiple errors is
> something
> > that will have to be handled in any case.
>
> I'm a bit hesitant about how this will interact with multi-statements
> containing embedded BEGIN or COMMIT, where a single protocol message
> contains multiple ; delimited queries. But at least at this time of
> night I can't give a concrete problematic example.

Unless I'm mistaken, in the extended protocol you can't combine multiple ;
delimited queries in a single Parse - that's a feature supported only by
the Query message of the simple protocol. But then, if you're in the simple
protocol Sync doesn't play any role, does it? I mean, a Query message
behaves as though it's implicitly followed by a Sync message - an error in
a previous Query message doesn't cause later messages to be skipped...

Note that Npgsql only rarely uses the simple protocol for user messages.
Npgsql is a binary-only driver, and the simple protocol doesn't support
requesting binary results. So only the extended protocol is used, except
for some edge cases where it's possible to use the simple protocol for
efficiency - statements with no parameters and where the ExecuteNonQuery
API is used (i.e. the user won't access any results).

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Craig Ringer 2016-10-14 14:28:21 Re: PATCH: Batch/pipelining support for libpq
Previous Message Albe Laurenz 2016-10-14 14:14:51 Re: Add PGDLLEXPORT to PG_FUNCTION_INFO_V1