Re: Non-blocking synchronization in libpq using pipeline mode

On Mon, 25 Mar 2024 15:47:00 +0100
Jan Behrens <jbe-mlist(at)magnetkern(dot)de> wrote:

> On Tue, 19 Mar 2024 12:49:23 +0100
> Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org> wrote:
>
> > On 2024-Mar-06, PG Doc comments form wrote:
> >
> > > [...]
> >
> > [...]
>
> [...]
>
> [...] the documentation [of PQsendFlushRequest] specifically states
> that:
>
> "Note that the request is not itself flushed to the server
> automatically; use PQflush if necessary."
>
> However, that sentence only applies to "PQsendFlushRequest", not to
> "PQpipelineSync". On the contrary, section "34.5.1.1. Issuing Queries"
> reads:
>
> "[Flushing to the server] occurs when PQpipelineSync is used to
> establish a synchronization point in the pipeline, or when PQflush is
> called."
>
> As I understand it, this means that PQpipelineSync will flush the
> client-side's output, while PQsendFlushRequest won't flush the
> client-side. So both commands act differently with regard to flushing,
> is that right?
>
> If that is the case, then the question is what happens if you call
> "PQpipelineSync" in non-blocking mode. That isn't clear to me (and I
> believe it is not explicitly made clear in the documentation).

I had a look into the source code of PostgreSQL 16.2 to answer that
question, and I found:

int
PQpipelineSync(PGconn *conn)
{
/* ... */

/*
* Give the data a push (in pipeline mode, only if we're past the size
* threshold). In nonblock mode, don't complain if we're unable to send
* it all; PQgetResult() will do any additional flushing needed.
*/
if (pqPipelineFlush(conn) < 0)
return 0;

return 1;
}

Moreover:

/*
* pqPipelineFlush
*
* In pipeline mode, data will be flushed only when the out buffer reaches the
* threshold value. In non-pipeline mode, it behaves as stock pqFlush.
*
* Returns 0 on success.
*/
static int
pqPipelineFlush(PGconn *conn)
{
if ((conn->pipelineStatus != PQ_PIPELINE_ON) ||
(conn->outCount >= OUTBUFFER_THRESHOLD))
return pqFlush(conn);
return 0;
}

It is not entirely clear to me what this means for the user of libpq.

It seems like invoking PQpipelineSync with pipeline mode switched on,
will *NOT* always flush the synchronization request out to the server.

While the comment on PQpipelineSync suggests that "PQgetResult() will
do any additional flushing needed", I believe that an event loop that
only waits for incoming data (e.g. POLLIN on the file descriptor) may
block indefinitely because a synchronization request hasn't been
flushed out yet.

If that is true, the documentation should clarify that PQflush might
need to be invoked manually by the user of libpq. Do I understand this
right?

>
> In section "34.4. Asynchronous Command Processing", we find:
>
> "In the nonblocking state, successful calls to PQsendQuery, PQputline,
> PQputnbytes, PQputCopyData, and PQendcopy will not block; their changes
> are stored in the local output buffer until they are flushed.
> Unsuccessful calls will return an error and must be retried."
>
> There is no reference to "PQpipelineSync" there. Does that mean that
> "PQsetnonblocking" has no effect on "PQpipelineSync"'s blocking
> behavior? If so, is it correct that "PQpipelineSync" always blocks when
> the socket is not ready for writing (since, as stated in the
> documentation, it does flush)?

Following the source code, PQpipelineSync will not block (but also
sometimes not flush, which is not documented, except in the source
code).

>
> I think section 34.5.1.4 should clarifiy how exactly PQpipelineSync
> acts in regard to flushing and blocking in that matter. It doesn't seem
> very clear at the moment.
>
> >
> > > 2. the synchronization or flush request need to be flushed manually with
> > > successive PQflush calls
> >
> > Yes.
>
> Regarding "PQpipelineSync", the documentation implies otherwise. See
> cite above:
>
> "[Flushing to the server] occurs when PQpipelineSync is used to
> establish a synchronization point in the pipeline, or when PQflush is
> called."
>
> That sentence doesn't make sense if PQpipelineSync wouldn't flush.

See above: PQpipelineSync seems to "sometimes" flush.

> [...]

Regards
Jan Behrens