pgsql: Support retrieval of results in chunks with libpq.

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: pgsql-committers(at)lists(dot)postgresql(dot)org
Subject: pgsql: Support retrieval of results in chunks with libpq.
Date: 2024-04-07 00:45:33
Message-ID: E1rtGf6-000t9x-SA@gemulon.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Support retrieval of results in chunks with libpq.

This patch generalizes libpq's existing single-row mode to allow
individual partial-result PGresults to contain up to N rows, rather
than always one row. This reduces malloc overhead compared to plain
single-row mode, and it is very useful for psql's FETCH_COUNT feature,
since otherwise we'd have to add code (and cycles) to either merge
single-row PGresults into a bigger one or teach psql's
results-printing logic to accept arrays of PGresults.

To avoid API breakage, PQsetSingleRowMode() remains the same, and we
add a new function PQsetChunkedRowsMode() to invoke the more general
case. Also, PGresults obtained the old way continue to carry the
PGRES_SINGLE_TUPLE status code, while if PQsetChunkedRowsMode() is
used then their status code is PGRES_TUPLES_CHUNK. The underlying
logic is the same either way, though.

Daniel Vérité, reviewed by Laurenz Albe and myself (and whacked
around a bit by me, so any remaining bugs are my fault)

Discussion: https://postgr.es/m/CAKZiRmxsVTkO928CM+-ADvsMyePmU3L9DQCa9NwqjvLPcEe5QA@mail.gmail.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/4643a2b265e967cc5f13ffa0c7c6912dbb3466d0

Modified Files
--------------
doc/src/sgml/libpq.sgml | 107 +++++++++++----
.../libpqwalreceiver/libpqwalreceiver.c | 3 +-
src/bin/pg_amcheck/pg_amcheck.c | 1 +
src/interfaces/libpq/exports.txt | 1 +
src/interfaces/libpq/fe-exec.c | 146 +++++++++++++--------
src/interfaces/libpq/fe-protocol3.c | 3 +-
src/interfaces/libpq/libpq-fe.h | 4 +-
src/interfaces/libpq/libpq-int.h | 10 +-
src/test/modules/libpq_pipeline/libpq_pipeline.c | 40 ++++++
.../modules/libpq_pipeline/traces/singlerow.trace | 14 ++
10 files changed, 243 insertions(+), 86 deletions(-)

Browse pgsql-committers by date

  From Date Subject
Next Message Tom Lane 2024-04-07 01:49:36 pgsql: Disable parallel query in psql error-with-FETCH_COUNT test.
Previous Message Tomas Vondra 2024-04-06 23:25:50 pgsql: Change BitmapAdjustPrefetchIterator to accept BlockNumber