Re: [patch] libpq one-row-at-a-time API

On 16 June 2012 23:09, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Marko Kreen <markokr(at)gmail(dot)com> writes:
>>> Now, looking at the problem with some perspective, the solution
>>> is obvious: when in single-row mode, the PQgetResult() must return
>>> proper PGresult for that single row.  And everything else follows that.
>>>
>>> * PQgetRowData(): can be called instead PQgetResult() to get raw row data
>>>  in buffer, for more efficient processing.  This is optional feature
>>>  that provides the original row-callback promise of avoiding unnecessary
>>>  row data copy.
>>>
>>> * Although PQgetRowData() makes callback API unnecessary, it is still
>>>  fully compatible with it - the callback should not see any difference
>>>  whether the resultset is processed in single-row mode or
>>>  old single-PGresult mode.  Unless it wants to - it can check
>>>  PGRES_TUPLES_OK vs. PGRES_SINGLE_TUPLE.
>
> I guess this raises the question of whether we ought to revert the
> row-callback patch entirely and support only this approach.  IMO
> it is (barely) not too late to do that for 9.2, if we want to.
> If we don't want to, then this is just another new feature and
> should be considered for 9.3.
>
> What I like about this is the greatly simpler and harder-to-misuse
> API.  The only arguable drawback is that there's still at least one
> malloc/free cycle per tuple, imposed by the creation of a PGresult
> for each one, whereas the callback approach avoids that.  But worrying
> about that could be considered to be vast overoptimization; the backend
> has certainly spent a lot more overhead than that generating the tuple.

I prefer the description of Marko's API than the one we have now.

Adopting one API in 9.2 and another in 9.3 would be fairly bad.
Perhaps we can have both?

Can we see a performance test? "Add a row processor API to libpq for
better handling of large result sets". So idea is we do this many,
many times so we need to double check the extra overhead is not a
problem in cases where the dumping overhead is significant.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services