Re: [patch] libpq one-row-at-a-time API

On Sun, Jun 17, 2012 at 2:07 PM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> I prefer the description of Marko's API than the one we have now.
>
> Adopting one API in 9.2 and another in 9.3 would be fairly bad.
> Perhaps we can have both?

I see no reason the keep the (public) callback API around,
except if we don't bother to remove it now.

> Can we see a performance test? "Add a row processor API to libpq for
> better handling of large result sets". So idea is we do this many,
> many times so we need to double check the extra overhead is not a
> problem in cases where the dumping overhead is significant.

Not sure what do to want to performance test.

PQgetRowData() uses exactly the same pipeline
that callbacks used. It will use few more C calls,
not sure it make sense to benchmark them.

Recent dblink change did change palloc() + copy
zero-termination dance to PQgetResult(), which
does malloc() + copy dance internally. This malloc
vs. palloc might be benchmarkable, but it seems to
go into micro-benchmarking world as the big win came
from avoiding buffering rows. So yeah, maybe using
PQgetRowData() might be tiny bit faster, but I don't
expect much difference.

But all this affects new users only. The thing that affects
everybody was the 2-step row processing change
that was done during rowproc patch.

I did benchmark it, and it seems there are column-size
+ column-count patterns where new way is faster,
and some patterns where old way is faster. But the
difference did not raise above test noise so I concluded
it is insignificant and the malloc+copy avoidance is worth it.

Ofcourse, additional any benchmarking is welcome, so feel
free to pick any situation you care about.

--
marko