Re: Bug in libpq implentation and omission in documentation?

Ah ha. Yes, you're correct. It does mention here that an Int16 is used
to specify the number of parameter format codes, values etc.

I suggest then that the documentation is updated to reflect this? Anf
again, perhaps the 'int' for nParams should be an int16_t or short?

Naturally I have already modified our code to batch or chunk rather
large numbers of parameters to fit within this restriction but I do
think it'd help others if the API interface reflected the same size data
types as the restricted back ends ;)

Thanks Dmitriy,

Jim

On Wed, 2012-08-08 at 13:27 +0400, Dmitriy Igrishin wrote:
> Hey Jim,
>
> 2012/8/8 Jim Vanns <james(dot)vanns(at)framestore(dot)com>
> Hello PG hackers. Yesterday I began diagnosing a peculiar bug
> in some
> production code that has been happily running for months. I
> finally got
> to the bottom of it despite the rather misleading error
> message. Anyway,
> within a section of code we are making a DELETE call to the
> database via
> the libpq call PQexecParams(). It failed with this message:
>
> 'ERROR: bind message has 32015 parameter formats but 1
> parameters'
>
> This was just plain wrong. In fact, the # of parameters was
> more like
> 80,000. The area of code is quite clear. Despite this being a
> particularly large number of parameters (as you can imagine
> this query
> is built dynamically based on arbitrarily sized input) the
> data type for
> nParams for is a plain old 4-byte int. Upon further and deeper
> inspection I find that this 4 byte int is truncated to two
> bytes just
> before going down the wire.
>
> There is no mention of any restriction in the 9.1.4
> documentation:
>
> http://www.postgresql.org/docs/9.1/static/libpq-exec.html
>
> And the interface quite clearly accepts a 4 byte int however,
> the PQsendQueryGuts()
> function on line 1240 of src/interfaces/libpq/fq-exec.c just
> blatantly truncates the
> integer - it's calls pqPutInt() for nParams with a literal 2
> rather than 4. It does this
> several times, in fact.
>
> Unless I'm barking mad, surely this should either
>
> a) Be fixed and send 4 with nParams for pqPutInt() rather than
> 2
> b) Documented (and the type changed) as only being a 2 byte
> int
> and therefore having a restriction on the number of
> parameters
> permitted in PQexecParams().
>
> Could someone either verify or correct me before I submit an
> official bug report!?
>
> Regards,
>
> Jim Vanns
> AFAIK, this is a limitation of the frontend/backend protocol which
> allows
> you to bind no more than 2^16 parameters.
> See the description of the Bind (F) message here
> http://www.postgresql.org/docs/9.2/static/protocol-message-formats.html
>
>
> --
> // Dmitriy.
>
>

--
Jim Vanns
Systems Programmer
Framestore