Re: [PROTOCOL TODO] Permit streaming of unknown-length lob/clob (bytea,text,etc)

On 12/01/2014 10:38 PM, David Fetter wrote:
> On Mon, Dec 01, 2014 at 02:55:22PM +0800, Craig Ringer wrote:
>> Hi all
>>
>> Currently the client must know the size of a large lob/clob field, like
>> a 'bytea' or 'text' field, in order to send it to the server. This can
>> force the client to buffer all the data before sending it to the server.
>
> Yes, this is not good.
>
>> It would be helpful if the v4 protocol permitted the client to specify
>> the field length as unknown / TBD, then stream data until an end marker
>> is read.
>
> What's wrong with specifying its length in advance instead? Are you
> thinking of a one or more use cases where it's both large and unknown?

I am - specifically, the JDBC setBlob(...) and setClob(...) APIs that
accept streams without a specified length:

https://docs.oracle.com/javase/7/docs/api/java/sql/PreparedStatement.html#setBlob(int,%20java.io.InputStream)

https://docs.oracle.com/javase/7/docs/api/java/sql/PreparedStatement.html#setClob(int,%20java.io.Reader)

There are variants that do take a length, so PgJDBC can (and now does)
implement the no-length variants by internally buffering the stream
until EOF. It'd be nice to get rid of that though.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services