Re: ps.setCharacterStream() and memory usage

Sebastiaan van Erk wrote:

> ps.setCharacterStream(1, reader, (int) messageFile.length());
> ps.executeUpdate();

> I already noticed that the postgres driver does not stream the data to
> the backend (I don't know if postgres actually supports this).

Character streams are going to be hard to support as we need to know the
length, in bytes, of the parameter before we start sending it to the
backend. With a character stream the number of bytes is unpredictable
because it's being encoded into UTF8, which has a variable number of
bytes per character.

Binary streams we can stream, and the development driver already does so
without intermediate copies.

> First of all, in setCharacterStream() it loads the file into a char array.
> Then this is cloned into a String and passed to setString. (Here one can
> already cause the char[] to go out of scope, at least allowing it to be
> cleaned up). Secondly, setString causes another clone in escapeString(). [...]

I suggest you look at the current development driver before looking at
solutions in this area. The parameter storage mechanisms have changed
substantially compared to the stable driver. It should be much easier
to, for example, encode the stream directly into a bytearray, then write
from that array to the server on demand.

-O