Re: Workarounds for getBinaryStream returning ByteArrayInputStream on bytea

On Thu, 25 Nov 2010 00:53:31 +0200, Александър Шопов
<lists(at)kambanaria(dot)org>
wrote:
> В 16:04 -0600 на 24.11.2010 (ср), Radosław Smogura написа:
>> I see only two possibilities
>> 1. Decrease fetch size, e.g. to 1.
> Even if I do, bytea is potentially 1GB. Plus peaks in usage can still
> smash the heap.
bytea is like varchar, and it's transmitted to client at all, even if you
don't want to read it, it is somewhere on heap.

> So refactoring to BLOBs is perhaps the only way out.
Here is other solution
http://archives.postgresql.org/pgsql-jdbc/2007-08/msg00078.php
You can write simple stream to do such reads on demand, select everything
without bytea column..., but probably blobs will be better.

> Will the JDBC driver always present bytea InputStream as
> ByteArrayInputStream? No plans to change that? (even if there are, I
> will still have to refactor meanwhile).
As above, the content is on heap, much more when you read you transform
this content and you creates new array (so heap 2x), maybe some chunked on
demand transformation will be better, but this is driver specific... or you
need to wait when I end binary JDBC and this will be in main release...

> Perhaps this behaviour should be better communicated to DB schema
> designers.
> It seems to me from the Npgsql2.0.11 readme.txt that reading in chunks
> is provided for .Net.
At a glance it looks that chunk reading means reading all network response
at all not bytea.

> Is there need to perhaps make patches for this in the jdbc driver?
> Kind regards:
> al_shopov

----------
Radosław Smogura
http://www.softperience.eu