Re: High memory retention by QueryExecutorImpl.pgStream.encoding.decoderArray

I have created this issue https://github.com/pgjdbc/pgjdbc/issues/1431 and
have also submitted a PR https://github.com/pgjdbc/pgjdbc/pull/1433

--
Atul

On Tue, 5 Mar 2019 at 16:30, Dave Cramer <pg(at)fastcrypt(dot)com> wrote:

> Hello,
>
> It would be useful if you could try the latest driver as that may have
> fixed the problem, alternatively it is also what we are developing against.
>
> As you already have a heap dump can you shed any more light on what's
> going on?
>
> Regards
> Dave Cramer
>
> davec(at)postgresintl(dot)com
> www.postgresintl.com
>
>
> On Mon, 4 Mar 2019 at 11:01, Atul Deopujari <atuldeopujari(at)gmail(dot)com>
> wrote:
>
>> Hello
>> I am trying to figure out reasons for high memory utilization by
>> PgConnection object. My application occasionally runs out of memory and the
>> heap dump shows PgConnection as one of the largest consumers of memory.
>>
>> The heap dump shows PgConnection having a retained size of 640 MB.
>> We use a connection pool of 8 connections. Each connection object is
>> retaining about 80 MB. If I drill down into the PgConnection object, I see
>> that QueryExecutorImpl taking almost all memory within which it is the
>> pgStream.encoding.decoderArray taking all the memory.
>>
>> Not all connection objects are 80 MB all the time. Sometimes the heap
>> dump also shows that some connection objects are as low as 3 MB in size.
>>
>> I would like to understand under what circumstances can
>> pgStream.encoding.decoderArray retain 80MB. I have checked places in the
>> code and have ensured that we are doing resultSet.close, stmt.close and
>> conn.close wherever we are using them. We do have big objects in the
>> database. I have tried to reproduce this problem using a simple program
>> that queries tuples that are large text, some are even more than 500MB. I
>> haven't been able to see such high memory usage of QueryExecutorImpl in my
>> local setup.
>>
>> We use the following JDBC driver and the server
>>
>> - JDBC driver build number: 42.2.2
>> - Server version: 10.4 (Ubuntu 10.4-2.pgdg14.04+1)
>>
>>
>> Will appreciate if someone can throw light on the reasons for excessive
>> object retention in QueryExecutorImpl.pgStream.encoding.decoderArray which
>> can help me dig this problem further.
>>
>> Thanks,
>> Atul
>>
>