Re: High memory retention by QueryExecutorImpl.pgStream.encoding.decoderArray

Hello,

It would be useful if you could try the latest driver as that may have
fixed the problem, alternatively it is also what we are developing against.

As you already have a heap dump can you shed any more light on what's going
on?

Regards
Dave Cramer

davec(at)postgresintl(dot)com
www.postgresintl.com

On Mon, 4 Mar 2019 at 11:01, Atul Deopujari <atuldeopujari(at)gmail(dot)com> wrote:

> Hello
> I am trying to figure out reasons for high memory utilization by
> PgConnection object. My application occasionally runs out of memory and the
> heap dump shows PgConnection as one of the largest consumers of memory.
>
> The heap dump shows PgConnection having a retained size of 640 MB.
> We use a connection pool of 8 connections. Each connection object is
> retaining about 80 MB. If I drill down into the PgConnection object, I see
> that QueryExecutorImpl taking almost all memory within which it is the
> pgStream.encoding.decoderArray taking all the memory.
>
> Not all connection objects are 80 MB all the time. Sometimes the heap dump
> also shows that some connection objects are as low as 3 MB in size.
>
> I would like to understand under what circumstances can
> pgStream.encoding.decoderArray retain 80MB. I have checked places in the
> code and have ensured that we are doing resultSet.close, stmt.close and
> conn.close wherever we are using them. We do have big objects in the
> database. I have tried to reproduce this problem using a simple program
> that queries tuples that are large text, some are even more than 500MB. I
> haven't been able to see such high memory usage of QueryExecutorImpl in my
> local setup.
>
> We use the following JDBC driver and the server
>
> - JDBC driver build number: 42.2.2
> - Server version: 10.4 (Ubuntu 10.4-2.pgdg14.04+1)
>
>
> Will appreciate if someone can throw light on the reasons for excessive
> object retention in QueryExecutorImpl.pgStream.encoding.decoderArray which
> can help me dig this problem further.
>
> Thanks,
> Atul
>