From: | Dilip Kumar <dilipbalaut(at)gmail(dot)com> |
---|---|
To: | Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> |
Cc: | Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Andres Freund <andres(at)anarazel(dot)de>, Petr Jelinek <petr(dot)jelinek(at)enterprisedb(dot)com>, Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com> |
Subject: | Re: Decoding speculative insert with toast leaks memory |
Date: | 2021-05-27 04:05:37 |
Message-ID: | CAFiTN-vqDWqqFqQzox3SnE9aP4XT7iCiftO4tZU=jPiYzoWSGA@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Thu, May 27, 2021 at 9:03 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Thu, Mar 25, 2021 at 11:04 AM Ashutosh Bapat
> <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com> wrote:
> >
> > Hi All,
> > We saw OOM in a system where WAL sender consumed Gigabttes of memory
> > which was never released. Upon investigation, we found out that there
> > were many ReorderBufferToastHash memory contexts linked to
> > ReorderBuffer context, together consuming gigs of memory. They were
> > running INSERT ... ON CONFLICT .. among other things. A similar report
> > at [1]
> >
> ..
> >
> > but by then we might have reused the toast_hash and thus can not be
> > destroyed. But that isn't the problem since the reused toast_hash will
> > be destroyed eventually.
> >
> > It's only when the change next to speculative insert is something
> > other than INSERT/UPDATE/DELETE that we have to worry about a
> > speculative insert that was never confirmed. So may be for those
> > cases, we check whether specinsert != null and destroy toast_hash if
> > it exists.
> >
>
> Can we consider the possibility to destroy the toast_hash in
> ReorderBufferCleanupTXN/ReorderBufferTruncateTXN? It will delay the
> clean up of memory till the end of stream or txn but there won't be
> any memory leak.
Currently, we are ignoring XLH_DELETE_IS_SUPER, so maybe we can do
something based on this flag?
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
From | Date | Subject | |
---|---|---|---|
Next Message | Dilip Kumar | 2021-05-27 04:10:08 | Re: Decoding speculative insert with toast leaks memory |
Previous Message | Michael Paquier | 2021-05-27 04:04:38 | Re: Move pg_attribute.attcompression to earlier in struct for reduced size? |