From: | Andrei Lepikhov <a(dot)lepikhov(at)postgrespro(dot)ru> |
---|---|
To: | Richard Guo <guofenglinux(at)gmail(dot)com>, Alexander Pyhalov <a(dot)pyhalov(at)postgrespro(dot)ru> |
Cc: | Alexander Korotkov <aekorotkov(at)gmail(dot)com>, Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: Assert failure on 'list_member_ptr(rel->joininfo, restrictinfo)' |
Date: | 2023-12-11 02:40:55 |
Message-ID: | 04633c1e-e1a2-4d37-87f6-7a118dd6c9ea@postgrespro.ru |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On 11/12/2023 09:31, Richard Guo wrote:
> On Fri, Dec 8, 2023 at 3:13 PM Alexander Pyhalov
> <a(dot)pyhalov(at)postgrespro(dot)ru <mailto:a(dot)pyhalov(at)postgrespro(dot)ru>> wrote:
> Andrei Lepikhov писал(а) 2023-12-08 07:37:
> > I'd already clashed with Tom on copying the required_relids field
> and
> > voluntarily made unnecessary copies in the project [1].
> > And ... stuck into huge memory consumption. The reason was in
> > Bitmapsets:
> > When we have 1E3-1E4 partitions and try to reparameterize a join,
> one
> > bitmapset field can have a size of about 1kB. Having bitmapset
> > referencing Relation with a large index value, we had a lot of (for
> > example, 1E4 * 1kB) copies on each reparametrization of such a
> field.
> > Alexander Pyhalov should remember that case.
> Yes. If it matters, this happened during reparametrization when 2
> partitioned tables with 1000 partitions each were joined. Then
> asymmetric pw join managed to eat lots of memory for bitmapsets (by
> lots of memory I mean all available on the test VM).
> By reparametrization did you mean the work done in
> reparameterize_path_by_child()? If so maybe you'd be interested in the
> patch [1] which postpones reparameterization of paths until createplan.c
> and thus can help avoid unnecessary reparametrization work.
Yeah, I have discovered it already. It is a promising solution and only
needs a bit more review. But here, I embraced some corner cases with the
idea that we may not see other cases right now. And also, sometimes the
Bitmapset field is significant - it is not a corner case.
--
regards,
Andrei Lepikhov
Postgres Professional
From | Date | Subject | |
---|---|---|---|
Next Message | Richard Guo | 2023-12-11 03:02:24 | Re: Oversight in reparameterize_path_by_child leading to executor crash |
Previous Message | David Rowley | 2023-12-11 02:32:06 | Re: Postgres db Update to Version 15 |