Re: Merge David and Goliath tables efficiently

From: nicolas paris <nicolas(dot)paris(at)riseup(dot)net>
To: pgsql-performance(at)lists(dot)postgresql(dot)org
Subject: Re: Merge David and Goliath tables efficiently
Date: 2023-06-19 15:45:06
Message-ID: 45316372426111f4b1dffa6de420804da3befa3d.camel@riseup.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

> But you wrote that in both cases the query is:

that was indeed yet another tipo, hope to do better in the future.

> I'm willing to continue to investigate, but only if you prepare a
> reproducer,

Thanks for your starter script. Please find attached 2 scripts which
now illustrates two troubles.

repro1.sql is a slight evolution of yours. When I play with david size
(as described in the comments) you will see plan going from nested loop
to sequential scan. Also note that the partition wise join is likely
working. This illustrate my initial problem: the sequential scan is not
going to work fine on my workload (david too large). How to suggest
postgres to use a nested loop here ? (suspect playing with costs should
help)

repro2.sql now I changed the table layout (similar to my setup) to
reproduce the partition wise join which does not triggers. I added a
partition column, and a unique index to be able to mimic a primary key.
Now partition wise (in my local docker vanilla postgres 15.3) does not
work. Eventually, if I do small batch, then the merge is working fast.
That's an other, unrelated problem.

> I suggest you do that directly, not through JDBC. Perhaps the JDBC
> connection pool does something funny (like connection pooling and
> resetting settings).

I can tell jdbc was working, and likely the reason might be in my
current table setup.

Attachment Content-Type Size
repro1.sql application/sql 17.9 KB
repro2.sql application/sql 18.7 KB

In response to

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message Benoit Tigeot 2023-06-19 16:30:12 Re: Forced to use UNION ALL when having multiple ANY operators and ORDER BY LIMIT
Previous Message Tomas Vondra 2023-06-19 13:13:39 Re: Merge David and Goliath tables efficiently