From: | Chris Borckholder <chris(dot)borckholder(at)bitpanda(dot)com> |
---|---|
To: | pgsql-performance(at)lists(dot)postgresql(dot)org |
Subject: | Bad selectivity estimate when using a sub query to determine WHERE condition |
Date: | 2020-02-10 10:34:01 |
Message-ID: | CADPUTkS-5yxj-ts9WS1DcyJLm=1fcUs_Fq0sk3SaxY1FOiaJ=w@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-performance |
Hi,
I have a large table of immutable events that need to be aggregated
regularly to derive statistics. To improve the performance, that table is
rolled up every 15minutes, so that online checks can aggregate rolled up
data and combine it with latest events created after the last roll up.
To implement this a query is executed that selects only events after the
time of the last rollup.
That time is determined dynamically based on a log table.
When using a sub select or CTE to get the latest roll up time, the query
planner fails to recognize that a most of the large table would be filtered
out by the condition and tries a sequential scan instead of an index scan.
When using the literal value for the WHERE condition, the plan correctly
uses an index scan, which is much faster.
I analyzed the involved tables and increased the collected histogram, but
the query plan did not improve. Is there a way to help the query planner
recognize this in the dynamic case?
Best Regards
Chris
==== Original query with a CTE to get the timestamp to filter on
https://explain.depesz.com/s/Hsix
EXPLAIN (ANALYZE, BUFFERS) WITH current_rollup AS (
SELECT COALESCE(MAX(window_end), '-infinity') AS cutoff
FROM exchange.ledger_zerosum_rollup
)
SELECT *
FROM exchange.ledger
WHERE created > (SELECT cutoff FROM current_rollup);
==== Query with literal value
https://explain.depesz.com/s/ULAq
EXPLAIN (ANALYZE, BUFFERS)
SELECT *
FROM exchange.ledger
WHERE created > '2020-02-10T08:54:39.857789Z';
From | Date | Subject | |
---|---|---|---|
Next Message | Luís Roberto Weck | 2020-02-10 12:31:30 | Re: Fwd: TOAST table performance problem |
Previous Message | Asya Nevra Buyuksoy | 2020-02-10 07:52:21 | Fwd: TOAST table performance problem |