Re: Draft LIMIT pushdown to Append and MergeAppend patch

From: Michał Kłeczek <michal(at)kleczek(dot)org>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Draft LIMIT pushdown to Append and MergeAppend patch
Date: 2023-10-07 21:01:34
Message-ID: D3DA92EA-F85C-45A5-86B6-D9274D6EDC08@kleczek.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi All,

Attached is a second version of the patch.

The goal is to:
1. Apply LIMIT as early as possible - especially to apply LIMIT in partition scans
2. Enable LIMIT pushdown for FDW partitions.

Main idea of the patch is:

1. Wrap children of Append and MergeAppend paths in LimitPaths.
2. Let FDW extension handle limit pushdown

The changes are mainly in pathnode.c:
- Introduced a new function: pushdown_limit() used by planner instead of create_limit_node
- pushdown_limit handles MergeAppend, Append and ForeignScan nodes specially
- it falls back to create_limit_node for other path types

Changes in fdw:
- added a new FDW operation PushdownLimitNode
- this operation is called by pushdown_limit in pathnode.c

Changes in postgres_fdw.c
- Added stub implementation of PushdownLimitNode operation that delegates to create_limit_node wrapping original ForeignPath node

I am going to work on tests right now as (obviously) they are failing due to different plans.

As this is my first time I dig into the internals of Postgres I would be really grateful for friendly review and some directions - I am not sure it the approach is the right one.

The need for this is real: we are struggling with slow queries on partitioned tables - the business requirements are such that the only way to avoid index scans yielding many records is to apply LIMIT early and not execute partition scans at all if enough rows are produced.

Kind regards,
Michal

Attachment Content-Type Size
WIP-limit-pushdown.patch application/octet-stream 11.9 KB
unknown_filename text/plain 690 bytes

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Thomas Munro 2023-10-07 22:42:20 Re: CREATE DATABASE with filesystem cloning
Previous Message Erik Wienhold 2023-10-07 18:41:04 Re: Fix output of zero privileges in psql