From: | Ron <ronljohnsonjr(at)gmail(dot)com> |
---|---|
To: | pgsql-admin(at)lists(dot)postgresql(dot)org |
Subject: | Re: Large number of partitions of a table |
Date: | 2022-01-18 14:47:20 |
Message-ID: | d45899c8-187d-4a1d-563b-f5842243e770@gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-admin |
On 1/18/22 2:19 AM, Victor Sudakov wrote:
> Tom Lane wrote:
>> Victor Sudakov <vas(at)sibptus(dot)ru> writes:
>>> Tom Lane wrote:
>>>> Well, yeah, actually. An ill-advised query will blow out your backend's
>>>> memory consumption, potentially leading to a SIGKILL from the dreaded OOM
>>>> killer[1] (if you're on Linux), resulting in a backend crash and cluster
>>>> restart.
>>> Why should "SELECT COUNT(*) FROM t" ever consume more than work_mem
>>> even if t has 10000 partitions?
>> Sure, COUNT(*)'s runtime memory consumption is negligible.
>> But you're not thinking about overhead --- specifically,
>>
>> 1. 10000 relcache entries for the base tables.
>>
>> 2. If you have N indexes per table, N*10000 relcache entries for
>> the indexes. (The planner will probably have looked at all those
>> indexes, even if it didn't find any use for them.)
>>
>> 3. 10000 SeqScan plan nodes and associated rangetable entries,
>>
>> 4. Likewise, 10000 instances of executor per-node state.
>>
>> 5. 10000 lock table entries (both shared and local lock tables).
>>
>> 6. Probably a few per-relation things I didn't think of.
> I see your point about all that query-related stuff. I hope the
> testing of queries in a staging environment should help to detect such
> situations.
>
> What about the system catalogs however? Will the extra 10000
> tables and 500000 indexes negatively impact the performance of the
> system catalogs? Are there any caveats you could think of?
EXPLAIN plans are going to be hilariously gigantic, which means that query
planning would take a loooong time, And the query planner (in v12, at
least) can generate some pretty bad plans in partitioned tables; I bet there
are edge cases in the QP code that don't work well with 10000 partitions and
50000 indices.
--
Angular momentum makes the world go 'round.
From | Date | Subject | |
---|---|---|---|
Next Message | Mladen Gogala | 2022-01-19 00:09:55 | Re: Large number of partitions of a table |
Previous Message | Victor Sudakov | 2022-01-18 08:19:11 | Re: Large number of partitions of a table |