Re: BUG #17321: count(*) on a 1,874,554,883 rows partitioned table takes several minutes.

From: Andres Freund <andres(at)anarazel(dot)de>
To: deni_(at)hotmail(dot)fr, pgsql-bugs(at)lists(dot)postgresql(dot)org
Subject: Re: BUG #17321: count(*) on a 1,874,554,883 rows partitioned table takes several minutes.
Date: 2021-12-11 05:48:22
Message-ID: 20211211054822.az3d4gpsvtvifgnj@alap3.anarazel.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

Hi,

On 2021-12-06 08:57:41 +0000, PG Bug reporting form wrote:
> I am auditing a database which has a really huge partitioned table.
> When working on the whole table queries are slower than the cumulated time
> for all partitions, as you can see with the following query. I don't have
> yet timings for more complex operations like group by, but it looks like it
> is slower than it should be.

What makes you think this is a bug?

A lot of the time is spent doing IO:
worker 0 took 440s, of which 249s where spent waiting IO
worker 1 took 440s, of which 252s where spent waiting IO

As ~60% of the time is spent doing IO, optimizing that would be your best
bet. Each worker appears to do roughly ~190MB/s in IO. What kind of storage do
you have, and how fast do you expect it to be?

> " -> Parallel Seq Scan on
> schema.partitioned_table_y2017_07 partitioned_table_43
> (cost=0.00..1265058.13 rows=24648313 width=0) (actual time=0.526..28599.401
> rows=59154890 loops=1)"
> " Buffers: shared hit=64 read=1018511"
> " I/O Timings: read=20446.894"
> " Worker 1: actual time=0.526..28599.401
> rows=59154890 loops=1"
> " Buffers: shared hit=64 read=1018511"
> " I/O Timings: read=20446.894"

Here you have 24648313 rows in 1018511+64 pages, i.e. ~24 rows / page,
~340bytes/row with 8KiB pages. Do you expect rows to be fairly wide and/or
your tables to have a lot of bloat?

Can you show the table definition?

Greetings,

Andres Freund

In response to

Browse pgsql-bugs by date

  From Date Subject
Next Message Strahinja Kustudić 2021-12-11 14:40:43 Re: BUG #17330: EXPLAIN hangs and very long query plans
Previous Message Andres Freund 2021-12-11 04:58:26 Re: BUG #17255: Server crashes in index_delete_sort_cmp() due to race condition with vacuum