| From: | Andres Freund <andres(at)anarazel(dot)de> |
|---|---|
| To: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
| Cc: | Fd Habash <fmhabash(at)gmail(dot)com>, "pgsql-performance(at)lists(dot)postgresql(dot)org" <pgsql-performance(at)lists(dot)postgresql(dot)org> |
| Subject: | Re: Select count(*) on a 2B Rows Tables Takes ~20 Hours |
| Date: | 2018-09-13 19:43:47 |
| Message-ID: | 20180913194347.sjnui3hajk7amcx6@alap3.anarazel.de |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-performance |
Hi,
On 2018-09-13 14:12:02 -0400, Tom Lane wrote:
> > This is an Aurora cluster running on r4.2xlarge (8 vCPU, 61g).
>
> Don't know much about Aurora, but I wonder whether you paid for
> guaranteed (provisioned) IOPS, and if so what service level.
Given that aurora uses direct-io and has the storage layer largely
completely replaced, I'm not sure how much we can help here. My
understanding is that access to blocks can require page-level "log
reconciliation", which can cause adverse IO patterns. The direct-IO
means that cache configuration / prefetching is much more crucial. If a
lot of those tuples aren't frozen (don't quite know how that works
there), the clog accesses will also kill you if the table was filled
over many transactions, since clog's access characteristics to a lot of
xids is pretty bad with DIO.
Greetings,
Andres Freund
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Fd Habash | 2018-09-13 19:49:41 | How Do You Associate a Query With its Invoking Procedure? |
| Previous Message | Fd Habash | 2018-09-13 19:35:23 | RE: Select count(*) on a 2B Rows Tables Takes ~20 Hours |