From: | Fabio Pardi <f(dot)pardi(at)portavita(dot)eu> |
---|---|
To: | Rick Otten <rottenwindfish(at)gmail(dot)com> |
Cc: | pgsql-performance(at)lists(dot)postgresql(dot)org |
Subject: | Re: benchmarking effective_io_concurrency |
Date: | 2019-07-22 12:28:12 |
Message-ID: | 27248f38-183f-e558-0b12-e8b03a365646@portavita.eu |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-performance |
Hi Rick,
thanks for your inputs.
On 22/07/2019 14:06, Rick Otten wrote:
>
>
>
> You didn't mention what type of disk storage you are using, or if that matters.
I actually mentioned I m using SSD, in RAID 10. Also is mentioned I tested in a no-RAID setup. Is that what you mean?
The number of cores in your database could also matter.
>
True, when scaling I think it can actually bring up problems as you mention here below. (BTW, Tested on a VM with 6 cores and on HW with 32. I updated the blogpost, thanks)
> Does the max_parallel_workers setting have any influence on how effective_io_concurrency works?
>
I m not sure about that one related to the tests I ran, because the query plan does not show parallelism.
> Based on your data, one should set effective_io_concurrency at the highest possible setting with no ill effects with the possible exception that your disk will get busier. Somehow I suspect that as you scale the number of concurrent disk i/o tasks, other things may start to suffer. For example does CPU wait time start to increase as more and more threads are consumed waiting for i/o instead of doing other processing? Do you run into lock contention on the i/o subsystem? (Back in the day, lock contention for /dev/tcp was a major bottleneck for scaling busy webservers vertically. I have no idea if modern linux kernels could run into the same issue waiting for locks for /dev/sd0. Surely if anything was going to push that issue, it would be setting effective_io_concurrency really high and then demanding a lot of concurrent disk accesses.)
>
>
>
My suggestion would be to try by your own and find out what works for you, maybe slowly increasing the value of effective_io_concurrency.
Every workload is peculiar, so I suspect there is no silver bullet here. Also the documentation gives you directions in that way...
regards,
fabio pardi
From | Date | Subject | |
---|---|---|---|
Next Message | Ken Tanzer | 2019-07-22 17:53:09 | Re: Speeding up query pulling comments from pg_catalog |
Previous Message | Rick Otten | 2019-07-22 12:06:09 | Re: benchmarking effective_io_concurrency |