Re: benchmarking effective_io_concurrency

On Mon, Jul 22, 2019 at 2:42 AM Fabio Pardi <f(dot)pardi(at)portavita(dot)eu> wrote:

> Hello,
>
>
> I recently spent a bit of time benchmarking effective_io_concurrency on
> Postgres.
>
> I would like to share my findings with you:
>
>
> https://portavita.github.io/2019-07-19-PostgreSQL_effective_io_concurrency_benchmarked/
>
> Comments are welcome.
>
> regards,
>
> fabio pardi
>

You didn't mention what type of disk storage you are using, or if that
matters. The number of cores in your database could also matter.

Does the max_parallel_workers setting have any influence on how
effective_io_concurrency works?

Based on your data, one should set effective_io_concurrency at the highest
possible setting with no ill effects with the possible exception that your
disk will get busier. Somehow I suspect that as you scale the number of
concurrent disk i/o tasks, other things may start to suffer. For example
does CPU wait time start to increase as more and more threads are consumed
waiting for i/o instead of doing other processing? Do you run into lock
contention on the i/o subsystem? (Back in the day, lock contention for
/dev/tcp was a major bottleneck for scaling busy webservers vertically. I
have no idea if modern linux kernels could run into the same issue waiting
for locks for /dev/sd0. Surely if anything was going to push that issue,
it would be setting effective_io_concurrency really high and then demanding
a lot of concurrent disk accesses.)