Re: BitmapHeapScan streaming read user and prelim refactoring

On Thu, Mar 13, 2025 at 9:34 PM Melanie Plageman
<melanieplageman(at)gmail(dot)com> wrote:
> On Thu, Mar 13, 2025 at 5:46 AM Jakub Wartak
> <jakub(dot)wartak(at)enterprisedb(dot)com> wrote:
> >
> > Cool, anything > 1 is just better. Just quick question, so now we have:
> >
> > #define DEFAULT_EFFECTIVE_IO_CONCURRENCY 16
> > #define DEFAULT_MAINTENANCE_IO_CONCURRENCY 10
> >
> > Shouldn't maintenance be now also at the same value (16) instead of
> > 10? Also, fc34b0d9de27 (5 years ago by Thomas) states about m_io_c:
> > "Use the new setting in heapam.c instead of the hard-coded formula
> > effective_io_concurrency + 10 introduced by commit 558a9165e08.", so
> > there was always assumption that m_io_c > e_io_c (?) Now it's kind of
> > inconsistent to see bitmap heap scans pushing more IOPs than
> > recovery(!)/ANALYZE/VACUUM or am I missing something? No pressure to
> > change, just asking what Your thoughts are.
>
> Yea, I'm not really sure what if any relationship the two GUCs should
> have to one another.
> As long as they are in the same ballpark, since
> they are tuning for the same machine, I don't see any reason their
> values need to be adjusted together. However, if it is confusing for
> maintenance_io_concurrency default to be 10 while
> effective_io_concurrency default is 16, I'm happy to change it.

Hi Melanie,

dunno, I've just asked if it isn't suspicious to anyone except me else
that e_io_c > m_io_c rather than e_io_c <= m_io_c. My understanding
was always that to tune max IO queue depth you would do this:
a. up to N backends (up to max_connections; usually much lower) * e_io_c
b. autovacuum_max_workers * m_io_c
c. just one (standby/recovering) * m_io_c

The thing (for me) is: if we are allowing for much higher IOPS "a"
scenario, then why standby cannot use just the same (if not higher)
IOPS for prefetching in "c" scenario. After all, it is a much more
critical and sensitive thing (lag).

> I just don't quite know what I would write in the commit message. "it seemed confusing that they weren't the same?"

"To keep with the historical legacy of maintenance_io_concurrency
being traditionally higher than effective_io_concurrency ..."
OK, so right, nobody else commented on this, so maybe it's just me and
it was just a question after all.

-J.