Re: BitmapHeapScan streaming read user and prelim refactoring

On Mon, Feb 10, 2025 at 1:11 PM Tomas Vondra <tomas(at)vondra(dot)me> wrote:
> Certainly for the "localized" regressions, and cases when bitmapheapscan
> would not be picked. The eic=1 case makes me a bit more nervous, because
> it's default and affects NVMe storage. Would be good to know why is
> that, or perhaps consider bumping up eic default. Not sure.

I'm relatively upset by the fact that effective_io_concurrency is
measured in units that are, AFAIUI, completely stupid. The original
idea was that you would set it to the number of spindles you have. But
from what I have heard, that didn't actually work: you needed to set
it to a significantly higher number. But in 2025, you probably don't
even have spindles any more, because you're probably on SSD or some
other modern storage medium rather than a rotating hard drive. And if
you do have spindles, do you know how many you have? Is that even a
meaningful concept?

I am not saying that it's this patch's job to replace
effective_io_concurrency with something better. I think adopting the
streaming read interface is pretty important, and if it works out that
we should also change the default effective_io_concurrency from 1 to 2
or 17 or 42715 in the same release, fine. But I think that in the
slightly longer term, it would be a really good idea for someone to
propose a more sensible model. It's hard to think of a clearer case of
a parameter being rendered meaningless by the march of technology. The
closest analogue that comes to mind is our sorting implementation used
to have a hard coded number of tape drives, when the underlying
implementation was a bunch of files all on the same filesystem, but
that wasn't a user-settable parameter.

--
Robert Haas
EDB: http://www.enterprisedb.com