Re: Some read stream improvements

From: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Some read stream improvements
Date: 2025-03-14 09:03:15
Message-ID: CA+hUKG+VoOgD92zZjRUC0+CgkaZ2Gqp4KkwVxy2LsoPW5HF4JA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I have pushed the new pin limit patches, after some more testing and
copy editing. I dropped an unnecessary hunk (in read_stream_reset(), a
change I'd like to make but it didn't belong in this commit) and
dropped the word "Soft" from GetSoftPinLimit() as it wasn't helping
comprehension and isn't even true for the local buffer version (which
I still think might be an issue, will come back to that, but again it
seemed independent).

For the record, here's a way to see 92fc6856^ or v17 misbehave and pin
too many buffers without involving any other proposed patches, only
v17 streaming seq scan: with shared_buffers=16MB, max_connections=4,
which gives MaxProportionalBuffers == 44, the attached shows three
cursors each pinning io_combine_limit = 32 buffers for a total of 96
at peak. That's because each cursor starts a stream and sees (the
only time it would check) that it is allowed 44, and then we fetch in
round-robin order until they all ramp up to full I/O size. In v17 we
can't really do much worse than that and it requires pretty contrived
settings and workload for it to be a real issue AFAIK but obviously
and hopefully we soon will eg BHS and more.

Attachment Content-Type Size
test.sql application/sql 1.9 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Jakub Wartak 2025-03-14 10:05:28 Re: Draft for basic NUMA observability
Previous Message Amit Kapila 2025-03-14 08:58:21 Re: Add an option to skip loading missing publication to avoid logical replication failure