Re: AIO v2.5

Hi,

On 2025-03-07 11:21:09 +0100, Jakub Wartak wrote:
> On Thu, Mar 6, 2025 at 2:13 PM Andres Freund <andres(at)anarazel(dot)de> wrote:
>
> > On 2025-03-06 12:36:43 +0100, Jakub Wartak wrote:
> > > On Tue, Mar 4, 2025 at 8:00 PM Andres Freund <andres(at)anarazel(dot)de> wrote:
> > > > Questions:
> > > >
> > > > - My current thinking is that we'd set io_method = worker initially - so we
> > > > actually get some coverage - and then decide whether to switch to
> > > > io_method=sync by default for 18 sometime around beta1/2. Does that sound
> > > > reasonable?
> > >
> > > IMHO, yes, good idea. Anyway final outcomes partially will depend on
> > > how many other stream-consumers be committed, right?
> >
> > I think it's more whether we find cases where it performs substantially worse
> > with the read stream users that exist. The behaviour for non-read-stream IO
> > shouldn't change.
>
> OK, so in order to to get full picture for v18beta this would mean
> $thread + following ones?:
> - Use read streams in autoprewarm
> - BitmapHeapScan table AM violation removal (and use streaming read API)

Yep.

> - Index Prefetching (it seems it has stalled?)

I don't think there's any chance it'll be in 18. There's a good bit more work
needed before it can go in...

> or is there something more planned? (I'm asking what to apply on top
> of AIO to minimize number of potential test runs which seem to take
> lots of time, so to do it all in one go)

I think there may be some more (e.g. btree index vacuuming), but I don't think
they'll have *that* big an impact.

> > > So, I've taken aio-2 branch from Your's github repo for a small ride
> > > on legacy RHEL 8.7 with dm-flakey to inject I/O errors. This is more a
> > > question: perhaps IO workers should auto-close fd on errors or should
> > > we use SIGUSR2 for it? The scenario is like this:
> >
> > When you say "auto-close", you mean that one IO error should trigger *all*
> > workers to close their FDs?
>
> Yeah I somehow was thinking about such a thing, but after You have
> bolded that "*all*", my question sounds much more stupid than it was
> yesterday. Sorry for asking stupid question :)

Don't worry about that :)

> > The same is already true with bgwriter, checkpointer etc?
>
> Yeah.. I was kind of looking for a way of getting "higher
> availability" in the presence of partial IO (tablespace) errors.

I'm really doubtful that's that worthwhile to pursue. IME the system is pretty
much hosed once this starts to happening and it's often made *worse* by trying
to limp along.

> OK the only question remains: does it make sense to try something like
> pgbench on NFS UDP mountopt=hard,nointr + intermittent iptables DROP
> from time to time , or is it not worth trying?

I don't think it's particularly interesting. But then I'd *never* trust any
meaningful data to a PG running on NFS.

Greetings,

Andres Freund