Re: WIP: WAL prefetch (another approach)

From: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
To: Julien Rouhaud <rjuju123(at)gmail(dot)com>
Cc: Ashutosh Sharma <ashu(dot)coek88(at)gmail(dot)com>, Daniel Gustafsson <daniel(at)yesql(dot)se>, Stephen Frost <sfrost(at)snowman(dot)net>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, David Steele <david(at)pgmasters(dot)net>, Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, Jakub Wartak <Jakub(dot)Wartak(at)tomtom(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: WIP: WAL prefetch (another approach)
Date: 2022-03-14 05:15:59
Message-ID: CA+hUKGKBxHZMVJFELZhC8iD57TBhboHtpmG+rzdOSdRGyxUGeQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, Mar 11, 2022 at 9:27 PM Julien Rouhaud <rjuju123(at)gmail(dot)com> wrote:
> > > Also, is it worth an assert (likely at the top of the function) for that?
> >
> > How could I assert that EndRecPtr has the right value?
>
> Sorry, I meant to assert that some value was assigned (!XLogRecPtrIsInvalid).
> It can only make sure that the first call is done after XLogBeginRead /
> XLogFindNextRecord, but that's better than nothing and consistent with the top
> comment.

Done.

> + * Returns XLREAD_WOULDBLOCK if he requested data can't be read without
> + * waiting. This can be returned only if the installed page_read callback
>
> typo: "the" requested data.

Fixed.

> Other than that it all looks good to me!

Thanks!

> > The other thing I need to change is that I should turn on
> > recovery_prefetch for platforms that support it (ie Linux and maybe
> > NetBSD only for now), in the tests. Right now you need to put
> > recovery_prefetch=on in a file and then run the tests with
> > "TEMP_CONFIG=path_to_that make -C src/test/recovery check" to
> > excercise much of 0002.
>
> +1 with Andres' idea to have a "try" setting.

Done. The default is still "off" for now, but in
027_stream_regress.pl I set it to "try".

I also fixed the compile failure with -DWAL_DEBUG, and checked that
output looks sane with wal_debug=on.

Attachment Content-Type Size
v23-0001-Add-circular-WAL-decoding-buffer-take-II.patch text/x-patch 52.5 KB
v23-0002-Prefetch-referenced-data-in-recovery-take-II.patch text/x-patch 72.5 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Bharath Rupireddy 2022-03-14 05:24:56 Re: add checkpoint stats of snapshot and mapping files of pg_logical dir
Previous Message Michael Paquier 2022-03-14 05:15:40 Re: add checkpoint stats of snapshot and mapping files of pg_logical dir