RE: Online checksums verification in the backend

From: "Shinoda, Noriyoshi (PN Japan A&PS Delivery)" <noriyoshi(dot)shinoda(at)hpe(dot)com>
To: Michael Paquier <michael(at)paquier(dot)xyz>, Julien Rouhaud <rjuju123(at)gmail(dot)com>
Cc: Justin Pryzby <pryzby(at)telsasoft(dot)com>, Masahiko Sawada <masahiko(dot)sawada(at)2ndquadrant(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>
Subject: RE: Online checksums verification in the backend
Date: 2020-10-28 23:50:35
Message-ID: TU4PR8401MB11523D42C315AAF822E74275EE170@TU4PR8401MB1152.NAMPRD84.PROD.OUTLOOK.COM
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

I have tested this great feature in the latest commit environment on Red Hat Enterprise Linux 7.8. I modified a few blocks in a relation file to raise a checksum error. When I executed the pg_relation_check_pages function, the backend terminated abnormally. The attached file is the operation log.

Regards,
Noriyoshi Shinoda

-----Original Message-----
From: Michael Paquier [mailto:michael(at)paquier(dot)xyz]
Sent: Wednesday, October 28, 2020 2:09 PM
To: Julien Rouhaud <rjuju123(at)gmail(dot)com>
Cc: Justin Pryzby <pryzby(at)telsasoft(dot)com>; Masahiko Sawada <masahiko(dot)sawada(at)2ndquadrant(dot)com>; Robert Haas <robertmhaas(at)gmail(dot)com>; PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>; Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>
Subject: Re: Online checksums verification in the backend

On Tue, Oct 27, 2020 at 07:47:19PM +0800, Julien Rouhaud wrote:
> I think it's also worth noting that the IOLock is now acquired just
> before getting the buffer state, and released after the read (or after
> finding that the buffer is dirty). This is consistent with how it's
> done elsewhere, so I'm fine.

Consistency is the point. This API should be safe to use by design.
I have done some extra performance tests similar to what I did upthread, and this version showed similar numbers.

> Other than that I'm quite happy with the changes you made, thanks a lot!

Thanks for confirming. I have gone through the whole set today, splitted the thing into two commits and applied them. We had buildfarm member florican complain about a mistake in one of the
GetDatum() calls that I took care of already, and there is nothing else on my radar.
--
Michael

Attachment Content-Type Size
pg_relation_check_pages_error.txt text/plain 6.4 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message osumi.takamichi@fujitsu.com 2020-10-29 00:07:51 RE: Disable WAL logging to speed up data loading
Previous Message Tomas Vondra 2020-10-28 23:46:05 Re: Track statistics for streaming of in-progress transactions