Re: Online checksums patch - once again

> On 11 Feb 2021, at 14:10, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>
> On Wed, Feb 10, 2021 at 01:26:18PM -0500, Bruce Momjian wrote:
>> On Wed, Feb 10, 2021 at 03:25:58PM +0100, Magnus Hagander wrote:
>>> A fairly large amount of this complexity comes out of the fact that it
>>> now supports restarting and tracks checksums on a per-table basis. We
>>> skipped this in the original patch for exactly this reason (that's not
>>> to say there isn't a fair amount of complexity even without it, but it
>>> did substantially i increase both the size and the complexity of the
>>> patch), but in the review of that i was specifically asked for having
>>> that added. I personally don't think it's worth that complexity but at
>>> the time that seemed to be a pretty strong argument. So I'm not
>>> entirely sure how to move forward with that...
>>>
>>> is your impression that it would still be too complicated, even without that?
>>
>> I was wondering why this feature has stalled for so long --- now I know.
>> This does highlight the risk of implementing too many additions to a
>> feature. I am working against this dynamic in the cluster file
>> encryption feature I am working on.
>
> Oh, I think another reason this patchset has had problems is related to
> something I mentioned in 2018:
>
> https://www.postgresql.org/message-id/20180801163613.GA2267@momjian.us
>
> This patchset is weird because it is perhaps our first case of trying to
> change the state of the server while it is running. We just don't have
> an established protocol for how to orchestrate that, so we are limping
> along toward a solution. Forcing a restart is probably part of that
> primitive orchestration. We will probably have similar challenges if we
> ever allowed Postgres to change its data format on the fly. These
> challenges are one reason pg_upgrade only modifies the new cluster,
> never the old one.
>
> I don't think anyone has done anything wrong --- rather, it is what we
> are _trying_ to do that is complex.

Global state changes in a cluster are complicated, and are unlikely to never
not be. By writing patches to attempts such state changes we can see which
pieces of infrastructure we're lacking to reduce complexity. A good example is
the ProcSignalBarrier work that Andres and Robert did, inspired in part by this
patch IIUC. The more we do, the more we learn.

--
Daniel Gustafsson https://vmware.com/