Re: better page-level checksums

On Tue, Jun 14, 2022 at 1:22 PM Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> I still am not clear on precisely what you are proposing here. I do
> agree that there is significant bit space available in pd_flags and
> that consuming some of it wouldn't be stupid, but that doesn't add up
> to a proposal. Maybe the proposal is: figure out how many different
> configurations there are for this new kind of page space, let's say N,
> and then reserve ceil(log2(N)) bits from pd_flags to indicate which
> one we've got.

I'm just making a general point. Why wouldn't we start out with the
assumption that we use some pd_flags bit space for this stuff?

> One possible problem with this is that, if the page is actually
> encrypted, we might want pd_flags to also be encrypted. The existing
> contents of pd_flags disclose some information about the tuples that
> are on the page, so having them exposed to prying eyes does not seem
> appealing.

I'm skeptical of the idea that we want to avoid leaving any metadata
unencrypted. But I'm not an expert on TDE, and don't want to say too
much about it without having done some more research. I would like to
see some justification for just encrypting everything on the page
without concern for the loss of debuggability, though. What is the
underlying theory behind that particular decision? Are there any
examples that we can draw from, from other systems or published
designs?

Let's assume for now that we don't leave pd_flags unencrypted, as you
have suggested. We're still discussing new approaches to checksumming
in the scope of this work, which of course includes many individual
cases that don't involve any encryption. Plus even with encryption
there are things like defensive assertions that can be added by using
a flag bit for this.

--
Peter Geoghegan