Re: [PATCHES] Post-special page storage TDE support

Hi,

On 2023-11-13 14:37:47 -0600, David Christensen wrote:
> On Mon, Nov 13, 2023 at 2:27 PM Andres Freund <andres(at)anarazel(dot)de> wrote:
> > On 2023-11-08 18:47:56 -0800, Peter Geoghegan wrote:
> > > On Wed, Nov 8, 2023 at 6:04 AM Stephen Frost <sfrost(at)snowman(dot)net> wrote:
> > > > In conversations with folks (my memory specifically is a discussion
> > with
> > > > Peter G, added to CC, and my apologies to Peter if I'm misremembering)
> > > > there was a pretty strong push that a page should be able to 'stand
> > > > alone' and not depend on something else (eg: pg_control, or whatever)
> > to
> > > > provide info needed be able to interpret the page. For my part, I
> > don't
> > > > have a particularly strong feeling on that, but that's what lead to
> > this
> > > > design.
> > >
> > > The term that I have used in the past is "self-contained". Meaning
> > > capable of being decoded more or less as-is, without any metadata, by
> > > tools like pg_filedump.
> >
> > I'm not finding that very convincing - without cluster wide data, like
> > keys, a
> > tool like pg_filedump isn't going to be able to do much with encrypted
> > pages. Given the need to look at some global data, figuring out the offset
> > at
> > which data starts based on a value in pg_control isn't meaningfully worse
> > than
> > having the data on each page.
> >
> > Storing redundant data in each page header, when we've wanted space in the
> > page header for plenty other things, just doesn't seem a good use of said
> > space.
> >
>
> This scheme would open up space per page that would now be available for
> plenty of other things; the encoding in the header and the corresponding
> available space in the footer would seem to open up quite a few options
> now, no?

Sure, if you're willing to rewrite the whole cluster to upgrade and willing to
permanently sacrifice some data density. If the stored data is actually
specific to the page - that is the place to put the data. If not, then the
tradeoff is much more complicated IMO.

Of course this isn't a new problem - storing the page size on each page was
just silly, it's never going to change across the cluster and even more
definitely not going to change within a single relation.

Greetings,

Andres Freund