Re: Temporary file access API

On Mon, Aug 8, 2022 at 08:27:27PM +0200, Antonin Houska wrote:
> > For instance, to go back to my earlier comments, if the data is
> > already block-structured, we do not need to insert a second layer of
> > buffering; if it isn't, maybe we should, not just for TDE but for
> > other reasons as well. If the data is temporary, TDE might want to
> > encrypt it using a temporary key which is generated on the fly and
> > only stored in server memory, but if it's data that needs to be
> > readable after a server restart, we're going to need to use a key that
> > we'll still have access to in the future. Or maybe that particular
> > thing isn't relevant, but I just can't believe that every single I/O
> > needs exactly the same treatment, so there have got to be some
> > patterns here that do matter. And I can't really tell whether this
> > patch set has a theory about that and, if so, what it is.
>
> I didn't want to use this API for relations (file reads/writes for these only
> affects a couple of places in md.c), nor for WAL. The intention was to use the
> API for temporary files, and also for other kinds of "auxiliary" files where
> the current coding is rather verbose but not too special.
>
> To enable the encryption, we'd need to add an extra argument (structure) to
> the appropriate function, which provides the necessary information on the
> encryption, such as cipher kind (block/stream), the encryption key or the
> nonce. I just don't understand whether the TDE alone (4) justifies patch like
> this, so I tried to make more use of it.

Yes, I assumed a parameter would be added to each call site to indicate
the type of key to be used for encryption, or no encryption. If these
are in performance-critical places, we can avoid the additional level of
calls by adding an 'if (tde)' check to only do the indirection when
necessary. We can also skip the indirection for files we know will not
need to be encrypted, e.g., postgresql.conf.

I can also imagine needing to write the data encrypted without
encrypting the in-memory version, so centralizing that code in one place
would be good.

--
Bruce Momjian <bruce(at)momjian(dot)us> https://momjian.us
EDB https://enterprisedb.com

Indecision is a decision. Inaction is an action. Mark Batterson