Re: Transparent Data Encryption (TDE) and encrypted files

Robert Haas <robertmhaas(at)gmail(dot)com> wrote:

> On Mon, Oct 7, 2019 at 3:01 PM Antonin Houska <ah(at)cybertec(dot)at> wrote:
> > However the design doesn't seem to be stable enough at the
> > moment for coding to make sense.
>
> Well, I think the question is whether working further on your patch
> could produce some things that everyone would agree are a step
> forward.

It would have made a lot of sense several months ago (Masahiko Sawada actually
used parts of our patch in the previous version of his patch (see [1]), but
the requirement to use a different IV for each execution of the encryption
changes things quite a bit.

Besides the relation pages and SLRU (CLOG), which are already being discussed
elsewhere in the thread, let's consider other two file types:

* Temporary files (buffile.c): we derive the IV from PID of the process that
created the file + segment number + block within the segment. This
information does not change if you need to write the same block again. If
new IV should be used for each encryption run, we can simply introduce an
in-memory counter that generates the IV for each block. However it becomes
trickier if the temporary file is shared by multiple backends. I think it
might still be easier to expose the IV values to other backends via shared
memory than to store them on disk ...

* "Buffered transient file". This is to be used instead of OpenTransientFile()
if user needs the option to encrypt the file. (Our patch adds this API to
buffile.c. Currently we use it in reorderbuffer.c to encrypt the data
changes produced by logical decoding, but there should be more use cases.)

In this case we cannot keep the IVs in memory because user can close the
file anytime and open it much later. So we derive the IV by hashing the file
path. However if we should generate the IV again and again, we need to store
it on disk in another way, probably one IV value per block (PGAlignedBlock).

However since our implementation of both these file types shares some code,
it might yet be easier if the shared temporary file also stored the IV on
disk instead of exposing it via shared memory ...

Perhaps this is what I can work on, but I definitely need some feedback.

[1] https://www.postgresql.org/message-id/CAD21AoBjrbxvaMpTApX1cEsO=8N=nc2xVZPB0d9e-VjJ=YaRnw@mail.gmail.com

--
Antonin Houska
Web: https://www.cybertec-postgresql.com