Re: Transparent Data Encryption (TDE) and encrypted files

On Mon, 7 Oct 2019 at 18:02, Bruce Momjian <bruce(at)momjian(dot)us> wrote:

> Well, do to encryption properly, there is the requirement of the nonce.
> If you ever rewrite a bit, you technically have to have a new nonce.
> For WAL, since it is append-only, you can use the WAL file name. For
> heap/index files, we change the LSN on every rewrite (with
> wal_log_hints=on), and we never use the same LSN for writing multiple
> relations, so LSN+page-offset is a sufficient nonce.
>
> For clog, it is not append-only, and bytes are rewritten (from zero to
> non-zero), so there would have to be a new nonce for every clog file
> write to the file system. We can store the nonce in a separate file,
> but the clog contents and nonce would have to be always synchronized or
> the file could not be properly read. Basically every file we want to
> encrypt, needs this kind of study.
>

Yes. That is the reason why our current version doesn't encrypt SLRU's.
There is some security in encrypting without a nonce when considering an
attack vector that only sees one version of the encrypted page. But I think
to make headway on this we need to figure out if TDE feature is useful
withour SLRU encryption (I think yes), and how hard would it be to properly
encrypt SLRU's? Would the solution be acceptable for inclusion?

I can think of 3 options:

a) A separate nonce storage. Seems pretty bad complexity wise. New
data-structures would need to be created. SLRU writes would need to be WAL
logged with a full page image.
b) Inline nonces, number of items per SLRU page is variable depending on if
encryption is enabled or not.
c) Inline nonces we reserve a header structure on all SLRU pages.
pg_upgrade needs to rewrite persistent SLRUs.

None of the options seem great, but c) has the benefit of also carving out
the space for SLRU checksums.

> As I also said to Stephen, the people who are discussing this here
> > should *really really really* be looking at the Cybertec patch instead
> > of trying to invent everything from scratch - unless that patch has,
>
> Someone from Cybertec is on the voice calls we have, and is actively
> involved.
>

As far as I can tell no-one from us is on the call. I personally missed the
invitation when it was sent out. I would gladly share our learnings, a lot
of what I see here is retreading what we already went through with our
patch. However, I think that at the very least the conclusions, problems to
work on and WIP patch should be shared on list. It's hard for anybody
outside to have any input if there are no concrete design proposals or code
to review. Moreover, I think e-mail is a much better media for having a
reasoned discussion about technical design decisions.

> > In other words: maybe I'm wrong here, but it looks to me like we're
>
> laboriously reinventing the wheel when we could be working on
> > improving the working prototype.
>
> The work being done is building on that prototype.
>

We would like to help on that front.

Regards,
Ants Aasma
Web: https://www.cybertec-postgresql.com