Re: storing an explicit nonce

On Wed, Oct 6, 2021 at 12:54:49PM -0400, Bruce Momjian wrote:
> On Wed, Oct 6, 2021 at 11:01:25AM -0400, Robert Haas wrote:
> > On Tue, Oct 5, 2021 at 4:29 PM Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> > > On Tue, Sep 28, 2021 at 12:30:02PM +0300, Ants Aasma wrote:
> > > > On Mon, 27 Sept 2021 at 23:34, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> > > > We are still working on our TDE patch. Right now the focus is on refactoring
> > > > temporary file access to make the TDE patch itself smaller. Reconsidering
> > > > encryption mode choices given concerns expressed is next. Currently a viable
> > > > option seems to be AES-XTS with LSN added into the IV. XTS doesn't have an
> > > > issue with predictable IV and isn't totally broken in case of IV reuse.
> > >
> > > Uh, yes, AES-XTS has benefits, but since it is a block cipher, previous
> > > 16-byte blocks affect later blocks, meaning that hint bit changes would
> > > also affect later blocks. I think this means we would need to write WAL
> > > full page images for hint bit changes to avoid torn pages. Right now
> > > hint bit (single bit) changes can be lost without causing torn pages.
> > > This was another of the advantages of using a stream cipher like CTR.
> >
> > This seems wrong to me. CTR requires that you not reuse the IV. If you
> > re-encrypt the page with a different IV, torn pages are a problem. If
> > you re-encrypt it with the same IV, then it's not secure any more.
>
> We were not changing the IV for hint bit changes, meaning the hint bit
> changes were visible if you compared the blocks.

Oops, I was wrong above, and my patch docs prove it:

Hint Bits
- - - - -

For hint bit changes, the LSN normally doesn't change, which is a
problem. By enabling wal_log_hints, you get full page writes to the WAL
after the first hint bit change of the checkpoint. This is useful for
two reasons. First, it generates a new LSN, which is needed for the IV
to be secure. Second, full page images protect against torn pages,
which is an even bigger requirement for encryption because the new LSN
is re-encrypting the entire page, not just the hint bit changes. You
can safely lose the hint bit changes, but you need to use the same LSN
to decrypt the entire page, so a torn page with an LSN change cannot be
decrypted. To prevent this, wal_log_hints guarantees that the
pre-hint-bit version (and previous LSN version) of the page is restored.

However, if a hint-bit-modified page is written to the file system
during a checkpoint, and there is a later hint bit change switching the
same page from clean to dirty during the same checkpoint, we need a new
LSN, and wal_log_hints doesn't give us a new LSN here. The fix for this
is to update the page LSN by writing a dummy WAL record via
xloginsert.c::LSNForEncryption() in such cases.

Seems my performance concerns were unfounded.

--
Bruce Momjian <bruce(at)momjian(dot)us> https://momjian.us
EDB https://enterprisedb.com

If only the physical world exists, free will is an illusion.