Re: storing an explicit nonce

On Wed, May 26, 2021 at 4:40 PM Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> You are saying that by using a non-LSN nonce, you can write out the page
> with a new nonce, but the same LSN, and also discard the page during
> crash recovery and use the WAL copy?

I don't know what "discard the page during crash recovery and use the
WAL copy" means.

> I am confused why checksums, which are widely used, acceptably require
> wal_log_hints, but there is concern that file encryption, which is
> heavier, cannot acceptably require wal_log_hints. I must be missing
> something.

I explained this in the first complete paragraph of my first email
with this subject line: "For example, right now, we only need to WAL
log hints for the first write to each page after a checkpoint, but in
this approach, if the same page is written multiple times per
checkpoint cycle, we'd need to log hints every time." That's a huge
difference. Page eviction in some workloads can push the same pages
out of shared buffers every few seconds, whereas something that has to
be done once per checkpoint cycle cannot affect each page nearly so
often. A checkpoint is only going to occur every 5 minutes by default,
or more realistically every 10-15 minutes in a well-tuned production
system. In other words, we're not holding up some kind of double
standard, where the existing feature is allowed to depend on doing a
certain thing but your feature isn't allowed to depend on the same
thing. Your design depends on doing something which is potentially
100x+ more expensive than the existing thing. It's not always going to
be that expensive, but it can be.

> Why can't checksums also throw away hint bit changes like you want to do
> for file encryption and not require wal_log_hints?

Well, I don't want to throw away hint bit changes, just like we don't
throw them away right now. And I want to do that by making sure that
each time the page is written, we use a different nonce, but without
the expense of having to advance the LSN.

Now, another option is to do what you suggest here. We could say that
if a dirty page is evicted, but the page is only dirty because of
hint-type changes, we don't actually write it out. That does avoid
using the same nonce for multiple writes, because now there's only one
write. It also fixes the problem on standbys that Andres was
complaining about, because on a standby, the only way a page can
possibly be dirtied without an associated WAL record is through a
hint-type change. However, I think we'd find that this, too, is pretty
expensive in certain workloads. It's useful to write hint bits -
that's why we do it.

--
Robert Haas
EDB: http://www.enterprisedb.com