Re: Better understanding checkpoint logs

On Wed, 2023-06-21 at 18:53 -0700, Wells Oliver wrote:
> Hi all. Trying to asses our WAL checkpoint during heavy ETL loads.
> Checkpoint timer is at 5min and it seems to be constantly checkpointing, maybe that's OK.
> We see an increased % of buffer percentages in the logs during the checkpoint. Here's a run:
>
> [...]
>
> I don't want to read too much into it

But that's how you learn...

> but I am trying to understand it better to consider any potential changes in max WAL sizes,
> or increasing our checkpoint timeouts if they are causing the ETL stuff to be slower than
> it should. Our wal_level is currently set to logical but we plan to change this to replica
> soon as it feels like overkill.

You should increase "max_wal_size" until most of your checkpoints are triggered by
timeout, once every 5 minutes. That will reduce the amount of WAL, because you will
have fewer full-page images, and you will avoid redundant buffer writes.

The only negative side effect is that recovery can take longer if you crash while
loading.

Yours,
Laurenz Albe