Reasons for not overwriting processed wal archives?

From: Koen De Groote <kdg(dot)dev(at)gmail(dot)com>
To: PostgreSQL General <pgsql-general(at)lists(dot)postgresql(dot)org>
Subject: Reasons for not overwriting processed wal archives?
Date: 2022-05-18 21:08:17
Message-ID: CAGbX52EgQ2_3pSTDybkH2HTmn3pXJbEo1Gg6aC2cqqVJ2otJ=Q@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

The documentation here:
https://www.postgresql.org/docs/11/continuous-archiving.html

States:

> It is advisable to test your proposed archive command to ensure that it
indeed does not overwrite an existing file, *and that it returns nonzero
status in this case*.

Why exactly is this?

Assuming a situation in which the files, as part of the archive_command,
are gzipped to their final location.

If the gzip fails, or stops halfway, perhaps due to a network issue...
you're left with a file that isn't the full representation of the wal
archive.

So it needs to be done again.

In cases where the file is simply moved, this can also occur. Wal size can
be configured, and transfers might fail.

So they'd have to be done again.

If we test first for a file with that name being present, and refusing to
go forward if there is... an admin is going to have to manually intervene
and delete the half-finished file.

So isn't it better to just overwrite?

What scenario am I missing here?

Regards,
Koen

Responses

Browse pgsql-general by date

  From Date Subject
Next Message David G. Johnston 2022-05-18 21:18:16 Re: Reasons for not overwriting processed wal archives?
Previous Message Koen De Groote 2022-05-18 20:51:01 In case of network issues, how long before archive_command does retries