Prevent restored WAL files from being archived again Re: Unnecessary WAL archiving after failover

On Tue, Jun 5, 2012 at 3:37 PM, Noah Misch <noah(at)leadboat(dot)com> wrote:
> On Fri, Mar 23, 2012 at 11:03:27PM +0900, Fujii Masao wrote:
>> (2) WAL files which were restored from the archive
>>
>> In 9.1 or before, the restored WAL files don't remain after failover
>> because they are always restored onto the temporary filename
>> "RECOVERYXLOG". So the issue which I explain from now doesn't exist
>> in 9.1 or before.
>>
>> In 9.2dev, as the result of supporting cascade replication,
>> an archived WAL file is restored onto correct file name so that
>> cascading walsender can send it to another standby. This restored
>
> The documentation still says this:
>
> WAL segments that cannot be found in the archive will be sought in pg_xlog/;
> this allows use of recent un-archived segments. However, segments that are
> available from the archive will be used in preference to files in
> pg_xlog/. The system will not overwrite the existing contents of pg_xlog/
> when retrieving archived files.
>
> I gather the last sentence is now false?

Yes. Attached patch removes that sentence.

>> WAL file has neither .ready nor .done archive status file. After
>> failover, checkpoint checks the archive status file of the restored
>> WAL file to attempt to recycle it, finds that it has neither .ready
>> nor ,done, and creates .ready. Because of existence of .ready,
>> it will be archived again even though it obviously already exists in
>> the archival storage :(
>>
>> To prevent a restored WAL file from being archived again, I think
>> that .done should be created whenever WAL file is successfully
>> restored (of course this should happen only when archive_mode is
>> enabled). Thought?
>
> Your proposed fix makes sense, and I cannot think of any disadvantage.
> Concerning only doing it when archive_mode=on, would there ever be a case
> where a segment is restored under archive_mode=off, then the server restarted
> with archive_mode=on and an archival attempted on that segment?

Yes, .done file should be created even if archive mode is not enabled.

Attached patch changes the startup process so that it creates .done file
whenever WAL file is successfully restored, whether archive mode is
enabled or not. The restored WAL files will not be archived again because
of .done file.

Regards,

--
Fujii Masao