Re: [9.3 bug] disk space in pg_xlog increases during archive recovery

On Fri, Aug 2, 2013 at 12:24 AM, MauMau <maumau307(at)gmail(dot)com> wrote:
> From: "Fujii Masao" <masao(dot)fujii(at)gmail(dot)com>
>
>>> However, isn't StandbyRequested true (= standby_mode set to on) to enable
>>> warm standby?
>>
>>
>> We can set up warm-standby by using pg_standby even if standby_mode = off.
>
>
> I see. However, I understand that pg_standby is a legacy feature, and the
> current way to setup a warm standby is to set standby=on and restore_command
> in recovery.conf. So I think it might not be necessary to support cascading
> replication with pg_standby, if supporting it would prevent better
> implementation of new features.
>
>
>
>>> I'm afraid people set max_wal_senders>0 and hot_standby=on
>>> even on the primary server to make the contents of postgresql.conf
>>> identical
>>> on both the primary and the standby for easier configuration. If so,
>>> normal
>>> archive recovery (PITR, not the standby recovery) would face the original
>>> problem -- unnecessary WAL accumulation in pg_xlog/. So I'm wonder if
>>> AllowCascadeReplication() is enough.
>>
>>
>> One idea is to add new GUC parameter like enable_cascade_replication
>> and then prevent WAL file from being kept in pg_xlog if that parameter is
>> off.
>> But we cannot apply such change into the already-released version 9.2.
>
>
> Yes, I thought the same, too. Adding a new GUC to enable cascading
> replication now would be a usability degradation. But I have no idea of any
> better way. It seems we need to take either v1 or v2 of the patch I sent.
> If we consider that we don't have to support cascading replication for a
> legacy pg_standby, v1 patch is definitely better because the standby server
> would not keep restored archive WAL in pg_xlog when cascading replication is
> not used. Otherwise, we have to take v2 patch.
>
> Could you choose either and commit it?

On second thought, probably we cannot remove the restored WAL files early
because they might be required for fast promotion which is new feature in 9.3.
In fast promotion, an end-of-recovery checkpoint is not executed. After the end
of recovery, normal online checkpoint starts. If the server crashes before such
an online checkpoint completes, the server needs to replay again all the WAL
files which it replayed before the promotion. Since this is the crash recovery,
all those WAL files need to exist in pg_xlog directory. So if we remove the
restored WAL file from pg_xlog early, such a crash recovery might fail.

So even if cascade replication is disabled, if standby_mode = on, i.e., fast
promotion can be performed, we cannot remove the restored WAL files early.

Regards,

--
Fujii Masao