Re: [9.3 bug] disk space in pg_xlog increases during archive recovery

On Wed, Aug 7, 2013 at 7:03 AM, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
> On Fri, Aug 2, 2013 at 12:24 AM, MauMau <maumau307(at)gmail(dot)com> wrote:
>> From: "Fujii Masao" <masao(dot)fujii(at)gmail(dot)com>
>>
>>>> However, isn't StandbyRequested true (= standby_mode set to on) to enable
>>>> warm standby?
>>>
>>>
>>> We can set up warm-standby by using pg_standby even if standby_mode = off.
>>
>>
>> I see. However, I understand that pg_standby is a legacy feature, and the
>> current way to setup a warm standby is to set standby=on and restore_command
>> in recovery.conf. So I think it might not be necessary to support cascading
>> replication with pg_standby, if supporting it would prevent better
>> implementation of new features.
>>
>>
>>
>>>> I'm afraid people set max_wal_senders>0 and hot_standby=on
>>>> even on the primary server to make the contents of postgresql.conf
>>>> identical
>>>> on both the primary and the standby for easier configuration. If so,
>>>> normal
>>>> archive recovery (PITR, not the standby recovery) would face the original
>>>> problem -- unnecessary WAL accumulation in pg_xlog/. So I'm wonder if
>>>> AllowCascadeReplication() is enough.
>>>
>>>
>>> One idea is to add new GUC parameter like enable_cascade_replication
>>> and then prevent WAL file from being kept in pg_xlog if that parameter is
>>> off.
>>> But we cannot apply such change into the already-released version 9.2.
>>
>>
>> Yes, I thought the same, too. Adding a new GUC to enable cascading
>> replication now would be a usability degradation. But I have no idea of any
>> better way. It seems we need to take either v1 or v2 of the patch I sent.
>> If we consider that we don't have to support cascading replication for a
>> legacy pg_standby, v1 patch is definitely better because the standby server
>> would not keep restored archive WAL in pg_xlog when cascading replication is
>> not used. Otherwise, we have to take v2 patch.
>>
>> Could you choose either and commit it?
>
> On second thought, probably we cannot remove the restored WAL files early
> because they might be required for fast promotion which is new feature in 9.3.
> In fast promotion, an end-of-recovery checkpoint is not executed. After the end
> of recovery, normal online checkpoint starts. If the server crashes before such
> an online checkpoint completes, the server needs to replay again all the WAL
> files which it replayed before the promotion. Since this is the crash recovery,
> all those WAL files need to exist in pg_xlog directory. So if we remove the
> restored WAL file from pg_xlog early, such a crash recovery might fail.
>
> So even if cascade replication is disabled, if standby_mode = on, i.e., fast
> promotion can be performed, we cannot remove the restored WAL files early.

But we should still be able to remove files older than the latest
restart point, right?

Cheers,

Jeff