Re: Re: BUG #13685: Archiving while idle every archive_timeout with wal_level hot_standby

On Sat, Jan 16, 2016 at 5:08 PM, Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
wrote:
>
> On Sat, Jan 16, 2016 at 7:10 PM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
wrote:
> > On Sun, Dec 20, 2015 at 6:44 PM, Michael Paquier <
michael(dot)paquier(at)gmail(dot)com>
> > wrote:
> >> On Sun, Nov 8, 2015 at 9:50 PM, Michael Paquier
> >> <michael(dot)paquier(at)gmail(dot)com> wrote:
> >> > On Sat, Nov 7, 2015 at 3:54 PM, Michael Paquier wrote:
> >> >> I thought about something like that at some point by saving a
minimum
> >> >> activity pointer in XLogCtl, updated each time a segment was
forcibly
> >> >> switched or after inserting a checkpoint record. Then the bgwriter
> >> >> looked at if the current insert position matched this minimum
activity
> >> >> pointer, skipping LogStandbySnapshot if both positions match. Does
> >> >> this match your line of thoughts?
> >> >
> >> > Looking at the code, it occurred to me that the LSN position saved
for
> >> > a XLOG_SWITCH record is the last position of current segment, so we
> >> > would still need to check if the current insert LSN matches the
> >> > beginning of a new segment and if the last segment was forcibly
> >> > switched by saving RecPtr of RequestXLogSwitch in XLogCtl for
example.
> >> > Thoughts?
> >>
> >> I haven't given up on this patch yet, and putting again my head on
> >> this problem I have finished with the patch attached, which checks if
> >> the current insert LSN position is at the beginning of a segment that
> >> has just been switched to decide if a standby snapshot should be
> >> logged or not. This allows bringing back an idle system to the pre-9.3
> >> state where a segment would be archived in the case of a low
> >> archive_timeout only when a checkpoint has been issued on the system.
> >>
> >
> > Won't this be a problem if the checkpoint occurs after a long time and
in
> > the mean time there is some activity in the server?
>
> Why? If there is some activity on the server, the snapshot will be
> immediately taken at the next iteration without caring about the
> checkpoint.
>

+ (insert_lsn % XLOG_SEG_SIZE) != SizeOfXLogLongPHD))

Do you mean to intend that it is protected by above check in the
patch?

Isn't it possible that so much WAL is inserted between bgwriter cycles,
that when it checks the location of WAL, it founds it to be at the beginning
of a new segment?

> > Another idea to solve this issue could be to see if there is any
progress
> > in the server by checking buffers dirtied/written (we can refer that
> > information using pgBufferUsage) since last time we log this record in
> > bgwriter.
>
> Yes, that may be an idea worth considering, but I really think that we
> had better measure that at WAL level..
>

I thought this is quite close to the previous patch you proposed where
Andres wanted some measurement in terms of progress since last
checkpoint. I understand as per current code your patch can work, but
what if some more similar WAL records needs to be added?

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com