Re: pg_receivexlog and feedback message

On Thu, Jun 7, 2012 at 6:25 PM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> On Thursday, June 7, 2012, Fujii Masao wrote:
>>
>> On Thu, Jun 7, 2012 at 5:05 AM, Magnus Hagander <magnus(at)hagander(dot)net>
>> wrote:
>> > On Wed, Jun 6, 2012 at 8:26 PM, Fujii Masao <masao(dot)fujii(at)gmail(dot)com>
>> > wrote:
>> >> On Tue, Jun 5, 2012 at 11:44 PM, Magnus Hagander <magnus(at)hagander(dot)net>
>> >> wrote:
>> >>> On Tue, Jun 5, 2012 at 4:42 PM, Fujii Masao <masao(dot)fujii(at)gmail(dot)com>
>> >>> wrote:
>> >>>> On Tue, Jun 5, 2012 at 9:53 PM, Magnus Hagander <magnus(at)hagander(dot)net>
>> >>>> wrote:
>> >>>>> Right now, pg_receivexlog sets:
>> >>>>>                        replymsg->write = InvalidXLogRecPtr;
>> >>>>>                        replymsg->flush = InvalidXLogRecPtr;
>> >>>>>                        replymsg->apply = InvalidXLogRecPtr;
>> >>>>>
>> >>>>> when it sends it's status updates.
>> >>>>>
>> >>>>> I'm thinking it sohuld set replymsg->write = blockpos instad.
>> >>>>>
>> >>>>> Why? That way you can see in pg_stat_replication what has actually
>> >>>>> been received by pg_receivexlog - not just what we last sent. This
>> >>>>> can
>> >>>>> be useful in combination with an archive_command that can block WAL
>> >>>>> recycling until it has been saved to the standby. And it would be
>> >>>>> useful as a general monitoring thing as well.
>> >>>>>
>> >>>>> I think the original reason was that it shouldn't interefer with
>> >>>>> synchronous replication - but it does take away a fairly useful
>> >>>>> usecase...
>> >>>>
>> >>>> I think that not only replaymsg->write but also ->flush should be set
>> >>>> to
>> >>>> blockpos in pg_receivexlog. Which allows pg_receivexlog to behave
>> >>>> as synchronous standby, so we can write WAL to both local and remote
>> >>>> synchronously. I believe there are some use cases for synchronous
>> >>>> pg_receivexlog.
>> >>>
>> >>> pg_receivexlog doesn't currently fsync() after every write. It only
>> >>> fsync():s complete files. So we'd need to set ->flush only at the end
>> >>> of a segment, right?
>> >>
>> >> Yes.
>> >>
>> >> Currently the status update is sent for each status interval. In sync
>> >> replication, transaction has to wait for a while even after
>> >> pg_receivexlog
>> >> has written or flushed the WAL data.
>> >>
>> >> So we should add new option which specifies whether pg_receivexlog
>> >> sends the status packet back as soon as it writes or flushes the WAL
>> >> data, like the walreceiver does?
>> >
>> > That might be useful, but I think that's 9.3 material at this point.
>>
>> Fair enough. That's new feature rather than a bugfix.
>>
>> > But I think we can get the "set the write location" in as a bugfix.
>>
>> Also "set the flush location"? Sending the flush location back seems
>> helpful when using pg_receivexlog for WAL archiving purpose. By
>> seeing the flush location we can ensure that WAL file has been archived
>> durably (IOW, WAL file has been flushed in remote archive area).
>>
>
> You  can do that with the write location as well, as long as you round it
> off to complete segments, can't you?

You mean to prevent pg_receivexlog from sending back the end of WAL file
as the write location *before* it completes the WAL file? If so, yes. But
why do you want to keep the flush location invalid?

Regards,

--
Fujii Masao