Re: New replication mode: write

On Mon, Jan 23, 2012 at 6:28 PM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> On Mon, Jan 23, 2012 at 9:02 AM, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
>> On Mon, Jan 23, 2012 at 4:58 PM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
>>> On Mon, Jan 16, 2012 at 12:45 PM, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
>>>
>>>>>> Please add the Apply mode.
>>>>>
>>>>> OK, will do.
>>>>
>>>> Done. Attached is the updated version of the patch.
>>>
>>> I notice that the Apply mode isn't fully implemented. I had in mind
>>> that you would add the latch required to respond more quickly when
>>> only the Apply pointer has changed.
>>>
>>> Is there a reason not to use WaitLatchOrSocket() in WALReceiver? Or
>>> was there another reason for not implementing that?
>>
>> I agree that the feature you pointed is useful for the Apply mode. But
>> I'm afraid that implementing that feature is not easy and would make
>> the patch big and complicated, so I didn't implement the Apply mode first.
>>
>> To make the walreceiver call WaitLatchOrSocket(), we would need to
>> merge it and libpq_select() into one function. But the former is the backend
>> function and the latter is the frontend one. Now I have no good idea to
>> merge them cleanly.
>
> We can wait on the socket wherever it comes from. poll/select doesn't
> care how we got the socket.
>
> So we just need a common handler that calls either
> walreceiver/libpqwalreceiver function as required to handle the
> wakeup.

I'm afraid I could not understand your idea. Could you explain it in
more detail?

>> If we send back the reply as soon as the Apply pointer is changed, I'm
>> afraid quite lots of reply messages are sent frequently, which might
>> cause performance problem. This is also one of the reasons why I didn't
>> implement the quick-response feature. To address this problem, we might
>> need to change the master so that it sends the Wait pointer to the standby,
>> and change the standby so that it replies whenever the Apply pointer
>> catches up with the Wait one. This can reduce the number of useless
>> reply from the standby about the Apply pointer.
>
> We send back one reply per incoming message. The incoming messages
> don't know request state and checking that has a cost which I don't
> think is an appropriate payment since we only need this info when the
> link goes quiet.
>
> When the link goes quiet we still need to send replies if we have
> apply mode, but we only need to send apply messages if the lsn has
> changed because of a commit. That will considerably reduce the
> messages sent so I don't see a problem.

You mean to change the meaning of apply_location? Currently it indicates
the end + 1 of the last replayed WAL record, regardless of whether it's
a commit record or not. So too many replies can be sent per incoming
message because it might contain many WAL records. But you mean to
change apply_location only when a commit record is replayed?

Regards,

--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center