Re: New replication mode: write

On Mon, Jan 23, 2012 at 9:53 PM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> On Mon, Jan 23, 2012 at 10:03 AM, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
>
>>>> To make the walreceiver call WaitLatchOrSocket(), we would need to
>>>> merge it and libpq_select() into one function. But the former is the backend
>>>> function and the latter is the frontend one. Now I have no good idea to
>>>> merge them cleanly.
>>>
>>> We can wait on the socket wherever it comes from. poll/select doesn't
>>> care how we got the socket.
>>>
>>> So we just need a common handler that calls either
>>> walreceiver/libpqwalreceiver function as required to handle the
>>> wakeup.
>>
>> I'm afraid I could not understand your idea. Could you explain it in
>> more detail?
>
> We either tell libpqwalreceiver about the latch, or we tell
> walreceiver about the socket used by libpqwalreceiver.
>
> In either case we share a pointer from one module to another.

The former seems difficult because it's not easy to link libpqwalreceiver.so
to the latch. I will consider about the latter.

>>>> If we send back the reply as soon as the Apply pointer is changed, I'm
>>>> afraid quite lots of reply messages are sent frequently, which might
>>>> cause performance problem. This is also one of the reasons why I didn't
>>>> implement the quick-response feature. To address this problem, we might
>>>> need to change the master so that it sends the Wait pointer to the standby,
>>>> and change the standby so that it replies whenever the Apply pointer
>>>> catches up with the Wait one. This can reduce the number of useless
>>>> reply from the standby about the Apply pointer.
>>>
>>> We send back one reply per incoming message. The incoming messages
>>> don't know request state and checking that has a cost which I don't
>>> think is an appropriate payment since we only need this info when the
>>> link goes quiet.
>>>
>>> When the link goes quiet we still need to send replies if we have
>>> apply mode, but we only need to send apply messages if the lsn has
>>> changed because of a commit. That will considerably reduce the
>>> messages sent so I don't see a problem.
>>
>> You mean to change the meaning of apply_location? Currently it indicates
>> the end + 1 of the last replayed WAL record, regardless of whether it's
>> a commit record or not. So too many replies can be sent per incoming
>> message because it might contain many WAL records. But you mean to
>> change apply_location only when a commit record is replayed?
>
> There is no change to the meaning of apply_location. The only change
> is that we send that message only when it has an updated value of
> committed lsn.

This means that apply_location might return the different location from
pg_last_xlog_replay_location() on the standby, though in 9.1 they return
the same. Which might confuse a user. No?

Regards,

--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center