Re: Streaming Replication patch for CommitFest 2009-09

On Thu, Sep 24, 2009 at 7:57 PM, Heikki Linnakangas
<heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
>> Anyway, I'll change walreceiver to retry connecting to the primary
>> after an error occurs in PQstartXLogStreaming()/PQgetXLogData()/
>> PQputXLogRecPtr(). Should we set an upper limit of the number of
>> the retries?
>
> I don't think we need an upper limit.

Without an upper limit, for example, mis-setting of the primary_conninfo
would make walreceiver repeat PQstartXLogStreaming() forever. Is this OK?

>>> - I know I said we should have just asynchronous replication at first,
>>> but looking ahead, how would you do synchronous?
>>
>> As the previous patch did, I'm going to make walsender read the latest
>> XLOG from wal_buffers, introduce the signaling between a backend
>> and walsender, and keep a backend waiting until the specified XLOG
>> has been written or fsynced in the standby.
>
> Ok. I don't think walsender needs to access wal_buffers even then,
> though. Once the backend has written the WAL, walsender can well read it
> from disk (it will surely be in OS cache still).

I think that walsender should not delay sending the XLOG until it has
been written by the backend, for performance improvement. Otherwise,
XLOG write and send are performed in serial, which would increase a
response time. Should those be performed in parallel?

>>> What kind of signaling
>>> is needed between walreceiver and startup process for that?
>>
>> I was thinking that the synchronization mode which a client waits
>> until XLOG has been applied is not necessary right now, so no
>> signaling is also not required between those processes yet. But,
>> HS requires this capability?
>
> Yeah, I think it will be important with hot standby. It's a much more
> useful guarantee that once COMMIT returns, the transactions is visible
> in the standby, than that it's merely fsync'd to disk in the standby.
>
> (don't need to solve it now, let's do just asynchronous mode now, but
> it's something to keep in mind)

Okey.

Regards,

--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center